Question

(This one is coming from @chris_mcmahon on Twitter).

Background to this posting:

Answer 1

As far as I understand error-guessing and pairwise are not interchangeable but complete each other. You should use pairwise to reduce the number of tests after you chose the relevant independent inputs and relevant values.

The methods resemble lossy vs. non-lossy compression methods, you can use one of them but using both methods combined wisely gives the best results. For example video compression combines both of the compression methods.

Answer 2

I guess it all depends on what kind of fortune-teller you hire for your error guessing. :)

I never heard of a real technique called Error guessing. But the ISTQB defines Error Guessing as (via wikipedia):

Error Guessing is a Test Design technique where the tester has to guess what faults might occur and to disign the tests to represent them. Error Guessing comes with experience with the technology and the project. Error Guessing is the art of guessing where errors can be hidden. There are no specific tools and techniques for this, but you can write test cases depending on the situation: Either when reading the functional documents or when you are testing and find an error that you have not documented.

If this is what Chris has in mind, then you can't do a good comparison study:

• Different people with different experience will show different results.
• Different experiences with different parts of the software will show different results.
• Different times in the software life-cycle will show different results.
• At any given moment in the comarison study, one can argue that the tester is either too experienced or too inexperienced with either Error Guessing or Pairwise.

An observation, though:
As one advances in the project and learns more about the system, a tester may become better in Error Guessing, but the kind of bugs that were to be found with Pairwise aren't in he system anymore.
If Error Guessing is a real formal technique, then it looks like Pairwise is one of the tools used to gain the necessary knowledge/experience.

Answer 3

Even if you could design an effective study to determine which is better (see Shmuel's answer), and even if Error Guessing turned out to be the winner, I'm not sure the study's result would end up being of much value. What would be the actionable result of that study? Error Guessing doesn't seem like something you can point to as part of your long term test strategy, because of the initial investment required to get a resource who's good at it. Add employee turnover to the mix and things get worse. Compare with pairwise testing, which can be defined fairly explicitly, and extended and repeated with relative ease.

But, of course, and as alluded to by Shmuel and Rsf, a good tester's brain will always see those weird cases (and increasingly so with experience), so there's no reason to say "don't do Error Guessing!" I'd call Error Guessing more of a perk associated with keeping your testers on long term, rather than a defined methodology.