Better to fail for real than fail to really fail. Huh?
We know you’ve experienced this. Let’s say you just added some new functionality into your software, and you run a new build. And let’s say that 50% of your test cases fail. What is the first thing you assume?
We’ve asked this same question as our “teaser pitch” last winter to 100 developers and QA professionals who walked up to our booth at a recent conference, and 95 of them had the same answer! The tests must be broken!
This creates a cascading set of bad assumptions that will make your manager recite the adage about “ASS out of U and ME” on the whiteboard at the next project meeting. Here’s why.
* You assume that the problem is not with your application, it is with the test cases themselves being broken or no longer valid.
* So you spend time comparing the test cases with whatever changed in your new build.
* Then you dig into the test scripts to try to figure out why the test case is no longer passing, and rework them until they pass.
* Or you just give up and try validating by clicking through your old Word document test...