Earlier this year, I started work testing a system that had been developed over about 5 years and there were no documented test cases. There was a huge suite of automated tests, but no test cases whatsoever. I didn’t have time to write 5 years’ worth of test scripts for a system I knew nothing about, so I decided to try something radical – checklists instead of scripts. For an excellent description on the difference between checklists vs scripts, see Cem Kaner’s presentation on the value of checklists (an article I only found just now, thanks to Quick Testing Tips).
I have to say, I had a few doubts about the approach, and at first thought maybe it could only be a temporary solution, but I have to say it has proven to be quite effective and efficient.
Some benefits are:
- Lots of time saved upfront not needing to write test scripts
- Quick and easy maintenance – checklists take much less time to update than detailed scripts
- Checklists force me and other testers to think about how the system works, and find out how to do things if we don’t know how (the users might not know how either – how easy is this information to find?)
- Checklists allow a different path to be taken each time (ie. different things are done to achieve the same goal), which results in greater coverage, and more bugs found
- If non-testers have to use the checklists, they may not cover as much ground as experienced testers, and test coverage may decrease (compared to using test scripts)
- Some information that might have been included in scripts is lost. I supplement this by keeping some reference information about the application in a wiki (Fitnesse actually, so automated tests can one day live on the same page as the relevant information)
If anyone else has had experience using checklists in place of scripts, I’d be very interested in hearing about the results – good or bad.