When to use automated tests

Some simple advice on automating tests.

Automated tests are a very useful and impressive tool used to make testing more efficient. However, automated tests are not suited to all projects – this may be due to lack of time available, or due to technical limitations.

Automated tests take time to create. Depending on the testers, it can take 3-10 times the amount of time to create an automated test as it takes to run the same test manually. Therefore, automated tests will only start to be valuable when they are run more than 3-10 times.

Automated tests are suitable for the following purposes:
–    Regression testing for a stable system that will be run on a regular basis
–    Fast data creation in test systems where the database must be wiped on a regular basis

Automated tests are NOT suitable for the following purposes:
–    Testing new functionality – this should be done manually before automated tests are created
–    Regression testing systems that are expected to have significant user interface changes. Large changes to the user interface require a lot of maintenance for automated tests.

When automating tests, it is wise to only automate as many tests as your team can easily maintain. If some tests are becoming difficult to maintain, it may be worth considering retiring those tests.

Above all, remember that automated tests will never detect as many bugs as a human tester executing the same steps. This is because the human tester’s eyes can pick up many things, whereas the test will only notice what it is programmed to notice.

For more information, please refer to the following books:
–    Lessons Learned in Software Testing by Cem Kaner, James Bach and Bret Pettichord
–    Just Enough Software Test Automation by Daniel J. Mosley and Bruce A. Posey