4 Objectives for an Automated Test Strategy

I recently explained to my team the objectives our automated test strategy had to fulfil:

  1. Provides “living” requirements documentation
  2. Results in a comprehensive regression test suite
  3. Ensures high functional coverage
  4. Runs fast for developers

Provides “living” requirements documentation

Stale requirements documents are the norm. MS Word documents in a folder on a file share. Printed documents on a shelf. I’m not interested in those.

I want “living” documentation. Documentation that has an impact if it is changed. Unit tests do that but only for the developers. A tool like Cucumber, with associated Gherkin scenarios, provides that at the business / functional level.

Results in a comprehensive regression test suite

I don’t particularly care what sorts of automated tests we have but I do care that we leave behind a comprehensive test suite. Sufficient tests, of any kind, to make regression testing fast and easy.

Ensures high functional coverage

Okay, I’m not entirely agnostic to the type of automated testing we have. There has to be a fair number of tests that assure the business that the system does what it is meant to. That means functional tests. Unit tests are good but not sufficient of themselves.

Runs fast for developers

The automated tests have to run fast. This benefits everybody, but the main beneficiaries are the developers. They won’t run slow tests. Because I want the developers to run the tests all the time, the tests have to run fast. Or at least there has to be a significant subset of tests that runs fast.

This post is part of a series on Specification by Example.