What Do You Want From Tests?
Automated tests are software. At first glance, this seems like a non-blinding non-flash of non-insight. But I'm learning a lot about testing by applying this non-insight mindfully.One thing I'm learning is how often I forget that automated tests are software. When I'm writing tests, I often neglect to apply all of the principles help me to write software well. What if I were to apply some of those principles mindfully?
A key principle is that we write software in order to serve some specific set of needs for some specific set of people. When I’m trying to understand what software to write, I apply this principle in the form of a few questions: Whose needs will the software serve? What needs will trigger those people to interact with the software? What roles will the software play in satisfying those needs?
Let’s apply this principle to the tests we write: Whose needs will these tests serve? What needs would trigger those people to interact with the tests? What roles will the tests play in satisfying those needs?
These days, I write software mostly for my own needs. And mostly I write the software alone. So the “whose needs” question is an easy one: When I write tests, I’m writing them mostly for me, for my own needs.
More enlightening for me—as a solo software developer writing tests solely for my own needs—are other questions. What needs trigger me to interact with the tests, either by running them or by reading test code? What roles do the tests play in satisfying those needs? Here’s a partial list of answers:
I want to know whether my software is ready to deliver.
- I want test code to help me understand which parts of the system are tested and which are not.
- I want tests to expose defects.
- I want tests to direct me to the defective part of the software.
- I want each test's code to indicate clearly how the test stimulates the software, and in what conditions.
- I want test reports to describe the test stimulus, the relevant test conditions, and the software's response.
- I want tests to tell me which of the feature's responsibilities the software fulfills, and which it does not.
- I want tests to detect changes in the behavior of the surrounding software.
- I want test code to describe clearly what the existing software does.
That’s a lot of stakeholders, and a lot of needs. I’m more likely to satisfy all of these people’s needs (including my own) when I’m aware of what the needs are. And I’m more likely to be aware of the needs when I ask questions like the ones I’ve used here. And I’m more likely to ask these questions I remember that tests are software.
What do you want from tests?