Beware accounting errors.

A highly visible part of TDD is that developers write more tests than they used to. It is tempting to conclude from this that programming takes longer with TDD than without. After all, we are writing tests that we didn’t write before, and this is additional work on top of all the work we used to do. Right?

That’s a perfectly reasonable conclusion. But it is based on a flawed assumption, an accounting error that leads us to overlook some important costs. The assumption is that in addition to the new work of writing tests, developers will continue to do all of the same work they were doing before.

This assumption turns out to be unwarranted in practice. For example, before TDD, developers typically spend a great deal of time debugging. Why are they debugging? Because they have identified a failure, but they don’t know what specific piece of code is broken that produces the failure. They step through the code in the debugger in order to trace the failure to the fault. Only when they identify the broken code can they fix it.

TDD reduces the amount of debugging in two ways. First, developers write fewer defects. Second, when a developer introduces a defect, the code immediately fails one or more tests. These tests point directly toward the broken code. If the tests tell me what specific code is broken, I don’t need to run the debugger to find the fault.

Also, if developers write fewer defects, they now spend less time fixing defects.

If we add up all of the effects of TDD, including the cost of writing tests and the cost savings we gain by writing tests, we find that the first few features cost slightly more than without TDD, but that the quality is noticeably higher. And we find that later features take less time than without TDD, because TDD keeps code changeable.

An additional wrinkle leads to another accounting error: not everyone sees debugging as a cost. In my pre-TDD days, I usually enjoyed debugging. I loved having a meaty puzzle to solve. It was often the most fun and interesting and engaging part of programming. And in one job, my managers (falling prey to yet another accounting error) rewarded fixing bugs far more visibly and vocally than preventing them.

If you’re a manager, learn to attend not just to the obvious costs and benefits of a given set of programming practices, but also the costs and benefits that require a little effort and insight to detect.

If you’re a developer, I can tell you that I don’t miss debugging. The puzzle of how to choose the next test that will drive my code toward completion is as fun and interesting and challenging as debugging ever was. I hope it’s the same for you.

Testing is an information service. The point of testing is to inform stakeholders about the system. This is not a new sentiment, nor does it originate with me. But I’ve found that many testers have not considered their role from this perspective.

I teach classes about how to test software. Early in each class I describe testing as an information service. Even in classes filled with experienced testers, there are always a few people for whom this is a new idea.

In one class, just as I finished saying that testing is an information service, a man in the back of the room said, “Oh, no!”

“You disagree?” I asked.

“No, no, I agree,” he said. “It’s just that I’ve never thought of it that way before.” He paused and frowned. “And I think I’ve been doing it all wrong.”

I thought it was unlikely that he’d been doing it *all* wrong, so I asked, “How have you been doing it?”

“I just try to break stuff. When I can break it, it’s like I win. And if I can’t break it, I feel like I’m failing.”

“Trying to break stuff,” I said, “is an important part of testing.” I mentioned James A. Whittaker’s excellent book How to Break Software, which teaches testers how to find the kinds of defects that arise from common programming errors.

“I know,” he said, “but that’s all I’ve been doing. And when I find a nice, nasty bug, I run over to the developers and rub it in their faces.”

“Oh, no”, I said.

He laughed and nodded. “Now you understand.”

“How does that work out?” I asked. (I know what you’re thinking, but you’ve got it backwards. Doctor Phil channels me.)

“They hate it. And hate to see me coming. They keep telling me to bring them some good news once in a while.”

“But if your job is only to break stuff…”

“Then I never tell them what’s working. But that’s information, too, and that’s what I just realized. And that’s what they’ve been asking for.”

I’ve had numerous similar conversations with testers who had found themselves mired in unproductive relationships with developers. Shifting your focus from breaking stuff to informing stakeholders (including developers) can help with that.

I’ll say more later about testing as an information service. In the meantime, I’d love to hear your questions and comments about it.