Hi Dale,

I think your paper is on the money. Testing is engineering. Maintaining any large body of code requires discipline. And maintainability is the enemy of most of the ATDD suites out there.

Just to add to the list of great tools in this area - I'm using Concordion. I really love this tool because the website author really understands this problem of maintainability and really tries to address it in the design of the framework. I also think Concordion doesn't get the attention it deserves and I've been on mission to improve that.

It could be argued that this tool requires more programming skill than others mentioned. But in my opinion that is a very good thing indeed.

I have written a blog entry with a full example of using Concordion here. http://bit.ly/cZhlVn. Concordion specifically enforces abstraction to enhance maintainability. Hopefully these ideas will catch on.

Neil

Good point, Paul. I hope people aren't seeing any of this as a tool war. I certainly didn't mean it that way.

I'm delighted that the developers of Robot Framework, FitNesse, and Cucumber (and perhaps other tools I'm not aware of) are rapidly learning from each other, each incorporating the good ideas from other tools into their own. I look forward to the day when people can represent tests in whatever way seems most natural and expressive to them, and each tool can directly execute tests written in a variety of those "natural" styles. The tool developers are clearly moving in that direction.

I deeply respect both Dale and Uncle Bob but I'm concerned that the true message of both articles might be lost in "proving that one tool is better than another". The point that I think that both are making is that you want to make your tests in WHATEVER TOOL YOU'RE USING be as maintainable as possible. Tests, like code, can be fragile, have duplication, can smell and they can also be refactored. Gerard Meszaros, author of "xUnit Test Patterns: Refactoring Test Code" gave a presentation this week at the company that I work for. He used the very same techniques that Dale and Uncle Bob used but used an xUnit based tool for his examples. His presentation, like Dale and Bob's, did not describe how to use their tools per se, but focused on removing the cruft, verboseness, etc from the tests to make them more maintainable.
All three of these presentations made me a better developer! Thanks Dale, Bob and Gerard!

I think the text writted in BDD style and automated within jbehave (www.jbehave.org) is more easy and intuitive for java systems.

Great article, Dale! I would change just one small thing: I think the tests might be a bit more expressive of system behavior if you changed the variable names "${aTooShortPassword}, ${aMinimumLengthPassword}, ${aMaximumLengthPassword}, and ${aTooLongPassword}" to ${aPasswordWith5Chars}, ${aPasswordWith6Chars}, ${aPasswordWith16Chars}, and ${aPasswordWith17Chars}". The reason is that as I looked at the tests you presented, I found myself wondering, "so just what are the minimum and maximum password lengths?", and the only way to answer the question is to examine the variables and count "1-2-3-4-5-6" etc. That said, there is a trade-off in my proposal, insofar as a reader would have to infer from looking at the tests that minimum and maximum password length are being tested. In my opinion, that's not as error-prone or tedious as counting string lengths--but as with any trade-off, opinions may vary.

Assuming a project will be implemented using Acceptance Test-Driven Development (ATDD), then analogous to Test-Driven Development (TDD) the steps of development are the following:
* Write a failing acceptance test
* Develop production code using TDD (test, code, refactor) satisfying the test
* Refactor the acceptance test.

So basically, the missing step was the last one, namely the refactoring of the acceptance test, which includes techniques and patterns similar to those applied during the development of production code.
I just wanted to point this out, because the word refactor or its derivatives aren't present in the paper. Since "test automation is software development", therefore improving the design and structure of the tests is refactoring.

I became aware of this article by Uncle Bob's blog and just want to express appreciation for this gap-filling work!

Bob Martin posted a great video showing how to apply these ideas using FitNesse.

Someone will translate this to Cucumber any day now...

Markus, yes, you'll see lots of similarities between what I do and what Elisabeth does. It was Elisabeth who turned me on to Robot Framework, and we often co-teach her wonderful Acceptance Test Driven Development class. And of course I stole the "login" example from her.

I liked your paper. It reminded me on the tutorial session with Elisabeth Hendrickson I joined at the Agile Testing Days in Berlin this year. She had the same example and it was very easy for me to follow. Maybe I'll be able to take some time and give Robot Framework a try in a week or so, when I've dealt with the last conference for me this year.