James Shore recently wrote some brillant words about acceptance testing:
I think “acceptance” is actually a nuanced problem that is fuzzy, social, and negotiable. Using tests to mediate this problem is a bad idea, in my opinion. I’d rather see “acceptance” be done through face-to-face conversations before, after, and during development of code, centering around whiteboard sketches (earlier) and manual demonstrations (later) rather than automated tests.
To rephrase: “acceptance” should be a conversation, and it’s one that we should allow to grow and change as the customer sees the software and refines her understanding of what she wants. Testing is too limited, and too rigid. Asking customers to read and write acceptance tests is a poor use of their time, skill, and inclinations.
This is pretty much where my head is at right now around automating acceptance tests. Automated tests are black and white, acceptance is gray.
“The color of truth is gray.”
~ André Gide
I prefer to have a handful of end to end automated functional tests that cover the typical journey of a user than a large set of acceptance tests constantly in a state of flux as the system is being developed and acceptance is being defined and changed.
We need to take feedback from the customer that we are building the right thing and ensure our automated tests model this, not make them responsible for specifying the actual tests.