skip to main |
skip to sidebar
Gojko Adzic has been doing some good writing on acceptance testing in agile development. See his article here: http://gojko.net/2008/09/17/fitting-agile-acceptance-testing-into-the-development-process/I'm not sure whether Gojko considers acceptance testing as a part of what he calls 'normal agile development' or not, but to me, it's an integral part of development. I see coding and testing as two parts of a whole. I wrote a long posting in response to his post, and so as not to waste it, I'm going to include it here too.
Here's an example of the business-facing testing activities we do during the iteration. I wish I could do a nice graphic like Gojko.Day before start of iteration - product owner goes over stories with us, we ask questions, get some examples, perhaps break a big story into thin slices, go away and think about it. Examples and overview may be put on wiki, along with any pictures/flow diagrams drawn. Product owner posts "conditions of satisfaction" for each story on wiki. Day 1 of iteration - Retrospective, then iteration planning, we write and estimate task cards for all coding and testing activities for each story, until we have enough stories to keep us busy for at least the first few days of the iteration. Testing cards for each story might be "Define high level tests", "Write FitNesse test cases", "Manual exploratory testing", "Write GUI smoke test", "Obtain test data". Then demo last iteration to customers. Then release last iteration's code. Day 2-3 of iteration - Write high level test cases, which involves more discussions with customers, perhaps design meetings. Do paper/whiteboard prototyping with customers where needed.Day 2+ - When a programmer picks up first task card for a story - start writing detailed test cases, in FitNesse if that's possible for the story. When a happy path FitNesse test passes, write more interesting test cases, collaborating closely with programmer (sometimes it's the programmer writing the FitNesse tests) When a testable chunk is available for exploratory testing, start on that. Show things to customers and get feedback whenever possible.When all coding task cards are done, do exploratory end to end tests, automate GUI smoke tests where appropriate. (Or if we're working in thin slices, do this for each slice). We focus on completing one story at a time, in priority order. Not everyone can work on one story, but the idea is we are trying to get stories done. Bring in more work as we have room. If we took on too much, figure out what to set aside, and focus on finishing. So not all our integration and exploratory testing is done the last couple of days, it's spread out more. On the last day, we are wrapping up the last story or two.
If we have a big theme coming up, we write cards to have brainstorming, research and design discussions for an iteration or two in advance of actually starting on the stories.
For my team, we've found that getting into too much detail in advance wastes time and leads to confusion. We do try to get examples, but we try to keep the discussion pretty high level and not get too much into details.
I am fortunate enough to have been invited to present at Agile Vancouver (www.agilevancouver.ca). They're calling their conference "Much Ado About Agile", and Ken Schwaber is giving a keynote. Here are just a few of the presenters: Jim Shore, David Anderson, Janet Gregory, Frank Maurer, David Hussman, James Grenning, Jennitta Andrea. It's scheduled for Nov. 4 - 6, and registration will open in the next few days.
I'm doing a track session called "Are Agile Testers Different?" and a tutorial on an iteration in the life of an agile tester. I'll only get to do the tutorial if enough people sign up, so if you're interested, please sign up early!
The best part of any conference is the informal connections you make, talking to people facing the same issues you face, getting lots of ideas and inspiration. This conference will be a wonderful opportunity for that kind of networking. And it's put on by a fabulous group of volunteers, who are going out of their way to give back to our agile community. Please consider it!
(And a note to U.S. residents - flying to Seattle and renting a car to drive to Vancouver is WAY less expensive than flying straight to Vancouver, plus, I think you still don't need a passport at the border if you drive, but I could be wrong about that).
Janet and I turned in our "final" manuscript (still have copy editing, proofreading etc to go) last night. The book is available for preorder on Amazon. on Amazon - how do they know how many pages it will be???
One cool thing (at least, in our opinion) is that we used agile practices to write the book. A year ago, we got together and mind mapped the whole book. Janet came up with a release plan, and we started in on it. We had two-week iterations. At the start of each iteration, we mind mapped two chapters, then started writing. We used an ftp site to keep the latest version of each chapter, and to pass chapters back and forth. (This was important, because otherwise it would've been easy to lose track of the latest version, or 'walk on' each others' changes). We each had a folder - 'Awaiting Janet Review', 'Awaiting Lisa Review'.
At the end of each iteration, we 'released' two chapters - that is, we posted them on our book site for review by our volunteer reviewers. We used a spreadsheet on Google Docs to help our 'source code control', checking chapters in and out. We also kept a 'to do' list and notes for the bibliography on Google Docs.
We got continual feedback throughout this development cycle. We were able to visually share ideas with our mind maps (using MindJet MindManager software). We kept in constant touch with IM, and met a couple of times in person. We build a couple extra weeks into the schedule during the holidays, and kept working at a sustainable pace (it's really hard to write a book when you have a full time job!)
By March, we had all the draft chapters posted and informally reviewed. Janet put together a new release schedule to revise all the chapters in time to send our draft manuscript to the publisher June 1. We did a major refactoring of the book, moving sections around and completely reorganizing the automation section. We made our deadline, although I confess we were more 'agile' about our release plan, we worked on the chapters in a different order than planned.
We waited a few weeks for our official reviewers to give us their feedback, then we did the last, most punishing stretch, making our final revisions. We again did a fairly major refactoring, eliminating two chapters and folding the information in them into different chapters where it made more sense.
If we took even more time, we're sure we could make the book even better. In fact, we've gotten some ideas here at the end that we could have used. However, people have been clamoring for this book (which is gratifying) since we started writing it. Every time we go to present at a conference or help a team, people want to know when it will be out, and complain when we say "January 2009". We feel (and so does our publisher) that it's better to get some really useful information out earlier, rather than to keep polishing the book and get a more perfect book out later. I don't know about you, but that's how my agile development team works, too.
It's been really interesting to see how we were able to apply agile values, principles and practices to writing a book, as well as to developing software. Wonder what else could be done better with agile!
Brian Marick (see link on side nav) has been blogging about new metaphors for technical debt. He used "fertile assets", a gardening metaphor, which I like. If you don't amend the soil and plant your seeds at the right depth and weed thoughtfully, you won't get a good harvest. You might garden rather lazily like me and use ground covers and minimum tillage, but those are still good practices and produce good results. But if you let all the weeds grow up, it might be hard to remove them without damaging the good plants. Just like code that's hard to change if you don't have automated tests to tell you something got damaged.
I like Brian's metaphor, but to me, it's too positive. If we don't protect our code with tests, and don't refactor continually, and just hack in code changes without thinking, we won't just have a bad harvest, we might ruin the soil so it can't grow anything again.
I think technical debt is more like global warming. Lots of people claim it's just a myth and that we can ignore it. If I, as a tester, try to use good practices, but the programmers don't, it's like Denmark being green but not China. If everyone collaborates on a solution, emissions will go down quicker, and we will all breathe easier.
But, since global warming is a politically charged subject, it will get people arguing about that instead of thinking about how to make teams understand how shortcuts, poor design and lack of automation will grow an ever-larger burden for them to carry.
I'm now working on a metaphor that involves donkeys, perhaps log skidding. Recently my friends cut some small trees and branches near their creek bottom to make room for a bridge. We used the mini donkeys to haul the wood and brush over to the cabin's woodpile. If we tried to put too many logs in the bundle, or tied them so they dragged too much in the dirt, the donkeys had trouble pulling them up the hill. If we didn't take the time to tie them together well, they fell off. But when we took enough time to prepare optimally-sized, securely-tied, drag-dynamic bundles, we could make quick trips, and the woodpile grew fast. A little experimentation taught us that cutting corners didn't pay off.
We needed to do a story which had several tentacles, some into ugly parts of the legacy app. The story was basically to fix invalid data that's already out there, because business rules weren't enforced through the app.
Product owner never wants to "spend points", so he wanted just the very basic things that would get the immediate need resolved, and leave some other parts of the process manual. That means humans have to remember a lot of detailed, obscure business rules. That means in a couple of years, we'd get right back to where we are, with invalid data.
We do this sort of thing fairly often. It usually starts as "OK, just to get this out the door we'll do parts A and B, and we'll do C next sprint", but we never do C because the business can work around it by asking for manual updates to the database, which they seem to feel are "free". In fact, the very problem we were solving was caused by the fact that we never did the second part of a two-part series of stories, so a system requirement was just flung into a black hole.
I'm "only" (as some would say, not me) a tester, but I made my case. With fairly minor and inexpensive (say, 6-8 hours each) to 3 pages of the UI, we would prevent a lot of requests for manual updates later, and there was a high risk that the person doing the manual updates wouldn't know all the rules and would make an error. We'd also present the end users with valid data instead of, well, no data.
The product owner didn't like it much, but the developers saw the same problems I did, and wrote task cards for the three extra changes.
No matter what your main activities are on a team, if you see an issue, raise it, and present the problems and potential costs clearly. If your team can listen to reasonable arguments, you may be able to show them your point of view, and they'll likely agree with you. This shouldn't be adversarial in any way, but you can use your passion for delivering value to good effect.
We testers aren't here to be the Quality Police, but agile development should be about doing the right things "right". When you smell a hack, point it out and ask if it's reasonable to take the time to do it right.
Why can't we Americans get a grip on how to reverse our energy problems? Thomas Friedman has a great column today.
And we could all step back and take a more generous, common-sense, and not-deathly-serious view of the world, like the Unitarian Jihadists.
I'm missing Tim Russert today.
Go, U.S. Dressage Team and especially Steffen Peters. The Chinese need to get their silly Olympic website working right, BTW.
OK, that's all off my chest, back to working on the final manuscript revisions for the Agile Testing book!
Agile 2008 was hectic and I couldn't stay for all 4 days, so I know I missed out on lots of great ideas. However, I have the cd, and the phone-book-sized book of research papers and experience reports, which I can delve into as soon as we get done with our final manuscript revisions!
It's always so good to see the friends I only get to see once a year, as well as meet people I only know through the Internt. There were so many attendees this year, I didn't run into everybody I know, but great to see the ones I did run into.
I was reminded of how welcoming the agile community is in general, and what good people they are. One of my role models is Kay Johansen. She rocks because she's good at developing AND testing. She's both highly technical and great with people. Her workshops are always fun and super effective. She and her husband Zhon taught me mind mapping, which Janet and I have used to great effect to write our book. I'm looking at her materials from the "testing with a purpose" clinic she did at Agile 2008, and it reminded me that I don't focus enough on declarative testing, I get too involved in the "how" instead of the "what".
I met Patrick Wilson-Welsh this year because he asked me to help him with his tutorial on 'flipping the automated test triangle'. Patrick is such an engaging speaker, and he has a gift for fun group exercises - who else would think of having each group compose a Haiku about obstacles to automated unit tests? Like Kay, he is one of those quietly competent people, who is ready to learn from others. I loved hearing how he has been pairing with the developers at his current gig, letting them see he knows what he's doing, before helping them think of ways to improve. People learn a lot more from that than from some "expert" just coming in and telling them what to do.
And the best part about Patrick and Kay is they are two of the nicest people I know. So, I will strive to achieve their level of expertise and knowledge, while trying to help people as best I can, learning something from everyone.
Someone at Agile 2008 observed that there are lots of women at this conference, as opposed to, say, Oopsla, and wondered if it was because Agile creates an environment that women enjoy. I think there's a lot of truth to that. Myself, and most women I know, like the people side of things, we like collaborating with people. Sitting in a corner writing code by myself isn't my idea of fun. If agile development draws more women into our profession, so much the better.
