An Example of How Testers Contribute

Here's an example to support my previous post about the "hidden resources" that may be lurking in testers, but not used for whatever reason (lack of support from team or managers, lack of time, lack of resources, lack of imagination).

Our web-based app manages all aspects of 401(k) plans, including doing trades. Lots of money moves around, but a lot of it moves around outside the app itself, in QuickBooks, via emails to a custodian, and so on. We, the development team, is quite expert in the 401(k) domain, and fairly lacking in the accounting domain.

We had a story three iterations ago to "tweak" a report so the accountant could use it to help track down the sources of problems when the cash accounts didn't reconcile. This was a small story, but because we never took time to really understand the big picture of how cash flows around the system (both inside and outside of the app itself), and really understand what the accountant needed to do, it grew into a giant story. There were plenty of issues that to me were smells. It was difficult to test because we didn't have production-like data for it and couldn't produce it; the results of running the report were erratic and not always explainable; it turned out that if certain batch processes ran at the same time, the data for the report would be corrupted and become useless.

Long story short, I raised all kinds of flags, but there was pressure from the business to just get this "little" story done. Although both the programmer and I spent time with the accountant trying to understand what was needed, we didn't even learn enough to know that we didn't know enough. We went through two iterations of fixing and re-releasing, to no avail. We wasted time in meetings discussing how to fix the report, when in truth the report was the wrong approach for what the accountant needed from the get go.

I finally convinced everyone that we needed to take a step back and have the accountant explain to us the larger picture of how she has to reconcile accounts, all the different ways money can move around and the possible sources of imbalance, and what information she really needed to have. It was finally decided that a completely different report was needed. The time on this story was wasted, except for the lessons we learned about how little we understood about the accounting domain.

Was it my job, as the tester, to put the brakes on this story and insist that we start from scratch to understand the requirements? No, the programmer could have raised that issue, or the ScrumMaster, or the product owner (the poor accountant tends to get ignored, for no good reason other than accounting doesn't sell 401(k) plans, and she was already doing all she could to try to make herself understood). However, my testing of this report showed that there were some basic issues in the system itself that would have to be fixed before this report could work, so either we had to address those with new stories, or take a different approach to get the necessary data.

I feel that as the tester, I had a better opportunity to see the big picture of what was going on. I took some time to sit with the accountant and understand more about her domain, and what she needed for her reconciliation process. We testers can detect smells and provide information about them to the rest of the team. We can, as I did in this case, bring together customers and developers to re-think a problematic story. I didn't throw myself down on the floor in a fit and say "we can't release this report because I don't think it works". I explained the various issues and asked if key players could meet to understand the problem and find a new approach.

I think many testers would do just what I did, but some may feel it's "not my job", or not feel confident enough to call for a dramatic change in approach. How can we empower all testers to dig into problems and bring together all the people needed to solve them?

Testers: The Hidden Resource

I spent last week at STARWEST, and enjoyed being amongst the fine testers who are interested in learning new things and delivering value. I learned a lot myself - great Ajax testing tutorial by Paco Hope, fun six hat idea from Julian Harty, and more.

I didn't have as much time as I would have liked to join in the interesting evening discussions at the Lost Bar (I had a lingering cough to tend) but we did have a brief discussion of how to raise the standard of the testing profession. I feel there are too many people calling themselves testers that are basically punching a clock, waiting for work to fly over the wall at them, and doing some manual scripted testing. It gives testers a bad name.

Someone suggested maybe we should invent a better title than "tester" to attract good people. Personally I am proud to be called a tester, but then I think back to my mom, a career secretary who had a business degree and good business knowledge, but insisted on being called a 'secretary'. She felt it was an honorable profession, and the title was not demeaning. Despite herself, she became an executive assistant.

Yesterday it occurred to me that maybe the way to approach this is to get employers interested in mining the hidden resources of their testing and QA teams. Testers have a lot to contribute. We're good at talking to customers and understanding requirements. We're good at raising a flag when we see a problem or a gap in communication. (I just raised one yesterday, because our accounting person is not getting the software support she needs, because we don' t understand her domain). We're good at collaborating with developers to help them understand the stories and help ourselves understand the architecture and design. We're good at writing test cases, and good at exploratory testing that helps the whole business learn more about how to improve the software.

But most companies leave testers' potential untapped. This is a waste. What if managers sent all their testers to at least one good conference a year, or if that's not in the budget, at least to local user group meetings? What if managers budgeted time for testers to learn and improve their skills? What if managers helped break down barriers between teams and enable collaboration? What if managers supported testers and gave them a chance to show how they can contribute?

Some of us (such as yours truly) are pushy and we step up and do these things for ourselves, but there's no reason businesses shouldn't help testers help themselves, and in turn, contribute more value to their teams. When I've managed QA teams, I helped each team member set goals for professional development, gave them time to learn, bought them the books they needed, got them the training they needed. I had amazing teams of self-motivated people who worked together well and collaborated with customers and developers well (and this was in my pre-agile days). I'm not saying I'm some genius of a manager, I'm saying that giving time, training and support pays off in spades.

How can we get the word out to those that don't understand the huge hidden resource which may lie in their testing and QA teams?

Agile Tester Skills

A StickyMinds column by Johanna Rothman about exploratory testing on agile projects prompted a discussion about what skills agile testers need on the agile-testing mailing list.

Janet Gregory posted this response, I thought it was insightful.

Johanna talks about the perfect tester skills but acknowledges generalists without coding skills may have a place. I have seen a few teams where none of the testers had coding skills, but worked with the developers to create the automation framework using FIT or some other similar tool.

If the team works closely, and develops a framework that the 'non-coding' testers can use easily, the testers can be very productive. I do agree that it helps the team if the testers can code, but it is not absolutely necessary. Because the whole team is responsible for quality, the whole team usually can solve any problem.

I think that some of the most valuable skills a 'tester' can bring to the table, are critical thinking and understanding of the big picture - implications of proposed changes. Exploratory testing helps to expose issues if testers have those skills, and can use them properly.

She adds in a later post:
I didn't take it one step farther which is to say, that many of the "manual" testers who started using the spreadsheets for Ruby/watir implementation, eventually got comfortable enough to start understanding the code and actually doing some of their own scripting. That shows initiative and proves that everyone is capable of learning :-)

I do think it is absolutely beneficial to understand scripting and coding, and I do encourage learning those skills. I think testers will go so much farther if they do. However, it is important that we don't discourage those testers that don't have any coding skills when they first come to an agile team. I guess those are the testers I work with mostly.

I've worked with "manual" testers who were happy to learn new skills such as automation when given the time and support they needed. I've met so many teams where the testing team freaked out when the development organization adopted agile. Telling testers they need these skills they've possibly never had the chance to acquire is not going to help with the transition.

I also know lots of testers who had no interest in learning new skills or improving. People like that won't fit on an agile team, for sure. But anyone who gets excited about agile and wants to work on an agile team is worth a bit of investment, in my book.

What an Agile Tester Does

This has been kind of a crazy week. I've had to multitask a lot, which I know is bad and inefficient. Here's a sample of the types of tasks I've done this week:
- Write high level test cases
- Automate tests in FitNesse
- Debug problems with Canoo WebTest GUI tool and tests
- Look into three different production issues
- Manual exploratory tests of various stories
- Participate in estimation meeting
- Spend a lot of time getting requirements and clarifications from various customers
- Help team strategize best way to make sure we get everything completed this sprint

If you're an tester on an agile team, are those in line with the things you do? Are you surprised by any of them?

I think some people might not expect a tester to work on production issues, but as testers, we tend to learn a lot about the business domain. It's natural for business people to come ask me where something is documented on our wiki, or how a piece of functionality is supposed to work (at least, how we delivered it, maybe we were wrong on how it should work!) I get interested in production problems because I want to know how we didn't run across a particular issue in testing. Working on production issues isn't in my job description, I suppose, but it helps me learn a lot about the app and what we can do to improve it.

I know a lot of agile testers who do similar tasks. I'm interested in what other things agile testers do.

Interdisciplinary Awareness

One of the most interesting sessions I attended at Better Software was "You Just Don't Understand Me: Interdisciplinary Awareness to the Rescue" by Mike Tholfsen, principal test manager of the Microsoft OneNote team. (See his blog at http://blogs.msdn.com/onenote_and_education/) He presented a "Team Pyramid" (don't we love all these pyramids?) showing that for a successful team, you need trust as your base. Results are the little top of the pyramid, they come from trust, healthy conflict, commitment, accountability and results.

Mike feels that one way to achieve this is to help people understand their peers' viewpoints better. He introduced an exercise to help people in different roles on the team understand the important traits of each discipline, and trade ideas on what teammates in other roles like or dislike about each discipline.

An interesting point of the presentation was that there was a development team where all the team members scored the same on a Myers-Briggs style test. The manager had hired himself four times. Lack of diversity is not a good thing.

This made me think about the times we've hired a tester onto my agile team. I brought in testers who were great with communication, collaborating with customers, exploratory testing, but not a lot of skills on the automation/technical side. I felt they could contribute value, but my developer coworkers gave them thumbs down. In each case, we hired a very techy tester (fortunately, also good at all the other things).

Mike's presentation made me realize that the developers were most comfortable hiring someone like themselves. This is understandable, but not always in the best interests of the team. Having realized this will help us in future hiring efforts, I think. Do we really not like something about the candidate, or is it just that they're different than we are? Could the differences make us better?

There appears to still be a lot of controversy in the agile community over testers and their role on the team. Interdisciplinary awareness might help agile teams realize that people with a different skill set might add tremendous value to their team.