I started out as the only tester on an agile team without any former knowledge of testing, experience with agile development or other testers to learn from. I evolved from novice to intermediate by falling into every possible pitfall on my way and learning from my mistakes. This is the story about those pitfalls and what I learned from them.
As many testers before me, I came into the testing business by coincidence. The only thing I had in my baggage was a couple of years working experience as a web developer and a Bachelor’s degree in Computer Science. I hadn’t learned much about testing during my studies, only a bit about the difference between white box and black box testing and something about usability testing.
The reasons for the company I work with now to hire a tester was a wish to increase the quality of their deliveries and to have less regressions. They had no past experience having a tester on the team and no knowledge of what exactly was needed. My job was therefore to begin mainly with manually testing new features and to look into the possibilities of doing automatic regression testing. For this of course my background in programming was a bonus, but my lack of testing skills and knowledge of how testing fits in the process – and especially an agile one – slowly showed its impact.
I’m telling this story because I have the idea, that others like me fall into the testing business by accident and have to figure things out by themselves. My hope is that some of the pitfalls I have encountered will amuse you and enlighten you.
The Gatekeeper
In the beginning I was assigned the responsibility of testing every single feature in the project. In the issue tracking system I had a “Ready for test” column, where the developers could hand over features they regarded complete and I would start testing. The problem with this approach soon appeared. Features piled up, the burn-down chart looked terrible because very few features moved fast enough to the “Done” column and the project leader got frustrated. I had turned into a bottleneck.
The problem with gate-keeping was that I got the role as “law enforcer”. Instead of collaborating with the developers – being a team – I had heated discussions about what was right and what was not. Now I know that neither of us had the right answers. The right person to judge would of course be the customer. My role should not be to guard – but to guide, inform and support.
Test it all
I also learned another important lesson. Everything was not equally important. I had to prioritize my testing. It didn’t make sense to test every single bit, it made testing expensive, my teams hair turn grey and the customers lost faith. Instead I had to ask the customers what mattered the most and use my time on that. Of course with input from the developers as well about which features constituted the biggest risks.
In love with the tool
My project manager had found a framework for doing automated functional testing and sent it my way for further exploration. With a background in programming I found it exciting to play with the tool and since it was easy to quickly automate a lot of manual flows, that’s what I did.
But soon the nightmare began and I learned some hard earned lessons. Just as it doesn’t make sense to test it all manually, it doesn’t make sense to automate it all. Not all tests are suitable for automation. Or put in another way – the benefits are not commensurate with the cost. I also learned that it’s not the start-up cost of automation that matters – but the cost of maintaining the tests in the long run. I had to think of return on investment and not just about making good tests.
Even though it felt safe and comfortable to rely on a tool it also became very clear that it’s not mastering a tool that makes you a good tester, but your experience and skills in test that do.
You can quickly automate a lot. But the value depends on how you write the tests. Automatic tests are code and should be treated as such. That means the same high standards to the quality of the code should apply to the test code as to the production code. The tests should be readable and understandable – even for the customers. And that brought me to Behavior Driven Development (BDD).
First things first
To ensure that we build the best possible product that satisfy the customer’s needs it’s not enough to just test that we build the things right but also to test that we build the right things.
After a while I discovered that some issues kept turning up again and again. Issues we thought were completed had to be discussed again. The customer was not quite satisfied with the solution, we had missed some important requirements. We could easily start a blame game about who had failed to deliver or collect the required information, but that would not be constructive. In agile development there’s a balance of not over-specifying things up front and still get the right information in time. Things change on the way in the project and there should always be room for that.
The problem of some small issues reappearing after a sprint is done might not be as big an issues in agile development as in a traditional approach, but it still breaks the flow, forcing you to halt. The way we gathered requirements could be improved.
Instead of just being a tester that did checks when a feature was complete, I started being involved from the beginning of the projects. Until now our approach to BDD was mostly focused on the benefits of automating the business requirements as tests, so we could have living documentation and regression tests. We found out that the real value of BDD was not the automation but having the conversations with the customers and discussing and gathering examples. The power of examples is that it forces you to ask questions, leading to rules, edge cases and a common knowledge.
Member of a team
From being a tester that measured my success in how many bugs I could find, I now define myself successful when I have helped the customers and the team to avoid misunderstandings and having the requirements fleshed out.
Being an agile tester is about collaboration and helping the team. The value you add is not about your individual contribution but about the success of the product. It takes courage to involve yourself and not just sit in the corner doing checks and validation. But it’s worth it, you grow as a human being and your value to the project rises.
Still being far from an expert on the field, I would love to hear your opinions. Feel free to contact me in any way.
About the Author
Rikke Simonsen works as a technical tester at Reload! A/S. Reload is a danish consultancy specialized in complex websites based on Drupal, an open source Content Management System. Rikke is passionate about BDD and is the organizer of the “Copenhagen BDD Meetup” http://www.meetup.com/Copenhagen-BDD-Meetup/ . She is @vanilleDK on Twitter and can be contacted at [email protected]