Target audience: Product People, Testers, Developers
I’m sure that in your workday with software development you’ve heard words like BDD, ATDD, and Specification By Example. Most of us link these terms to just the test portion of an application but do not observe how powerful the practices are.
Rather than just contributing to the test of what your team is developing, they help ensure that it is actually doing the thing — what your customer needs!
To try to clarify a little better where these words came from and how we got here, I set up a timeline based on several different authors. These authors have written books covering the breadth of software testing and its principles. In the end they’re all basically saying the same thing, but they’re also presenting the theme according to their point of view.
A Little History
Figure 1
1999 — Kent Beck (Extreme Programming Explained: Embrace Change)
In this book, Kent Beck presents the values, practices, and principles of XP. Although he doesn’t talk about the term “Specification By Example”, he talks about continually specifying and refining during development, so that customer learning is reflected in the developed software.
In the cycle he proposes, the stories are specified immediately before they are implemented. After which, they become extracted tests of those specifications. In the cycle that follows —
- The interface is designed to meet test needs
- The code is written to match the tests and the interface
- The design is refined to match the needs of the written code
This leads to a decision about which story to specify next. Meanwhile, the rest of the stories remain stationary, until they are chosen for implementation.
Figure 2
2004 — Martin Fowler — Specification by Example
In this article, Martin uses the phrase “Specification By Example” to refer to what Kent Beck had called specs in the XP part of testing. Instead of dealing only with behavioral specifications (thinking about preconditions and postconditions), the word example is added to make us think in a more humanized way in terms of actual use.
An important point that he raises is that Specification By Example alone does not solve all problems. You need to always have the support of more than one tool to achieve your goals and solve problems.
He also reinforces that this only works in collaborative environments — where all the people involved in software development are together building the best product.
“Always have the support of more than one tool to achieve your goals and solve problems”
2005 — Lisa Crispin — Customer Test-Driven Development
Communication and collaboration are two key words in this article by Lisa. For each story, she asked the client how he would know that the story was complete — in the form of tests and examples.
Just like in TDD, before writing any code, the tests are written first — and then if they pass, it means that the code meets the requirements. The tests should ideally be written in a way that can be understood by an automation tool. It should also be at a high level so that they can guide the exploratory tests.
“Write tests that can be understood by an automation tool”
2006 — Dan North — Introducing BDD
During his journey using TDD, Dan North noted some difficulties, such as — where to start, what to test and what not to test, how to call the tests, and how to understand why a test fails.
BDD is an agile practice that allows better communication between all technical and non-technical people involved during a software project. It describes a cycle of iterations with well-defined outputs and resulting in the delivery of tested and functioning software.
“BDD allows better communication between technical and non-technical people involved in a software project”
2008 — Elisabeth Hendrickson — Driving Development with Tests: ATDD and TDD
In this material, Elisabeth talks about executable requirements, where you create tests before coding. These tests represent the behavior that the software should have. It reminds you that it is very important that this work be done together with the stakeholders.
According to her, the specifications give feedback on how close the team is to the “done” of a story, giving even an idea of progress toward the goal.
2009 — Gojko Adzic — Bridging the Communication Gap: Specification by Example and Agile Acceptance Testing
The most relevant point here in this Gojko publication is — specification is not a programming technique, but a communication technique. It brings people involved in software development closer to the project.
“Specification is not a programming technique, but a communication technique”
2010 — Lisa Crispin — ATDD X BDD X Specification By Example
At Agile 2010, during a workshop on Functional Testing, about 20 people from the international software community were discussing what ATDD really meant.
As you can see in the summaries of each of the sessions above, several people were saying the same thing but with different names and some particularities — (examples, acceptance tests, scenarios, user tests, behaviour, functional tests, executable specification …).
As a result of this workshop they came up with a definition of Specification By Example that would be described in 2011 in Gojko’s book of the same name.
Figure 3
2010 — Craig Larman — Specification By Example
When Craig talks about Less, he devotes a part to techniques that help achieve technical excellence. And guess what — the Specification For Example is there (as well as Integration and Continuous Delivery, Clean Code, TDD and others).
According to him, tests should be regarded as requirements and requirements as tests. The book reminds you that if you have a test that describes the behavior, and at the end of development this test passes — it means that you have reached the goal of that functionality.
In addition, he cites other elements we have seen in previous blocks, such as — doing workshops to collaborate and discovering the requirements, working on short development cycles, and focusing on problem prevention rather than finding.
“Tests should be regarded as requirements, and requirements as tests”
Figure 4
2011 — Gojko Adzic — Specification by example: How successful teams deliver the right software
It was through this book that I came to know the subject. Two years ago as I entered Concrete, it was the first book I received as a reference on QA’s work.
In this book, Gojko brings together a series of standards that have been observed in several successful projects. He defines Specification by Example as a set of these standards that help to develop the right software.
Since then I have been using the references in this book by Gojko when talking about Specification By Example. But in doing the research for this post, I saw that his book is a summary of what many other people were talking about the world.
Get Down to Business
Going back to the beginning of this text, many people associate Specification Examples with only the testing part of the application. But they do not use all of their tool-ing to effectively build quality products — both in terms of meeting functional specifications, and actually being what the customer needs.
I consider Specification By Example as a very focused approach to discovery. Here collaboration between those involved in the development process (product and engineering) is extremely important in order to discover the requirements needed to meet the customer’s need. It helps to exemplify them in a way that all of these people can understand what is being said (so it uses a domain language) and using engineering support (with automated testing and Continuous Delivery and Integration practices). This ensures the
Applying SBE in Practice
For each of the practices of the Specification By Example, I have examples of how we can apply it:
#1 Derive the scope from the objectives
Find out what the customer really needs. Then, give the development team the freedom to suggest the best way to do it. This is simpler to implement, faster and even cheaper.
Our focus, then, should be to understand what problem the customer wants to solve, or the goal he wants to achieve. And from there, the whole team collaborates to arrive at the solution design.
#2 Business Objectives -> Derive the Scope -> Create the Stories
For example, if we aim to solve a problem of customers who want to pay an account quickly, we can derive the following scope:
The bank account holder will pay a ticket through the barcode. He can type the barcode, which is slow, or can use the camera of the mobile phone to capture the image of the barcode. The application must interpret this image and fill in the ticket data to be paid without the need for user intervention. The user can review the data and confirm the payment.
The faster option should be offered as the first option to the user, while the slower option should be offered as an option.
Story 1: I, as a checking account holder, want to use my cell phone camera and identify a barcode, so that my ticket data is filled up quickly.
Story 2: I, as a current account holder, want to manually fill in the barcode number of a ticket, to pay it even if the bar code is unreadable.
Story 3: I, as a current account holder, want to review the completed data of a ticket informed via barcode, to confirm them and to effect payment.
#3 Specify Collaboratively
Here is where some people who try to implement Specification By Example end up getting lost. Often only one person within the team (usually the one who plays the role of QA) writes the specs, and the rest of the team just looks. If this is what happens, we totally lose the sense of collaboration.
The idea is that — all involved should participate, and this can be through workshops, or writing and review. The important thing is that the specifications are written by the team including the client. Language has to be part of the business domain and be understood by everyone.
#4 Illustrate Using Examples
Human beings better understand practical examples. This helps avoid ambiguity and communicates accurately. It is important to use concrete business examples and think about expected outputs.
In addition, the examples help to generate discussion and eliminate doubts. Here’s one —
Functionality: Free Delivery
1: Offered to VIP customers, once they buy a certain number of books;
2: It is not offered for ordinary customers or for VIP clients who buy anything other than books;
3: The minimum number of books for free delivery is 5
Examples:
Figure 5
What do good examples look like?
- Must be precise: it helps to avoid ambiguity, it should clearly inform the context and how the system should behave in each case
- Must be complete: it is important to combine different inputs and think about the expected results, as well as to experiment with data
- Must be realistic: always think of real user situations (caution with sensitive information)
- Easy to understand: everyone involved should read the examples and understand without much difficulty. You can use abstractions — for example, use “car” as an example instead of describing exhaustively the characteristics of the car: four-wheel car, two-door etc., unless these characteristics influence the final result.
You can also use examples for non-functional requirements.
- Performance: “The system needs to import X records into Y minutes by consuming CPUs.”
- UI: Screen prototypes are good examples as well.
#5 Automate Specifications
Today we have several tools available that allow you to automate specifications. The best known is Cucumber. Here, all the scenarios you wrote using the Gherkin format are interpreted and implemented from any other automation framework. Some examples are — Appium or Selenium.
With each new feature or change to an existing one, you can add or modify its specifications and ensure that the regressive continues to work.
#6 Validate Frequently
It’s no good getting several automated tests that do not run continuously. To ensure that your specification is alive once you have implemented them, you need to use mechanisms such as Continuous Integration for these tests.
It helps to remember that they are tests built from the specifications — they ensure that what has been developed meets the business needs. They need to be run frequently with each new change. This ensures that if there is a problem, the tests give you quick feedback and indicate what needs to be fixed. Here you can use tools like Jenkins, Bamboo, CircleCI, among others.
Conclusion
The purpose of this text was to show that developing from specifications is not new. We have been talking about this for a long time in software engineering.
But the most important point of the specification as we know it today is the constant discovery of requirements related to the product. It’s also about the use of that knowledge to generate a business domain language, which will be used in the writing of live and executable documentation.
Image Credits:
Figure 2: Extreme Programming Explained: Embrace Change — Kent Beck
Figure 3: Results from Workshop in Agile 2010
Figure 4: https://less.works/less/technical-excellence/specification-by-example.html
Figure 5: https://www.amazon.com.br/Specification-Example-Successful-Deliver-Software/dp/1617290084
References if any
Author bio:
Linkedin: https://www.linkedin.com/in/samantacici/
Twitter: https://twitter.com/samantacicilia
QA Manager at Concrete (an Accenture Company), helping teams to define quality strategies to create incredible digital products. Today I lead 40 QAs in the Rio de Janeiro, São Paulo, Recife, Belo Horizonte (Brazil) and Colombia offices.
I am passionate about agility. I believe in empiricism, continuous improvement and diversity.
Co-founder of the Carioca Testing Group that is now part of Ministry Of Testing (https://www.meetup.com/pt-BR/Ministry-of-Testing-Rio-de-Janeiro/)
Articles of QA published in https://medium.com/@samantacicilia/ and https://code.likeagirl.io/portugues/home