From Stories to Test Cases

The first artifact we embrace in Agile is stories. Writing stories is usually challenging for Product Owners and developers until they recognize stories as a unique genre of expression. Then questions arise about the proper structure for the content of a story, and how that relates to our eventual Test Cases.

Stories are a Genre

As a genre of ‘literature’, stories have a common structure. This is true for epics, user stories, system stories, and even spike stories. Defect stories are a bit different, but only a little bit.

The shared structure includes, at a minimum:

1. A title that should start with a verb,
2. A description defining the problem to be solved, and the goal of the story,
3. Acceptance criteria (ACs) defining the scope and execution scenarios of the story, and
4. An area for questions and technical notes.

The relationship of a story to its ACs is especially interesting, and a simple UML class diagram captures this:

Acceptance Criteria Structure

I am diving directly into ACs because they are critical to the successful definition of a story. Every story should have ACs that capture the expected behavior and states that the solution should express. That is, ACs define what the solution should produce and exhibit. ACs should never define the activities to build the solution. This makes sense because acceptance criteria are the propositions that should be found true when a Product Owner evaluates runtime behavior to accept the story’s solution.

But what structure should be used to express an AC?

Some teams take a verbose approach: listing internal flow and then the expected result. However, listing internal component interfaces and interactions is not something a Product Owner, or an end-user, can observe or even care about.

Assuming the ACs are limited to describing the expected results or final states, some teams will express these ACs in simple bullet statements. This is fine, but the next structure will give you more bang-for-the-buck.

I use a Given-When-Then (GWT) AC structure with my teams, especially the immature teams. These inexperienced teams benefit greatly from using a consistent, prescribed format rather than using a free-form structure such as bullet statements.

The GWT format is:

GIVEN (a known initial condition or state)
WHEN (a specific event or action occurs in the system)
THEN (list the final results or states that should be achieved).

An immediate question that arises with ACs is: how should I identify different possible results or end-states that are to be achieved?

First, you may have noticed that the GWT format is exactly analogous to the definition of Finite State Automata (FSA). In an FSA state diagram there are three basic elements, shown below with a GWT diagram:

Second, every story—when implemented in code—will follow one or more paths between one starting point and ending at one of possibly multiple ending points. This leads us to represent these paths as a directed graph, such as:

Note that each path from the beginning node to some end node represents a distinct execution scenario.

This graph reveals that we have two types of possible behavior to organize the end-states:

A.    The first behavior is the happy paths, in which the expected result of the scenario is achieved.

B.     The second behavior is the exception paths, in which an impediment occurs and the solution must attempt to recover and return to a happy path, or abandon the scenario.

Happy paths and Exception paths offer us categories for organizing our execution scenarios: for each execution scenario we create a GWT specification that describes behavior that falls into a Happy path, or an Exception path.

In the directed graph above:

• Path A-B-C-D…to H is a Happy path with no errors anticipated.
• Path A-B-J-K-… is a path that might hit a stumbling point in node K, where recovery would be attempted. If that recovery is successful, execution continues on the same path (still a Happy path) until execution ends at node H.
• Path A-B-C-L-M is an exception path. A significant error at node M (for example, HTTP code 500 System Error) would lead to an attempt to recover, but if recovery fails (as it would for a 500 error) processing would abort at that point.

With these additions of execution scenarios to our conceptual model, our UML diagram expands to the following:

I hope this has made sense, but we are not finished! We have now actually come to the most interesting aspect of story definition: Test Cases.

We have a story that defines a business need of a customer, end-user, or other stakeholder. In the ACs (and any other requirements affecting this story) we have defined how we know that we are ‘Done’—that we have a successful and complete solution. We have partitioned those ACs into two execution scenario categories—Happy paths and Exception paths—so we can have conceptual coverage that cohesively groups our expected scenario behavior in each category.

The next step is to figure out how we will verify, that is, test, each execution scenario. This leads us to produce one or more Test Cases for each execution scenario to validate the end-state of each scenario.

We must again extend our UML diagram by adding Test Cases as follows:

Here I want to return to the ACs—specifically the GWT format. If you use this format, you have already started identifying the Test Cases you need to define. Here is a simple example to illustrate this. (“SC” is a marker for “SCenario”)

Story: Create an OData API for obtaining a valid tax jurisdiction

Acceptance Criteria

Happy Paths

SC01 – Valid tax jurisdiction fields (City, State, Zip)

GIVEN the Tax database contains all valid combinations of the tax jurisdiction fields
WHEN a GET request with a valid combination of the tax jurisdiction fields is sent to the OData API
THEN the OData API returns an HTTP 200 response with a list of tax jurisdictions.

Exception Paths

SC02 – Invalid tax jurisdiction fields (City, State, Zip)

GIVEN the Tax database contains all valid combinations of the tax jurisdiction fields
WHEN a GET request with an invalid, or non-existent, combination of the tax jurisdiction fields is sent to the OData API
THEN return an HTTP 404 response to the requestor.

SC03 – Server Error

GIVEN the Tax database is not accessible
WHEN a GET request is sent to the OData API
THEN return an HTTP 500 response to the requestor.

Each scenario has a defined beginning, a designation of what happens to produce subsequent actions, a statement of the end-state or final behavior for the scenario. In short, these are your Test Cases, and you can see that your testing responsibility is to enumerate your Test Procedure for each Test Case:

• Establish the GIVEN initial conditions including the proper, or improper, data to be manipulated,
• Create the WHEN triggering event to occur within the initial conditions, and
• Verify the THEN actions and results actually occur.

Conclusion

My hope is that through this discussion you appreciate that a story is actually a community of specifications that should be meaningfully linked together as in the UML diagrams. My consulting experience has shown over and over that many teams unfortunately view stories as isolated narratives, and only much, much later do they start thinking about the finish line of their development, which should be absolutely clear from properly defined ACs. Then they start to define how they will test what they have developed. I came to the organic model in this essay over many years, and I have found with many different teams that it provides significant value and guardrails for them, so they know and can clearly express at the beginning of story definition what target they need to hit.

I love working with teams and executives to find effective solutions. Email me at gary@effectivescrummaster.com and let's explore how we might partner on your challenges.