Improving Testing through Testing Debt Quadrants

By Sérgio Freire

I was lucky to attend Euro Testing Conference 2020, in Amsterdam, last month; I learned so much. One of the workshops that I attended was Rob Meaney’s “Exploring Testability” which inspired me to try something within my team.

At the conference, we did a hands-on exercise related to testing debt. Testing what?! Debt!

This article will explain the testing debt concept, how to make it visible and how to address it.

We all heard about the technical debt concept before — the development decisions made to provide benefits in the short-term that will, however, inhibit benefits in the future; in other words, imagine doing a quick “hack” to provide a feature instead of properly choosing the right architecture to support it. Testing debt is similar.

Peter Varhol, puts it slightly differently:

The effort is needed to find and fix issues that remain in the code when an application is released.

In simple words: testing debt is all about the testing that you should have done but for some reason, you didn’t.

The testing debt quadrants provide a way to visualize debt, including:

1. untested things
2. testing that is slow or impractical and that can lead it to be forgotten/not performed
3. items/components that are complex to test
4. items you cannot rely on

Therefore, we split the cardboard into 4 sections:

1. impossible/impractical
2. slow
3. complex
4. untrustworthy

Note: instead of seeing all of the above as problems, they are in fact opportunities.

The process starts by adding post-its and identifying a “piece of debt” (i.e. something that we are not testing thoroughly or the way we should).

As a team, we identify the most relevant ones (i.e. the ones that contribute the most to the testing debt). For each one of these, we discuss possible ways of addressing them and adding them to new post-its. And now we have concrete actions that we can take to reduce our debt and improve the overall testing.

After the conference, I wanted to try this exercise internally and see if it was valuable or not. I also wanted to make visible some pain points in our testing process.

We did this with a small group, including myself, part of our testers and also the team lead. Even though my role was essentially a facilitator, as I’ve been the product owner of the product we are testing, I do have particular insights about it which I cannot neglect 🙂

We started by doing a retrospective on our current development & testing process. By drawing it, we could easily see who was doing what and when. It also allowed us to depict some improvement opportunities right away.

I see these kinds of diagrams as starting points for discussion; some care should be taken though, or else this can lead to long discussions.

We then moved to the Testing Debt Quadrants and started writing and adding post-its to the 4 quadrants. Each one did this alone and in the end, we could see some overlap (which I’d say it is “normal” to happen).

Where we position each post-it in the cardboard is somehow subjective; however, as we reviewed the whole cardboard, we move some of them and agreed on it. Note that, for example, some items can be slow and complex at the same time. Where the post-its end up doesn’t need to be scientific; use “positioning” as a means for further discussion/insights.

We saw that we had some unfinished work that could tackle some of the identified problems. We just need to turn that into reality.

Having discovered and discussed part of the items that contribute to our testing debt, it’s important to have actionable, prioritized items.

Thus, we decided to create a very simple Trello board to track their implementation and/or discuss anything around them.

It’s interesting that as per our initial discussion during the “visual mapping” of our development/testing process, we could depict some opportunities to improve testing considerably. Thus, the board contained actions not only found during the Testing Debt Quadrants but also from the initial sketch we did about our overall process.

Whenever looking at its full extent, it’s easy to get lost and start discussing a ton of different topics.

Some recommendations for us and anyone trying it out:

• timebox it, so we don’t extend it endlessly;
• focus (no interruptions);
• involve other roles;
• think on improving gradually, a step at a time (this will avoid finding the “perfect solution”);
• revisit the cardboard regularly.

I would also recommend having the conversations in such a way that there is no pinpointing. This should be a team exercise, open but not individual-focused.

I think mapping your testing issues, which are also opportunities to improve your overall testing process, using Testing Debt Quadrants is a simple and valuable exercise.

All the conversations you have are important to have a joint overview of how testing is being addressed and how it can be improved.

Just make sure that these conversations aren’t pointless and, instead, lead to actions that you can track and revisit together somehow.