Agile Methods for Tackling Technical Debt

Managing technical debt is such an important part of software development we include this goal in every contract we send out:

Reduce or eliminate technical debt.

All complex projects accumulate some form of technical debt. In extreme cases, it can cause project velocity to slow to a crawl. In this post we’ll review a few methods we use to help teams identify and pay down their debt before things get out of control. It’s also worth mentioning that these agile methodologies can be used when implementing complex new features to prevent the creation of additional technical debt.

Anchoring Our Discussion

To better illustrate some of these techniques, we’ll refer to an example application called LoanShark.

The public side of LoanShark is for debtors, providing quick access to current interest rates for various lenders. The private, login restricted side is for both lenders and debtors, offering an interface for users to manage their accounts.

Under the hood we’ll say the front end is built with a deprecated JavaScript framework and manual vendoring of third party dependencies. The backend is authored in Ruby and depends on both a relational database and key-value store for secure session storage.

Our Debt

Our hypothetical application shares code in a way that creates a single point of failure where there should be none. If the key-value store for sessions goes down, the account management side of LoanShark gracefully refuses login attempts, but the public facing side stops functioning entirely. Non-members don’t require a session to browse the site, so what’s going on here?

Picking apart a problem like unintentional code sharing or single points of failure can be tricky; especially without a robust test suite to ensure fixes don’t resurrect old issues or manifest new ones. A common problem we see teams dealing with in these cases is paralysis. They know the existing implementation isn’t optimal for one reason or another, but it’s not obvious how to proceed.

Research Spikes

A research spike is a time boxed exploration of a problem, usually no more than a day or two. The primary goal of a research spike is to better understand an issue; not implement a solution. For this reason any code that gets written as part of a spike should be thrown away.

This approach gives engineers room to experiment without seeking a perfect solution, and hopefully makes it easier for the entire team to make an informed decision about what to do next. It’s impossible to burn down technical debt if problems aren’t deeply understood.

Here are some indicators that a research spike may be needed:

• Team members are regularly bikeshedding back and forth about how to solve the same issue.
• A team member shuts down when a problem is raised, turning a blind eye, hoping nothing too terrible happens.
• Folks are burning out as a result of individual heroics that attempt to solve everything.

Multiple research spikes can occur during the same sprint. A decision can be made about what to do at the end, with an agreed upon method slated for future implementation.

Put an end to bikeshedding: don’t tell the team what to do to fix a problem, show them. Research spike reports also help engage folks that don’t feel like they have enough background to meaningfully contribute to a discussion by helping everyone make an informed decision together.

Tracer Bullets

Sometimes a problem’s underlying cause and its desired fix is fairly well understood. In these cases tracer bullets can be helpful. Like a research spike, a tracer bullet is intended to investigate an issue. What sets it apart is that the resultant code should be closer to production-level quality. The solution may not merge at the end of the sprint, but a fellow developer should be able to pick up where the investigation left off without having to throw everything away and start from scratch. It’s also worth mentioning that tracer bullets can prove early on that a given approach is wrong and should be abandoned.

In all cases tracer bullets should be time boxed. Depending on the problem it’s usually acceptable to run a tracer bullet issue for the majority of a sprint to ensure the proof of concept solution is solid and can be built upon.

You probably want a tracer bullet if:

• Everyone on the team actively agrees on the desired outcome and mostly agrees on how to get there.
• Time is tight and you’re reasonably sure that even though you’re faced with several approaches to a given problem there’s no true “right” answer and instead many acceptable ones.
• The problem is particularly tricky and has already had a successful research spike, but it’s not totally clear if the researched option will work in all cases.
• Some team members think a particular task will be straightforward, others think there’s still lots of gotchas and uncertainty

In our example application, a good candidate for a tracer bullet would be upgrading the aging and unsupported front-end framework, or adding a module-bundler. The team may already know what they want to use, but are not sure exactly how everything should be ported.

A successful tracer bullet issue could implement a new application directory structure, where one or more complex views are ported to prove functionality and stability. That work could then be built upon in future sprints until it was far enough along to officially be merged into the codebase.

Documentation

Now to get to the more heretical opinion. Technical debt isn’t always bad. It’s often the product of successfully working within the constraints of a project to meet goals other than engineering purity.

Sometimes the risk and time investment of a fix is not worth the reward. Some technical debt results in problems that have a negative impact on the usability of your software. Like deciding every view of an application should have a hard coded pixel width. In all but the rarest of cases, fixing technical debt that harms user experience should be prioritized, and soon. Not only for the user, but because reduced engagement is probably costing you money.

Sometimes though, technical debt just irks us. It’s using old syntax, or methods that have been shown to be negligibly less performant in the majority of cases, like for loops vs forEach or echo vs print. It’s using some old version of a dependency or a framework that has fallen out of fashion.

Our recommendation in these cases is simple: If your technical debt isn’t resulting in poor user experience, instability, or inefficient and ineffective developer collaboration, document it somewhere visible and move on. Don’t waste precious time focusing on micro-optimizations, pet peeves, or trying the latest shiny new technology without good reason. In some cases doing so causes more technical debt than it cures.

Living Debt Free

You may be left asking yourself about that initial goal:

Reduce or eliminate technical debt.

How, if technical debt is a normal part of working within the constraints of a complex system, can we ever totally eliminate it? Whenever we work with a customer and evaluate their existing technical debt we use the above methods to break it down with their team so that paralysis can be avoided.

If your team is working well, technical debt is not left to build up until no one knows what to do. Instead, like real debt, it’s constantly being created, prioritized, and paid off so your team can continue to deliver results.

// DEBT: Wrap up sentence. Bikeshedding too long on inspirational closing sentence. Ship it.

Need some help burning down your own tech debt? We’ll give you an hour of free consulting. Tell us about your biggest challenge: tyler@bocoup.com