Technical Debt and Delivery Times

Today, I want to muse on how new code features take ever-so-longer, specifically, how increasing delivery times are the symptoms of bad code. I’ll focus primarily on recognizing the issue (rather than avoiding or solving it).

The Mystery of Increasing Estimates

Here’s a typical scenario: You are a project manager and you hire a programmer for a new feature set. You request the first core feature. It gets delivered. You request an expanded feature. It gets delivered somewhat late. You request a further expanded feature. It gets delivered way past the deadline. (P.S. every estimate runs twice over.) You panic and have your senior programmer take over. They estimate the next delivery at twice your worst expectations!

So what’s wrong with this picture; how come it’s consistently taking longer? Here’s where the senior programmer shakes their fist grumbling, “is your code clean?” Or did you just add technical debt to the project? It takes an experienced programmer only a moment to spot bad code. Hell, they could spot it from the class outline alone.

Hidden Technical Debt of Estimates

There are two (grossly simplified) time consumers when programming:

Features are all the good things you want and it takes time to do. Technical debt is the stuff you don’t want and it takes time to prevent and undo. Features are immediate and highly visible. Debt is expert at hiding and won’t appear straightaway.

Let me illustrate with some example programmers. We are comparing the time it took to implement a working feature versus time unspent it would take to fix the left-over technical debt.

Programmers A and B gave realistic estimates and delivered on time. They left technical debt. They are pretty average cases if you have no oversight by a lead. Programmer A took a bit longer, but ensured a bit less work later. B hurried it a little and left over more stuff to fix.

Programmer C is a rock star programmer and everyone talks about how he delivers in record times. He is on feature #3 already. He changes jobs every 4 months. (He never finds out what happens to the codebase down the line.)

Programmer D had the longest estimate. He left almost no debt. He uses smart programming words. He keeps talking about some religion called “unit testing”.

Lastly, U is the ideal genius programmer. Before you get any ideas, U stands for “unicorn”.

Deceptive Results

This is what the manager looking at the product sees as immediate time invested:

But this is the future time required that a (lead) programmer sees:

The main point to take from this: it’s not externally obvious if a feature comes with little or a lot of debt. It may appear that programmers and features take roughly the same time or are really productive/slow. Such cases actually also carry hidden costs and benefits. Fast features are never “free”. In contrast, time invested is time saved.

Growing Debt

Okay fine, but we’re on a deadline – we’ll fix it later! So what if the feature is not perfect? Right?

The reason this is bad long-term is that technical debt bends the rules of math:

Debt creates debt creates debt creates debt… Not only are individual code parts smelly, but the connections between them start to smell and it leaks into other parts. One hack becomes two hacks just to use the first hack. The very nature of technical debt is that it intertwines and couples code, creating ever-increasing inter-dependencies.

And features don’t exist in vacuum. You cannot make new features without fixing at least some technical debt to keep it from growing exponentially:

Or to put it another way, when you encounter unexpected technical debt that needs fixing:

An experienced programmer can spot the debt and will adjust their estimate accordingly. An inexperienced programmer won’t and will run over the estimate trying to make their code work. Worst yet, they won’t fix existing debt nor recognize their own debt. In short, your next feature will cost more.

The Cost of “Fixing it Later”

Sure, you can fix it later, it’s not that much. But “later” will arrive sooner than you think. And it will arrive suddenly and irreversibly. Let’s say we only fix the debt that is absolutely critical to making new features. You will see delivery times something like this:

A and B started off great, but their estimates and delivery times keep growing and growing. They assure you that they are working exactly as before. Not only that, they are occasionally fixing old code too. You are perplexed, but shrug it off as typical costs of increasing complexity.

C was doing so amazing, but they had to leave after the fourth feature. Perhaps someday you’ll find another one like them! At least you had their photo framed.

D seems to always have exactly the same estimate and delivery. In fact, he seems to suffer no increased delivery time symptoms that A and B have. He must be putting in extra hours or something.

Of course, we know exactly what’s going on:

A and B have a steadily growing debt. D is managing theirs at a near-constant level. C ran away when they saw the avalanche.

Suddenly D is the most efficient programmer and A and B just cannot get back to how they started. (It may not even be their fault with such tight deadlines.) This is the long-term price of technical debt.

As a side note: of course bad code is hardly the only reason for increasing deadlines. But it’s the biggest reason when it comes to sudden and hard-to-explain delays. Ideally, the feature estimates already include modifying and connecting to the existing code.

Caveats

There is one “upside” to technical debt though: code can only get so bad before it no longer matters:

At some point, you can just delete it all and redo, as that’s going to be faster. Hopefully, without repeating the same mistakes.

Secondly, not all technical debt is equal in scope versus time. 80/20 rule can probably be applied to this. 80% of time would be spent fixing 20% of the debt:

So you can deliberately choose to not fix the last 20%. A sufficiently modular and independent code can survive with moderate debt without affecting the rest of the codebase.

Finally, it takes longer to fix someone else’s debt than your own. At least, you yourself know what you hacked together. Can you tell with any confidence what someone else hacked up?

To sum up

Beware short estimates, because they can carry new technical debt. Beware short estimates, because they likely don’t account for existing technical debt.

Get ahead of the programmer. Expect unclean code. Features are only half the work; prepare for the other half.

 

P. S. I hope to also write up a generic “Preventing Technical Debt” post in the future.