The Hardware Store and the Cost of Software Maintenance

A Bag of Bolts

So, a few weeks ago, I went to the hardware store to buy some casters and the hardware to mount them to a couple of cabinets. To save a few bucks, I decided to buy the hardware from the bulk bins. But, as I began filling my bag, I quickly noticed that not every bolt in the bin was the same size. Most of the bolts matched the label. But, some had clearly been misplaced.

Having experienced what it is like to make 2 or 3 trips to the hardware store to finish a project, I decided to pad my quantities by 10%, to mitigate the risk that I would need to return to the store midway through my project.

Sure enough, when I got home and began the installation, I discovered that some of the hardware I'd purchased was the wrong size. Fortunately, I did not need to return to the store. Thanks to my risk mitigation, I had enough of the correct sizes on hand to finish my project.

"Now, what," you might ask, "does that have to do with maintaining software?"

The Bulk Bin Bolt Formula

Allow me to explain. But, first let's talk about calculating the total cost of that hardware...

• Each nut, bolt, and washer had a cost. (price)
• I needed some number of each. (quantity)
• And, there was a risk that I'd need to return to the store later if I only bought the exact number I needed. (risk)

So, the formula is:

Cost = Price * (Quantity * Risk)

Note: The parentheses are just there to denote that the risk is associated with the quantity.

So, if the bolts cost $0.50, and you need 32 of them, and 10% of the bolts in the bin are in the wrong bin, then the formula would look like this:

Cost = $0.50 * (32 * 1.1) = $0.50 * 36 = $18.00

"Great! You bought $18.00 worth of bolts. How does that relate to software?"

The Cost of Maintaining a File

Well, my theory is that the formula to calculate cost is the same for of a bag of hardware as for a bag of software. There's a price, a quantity, and a risk factor. Where:

• Price = Total Complexity per file (using the ABC metric)
• Quantity = Churn over the last X months
• Risk = The lack of test coverage, which I call Exposure

Or, in other words:

Cost = Complexity * (Churn * Exposure)

Let's look at that in more detail:

• Complexity (as calculated by something like the ABC metric) represents the effort you'll expend to understand and modify a file. In other words, it is the price you will pay to modify that file.
• Churn is simply the number of changes you've made to a file over some period of time. It is quite literally the quantity in this equation.
• Exposure represents the risk that you'll need to return to the file to modify it again because of missing tests.

So, if File A has a complexity of 300, and we changed it 25 times in the last six months, and it has 80% code coverage, the total risk adjusted cost to maintain that file over that six month period would be:

Cognitive Cost of maintaining File A = 300 * (25 * 1.2) = 300 * 30 = 9,000

"9,000 what? What is the unit of measure?"

It's complexity points – the number from the ABC metric which measures cognitive load (or cognitive cost).

The Cost of Maintaining a System

Now that doesn’t really tell us anything unless we look at the whole system:

If File B has a complexity score of 200, has been modified 60 times in the last six months, and has only 60% test coverage, then:

Cognitive Cost of maintaining File B = 200 * (60 * 1.4) = 16,800

And if File C has a complexity score of 140, has been modified 5 times, and has perfect test coverage, then:

Cognitive Cost of maintaining File C = 140 * (5 * 1.0) = 700

And if File D has a complexity of 780, has not been modified at all in the last six months, and has 70% test coverage, then:

Cognitive Cost of maintaining File D = 780 * (0 * 1.3) = 0

Wait! What? Why is the cost of maintaining File D equal to zero? Because it didn't change at all in those six months.

All together, that means the total cost of maintaining this small system over the last six months was:

Cognitive Cost = 9000 + 16,800 + 700 + 0 = 26,500

"How does that help me understand the cost of maintaining my system?"

Well, if you look at the relative percentage of complexity points per file, you can see where your engineers are spending their mental energy, like this:

File A = 9000 / 26,500 = 34%
File B = 16,800 / 26,500 = 63%
File C = 700 / 26,500 = 3%
File D = 0 / 26,500 = 0%

We now know several things:

• File B is where your engineers are spending their brain power and time.
• Your team has a capacity of 26,500 complexity points per six months.
• If you could lower the cost of File B, you could free up mental capacity and time to add new features to the system.

For example, let's say we extracted half of the complexity out of File B into File E:

Cognitive Cost of maintaining File B = 100 * (61 * 1.4) = 7930
Cognitive Cost of maintaining File E = 100 * (1 * 1.0) = 100

Note: This assumes you'll add 100% test coverage to the extracted code. Otherwise, how will you know if you broke it during the extraction.

So, now, the total cost of the system has been reduced to:

Cognitive Cost = 9000 + 7930 + 700 + 0 + 100 = 18,630.

That's 7,870 (or 30%) fewer complexity points! So, in theory, splitting that one file by testing it and extracting code into another class lowered the cognitive cost so much that you freed up 30% of your capacity for other work?

"Wait! What? Are you saying that I can add 30% more work to the team's backlog if they split that one file in half?"

Yes. That's the (untested) theory.

Cognitive Cost = Complexity * (Churn * Exposure)

As I noted at the top of this post, this theory came out of a thought exercise. I was literally adding casters to a cabinet while thinking about code quality: complexity, churn, and coverage. The price * (quantity * risk) calculation struck me as essentially the same thing as complexity * (churn * exposure).

Now, who's going to test it with me?