gun-hat-glasses-black@2x.png
Ë

Why Are We Still So Bad At Software Estimates?

Let's face it, estimating the budget or timeline of a software development project is extremely difficult. Even as the industry on the whole continues to innovate at lightning speeds, accurate estimation remains the “white whale” of product companies and agencies alike.

A study conducted by McKinsey & Company found that 66% of enterprise software projects have cost overruns, 33% of projects exceed estimated timelines, and a whopping 17% of IT projects can go so far off the rails that “they threaten the very existence of the company."  Translation: not good. And while most of us agree that estimation is a necessary evil (although some have suggested doing away with them all together), the best practices to make them reliable are a widely debated topic.

Why does estimation continue to be so difficult? Let’s break down the moving parts and explore a few frameworks and formulas that will help you improve your process. 

Why It’s Hard
There are lots of analogies floating around that illustrate the reasons why accurate software development estimation is so hard. A particularly good one comes from entrepreneur Michael Wolfe on Quora back in 2013, which TechCrunch excerpted nicely:

Let’s take a hike on the coast from San Francisco to Los Angeles to visit our friends in Newport Beach. I’ll whip out my map and draw our route down the coast … Let's see, the line is about 400 miles long; we can walk 4 miles per hour for 10 hours per day, so we’ll be there in 10 days. We call our friends and book dinner for next Sunday night, when we will roll in triumphantly at 6 p.m. They can’t wait! 
Map Far Shot
We get up early the next day giddy with the excitement of fresh adventure … Reality sets in. Wow, there are actually a million little twists and turns on this coast. A 40-mile day will barely get us past Half Moon Bay. This trip is at least 500, not 400 miles … 

main-qimg-80cf13a41d184990890d473aa3f84bb6

As the example illustrates, even the most seemingly straightforward engagements can provide their fair share of unknown complexities and delays. They can take the form of technology complexity - such as a third party plugin that proves difficult, API integration snafus, or unexpected bugs - or project complexity; like delays getting repo access or key assets from another project partner, client side feedback delays or an unexpected vacation from a key team member. The possibilities for delay are truly endless. Good times.

Common Pitfalls
Project estimation nearly always occurs at the very start of a project, or even perhaps in the pitching stages of the business -- what software development and design company Atomic Object refers to as “the point of maximum ignorance”. When faced with this reality, it can be tempting to go one of two directions to try to account for the lack of clarity: spend an extensive amount of time breaking every estimate down into the smallest possible feature minutiae and estimate each out, or use a general rule of thumb, such as multiplying every estimate by a factor of 2 or 3 to ensure some “padding” should things take longer than expected. Both approaches pose issues.

While conventional wisdom may lead you to think that getting as specific as possible when estimating would lead to more accurate estimates, it can actually have the opposite effect. As developers are forced to estimate dozens and dozens of features, this can lead to estimation fatigue, causing estimates to get unruly, less accurate, and generally dreaded by the production team


Additionally, software features often share the same technical foundations which produce efficiency gains that may be overlooked in a highly granular estimation. Plus, while estimation is vital to successful companies, and group estimates generally are more accurate (more on that later), companies should still remain weary of the overhead cost of lengthy, team estimation sessions. 


Simply doubling or tripling every estimate that leaves your doors does not always improve accuracy either. While including some level of “padding” onto an estimate for unexpected complexities is prudent, and may lead you to believe you’re increasing your company’s chances at recognizing good margins on a project, in reality it can actually have the opposite effect. As articulated by Parkinson’s Law, work expands to fill the time allocated to it. While this is a purely psychological pitfall, if a half day task is estimated to take a day for safety sake, it is likely a full day, not a half day will be spent on that task. Estimate padding has the tendency to accidentally slow certain teams down.

Despite these frustrations, estimates are truly a necessary evil. Software, even for companies who claim to “not be technology companies”, is instrumental to a company’s overall business strategy, operations, and customer experience. As such, software projects come with external dependencies, be it marketing launch plans, sales outreach, or customer support training. An accurate estimate is vital not only to guide the work of engineering teams or account managers who interface with clients (or investors, C-suite leaders, etc), but every member of a company that will touch that piece of software. Additionally, estimates help projects gain consensus and shared understanding of how a product is going to be built across developers, designers, and product managers. So how do we make them better?  


estimation

Source: https://xkcd.com/612/

We Can Make Estimates Better Though, right? Yes! Sort of ..;

Clarify any ambiguity early.

Try to cut down on the effects of estimating at “the point of maximum ignorance”. Speak with clients and stakeholders in depth prior to providing a final estimate to get as much clarity around the business goal of the software and what features it will demand as possible. It is harder than you think, but will serve you in the long run.

Build in ample time for technical research.
Whether it is research surrounding the integration of a plugin, API, database, etc, build in time for developers and strategists to perform proper technical research and room to troubleshoot if needed.

Estimate as a team whenever possible.
Having multiple people perform an estimate can be hugely helpful. Group estimates allow for built in gut checks, and raise additional questions an individual developer may not have thought of. It can also be helpful to record the percent confidence the group feels for each portion of its estimate. This can help call attention to potential ambiguities or future hang-ups early.

Have those who will be performing the work perform the estimate.
Hubspot recommends one estimation strategy they call “bottom-up estimating”, where the individuals who will actually do the work are involved in the estimating process. Take this one step farther and make sure estimates are always performed by those who will be doing the work whenever possible. Although last minute project staffing may change, by having those who will be actually performing the work allow them to work against their own estimates which were made with their own skills and expertise in mind.

Stay aware of the effects of cognitive biases.
There are several common cognitive biases that crop up during estimations. One of the largest is Hofstadter’s Law. As Douglas Hofstadter famously said: “It always takes longer than you think, even when you take into account Hofstadter’s Law”. Everything in life takes longer that you think. Be weary of over optimistic estimates, and the overconfidence of less experienced developers.

Review historical data.
Your best shot at an accurate estimate may come from analyzing historical data. While every software project is different, data from previous engagements including initial estimates and timelines, as well as actual realized hourly rates, final budget spent, and actual timeline are vital to improving a team’s estimates. Be sure to take the time at the conclusion of a project to sit down and review final project data, make note of any particular tools or plugins that added unexpected complexity or even team dynamics that caused an estimate to be inaccurate (things like: Did giving the client access to a team Slack channel slow down us down or speed us up? Did we fail to calculate our weekly company meeting into an estimate? Did using Trello for a feature backlog waste more time than it helped save time?).

Outline negative scope.
We spend a lot of time outlining the scope of a particular project. But sometimes outlining what is not included is just as helpful. Making note of the “negative scope” can help provide clarity as well as protect your team and estimate down the line if the client comes back expecting more.

Don’t overlook costs beyond labor.
Don’t forget to consider other costs beyond labor that will need to be paid for. Things such as domains, certificates, licenses, photo assets, fonts, plugin access, customer testing, etc. should be considered. So called “time costs” should be considered as well, such as internal meetings, client communication, time in between rounds of feedback, etc.

Provide all estimates as ranges.
Experience has taught many that estimates are best expressed in ranges as opposed to one hard and fast number. Ranges allow for developers to make both a best case and worst case estimate for a particular project, or as Mike Cohn calls them in his book Agile Estimating and Planning, an “aggressive but possible” estimate and a “highly probable estimate”. Ranges can also be a good indicator of how confident a team is of their estimate. A large range usually indicates a lack of certainty, while a smaller range demonstrates a higher level of confidence that the estimate is accurate.

Speak up early if an estimate is inaccurate.
And finally, if you start to see warning signs that an estimate is inaccurate, speaking up early is key. It will be a lot easier to course correct early rather than approaching a client a week before launch letting them know you need an extra $10,000 to get them to launch.

Estimation will always be hard -- but we can make it better. Good luck, friend!