A good development team is not cheap (nor should it be!) ⎼ whether you’re a development shop, product company, or traditional company with a technology arm, building and maintaining a skilled development team will likely take a healthy portion of your company’s budget. Given this reality, how will you determine if your investment in specific developers, development teams or frameworks is as efficient as possible?
Surely there must be an accurate, sustainable metric you can use to track the value of particular developers, right? Wrong.
It simply does not exist.
For some time, the technology industry has focused on closely monitoring key performance indicators to deduce the value of individuals within a company. While I am enthusiastically in favor of the diligent tracking of metrics across organizations as well as quantitative individual feedback, I would like to plead with leaders to loosen their grip on the need for measurement in one key portion of the business: software developer performance.
So much of software development, and the technology industry as a whole is shaped around concrete, data driven feedback. So it's not suprising or unreasonable to want a quantitative measure of success for our software developers and development teams. After all, how will we know which developers are pulling their weight, which development teams need to be restructured, and which individuals may be stunting the growth of a company as a whole? While many smart people have sought to establish an accurate, worthwhile metric to help compare the performance of developers across time, teams, and organizations, the fact remains: nothing is getting it consistently right, so let’s focus our time and energy elsewhere.
Software Metrics That Fall Short
Finding a suitable metric to measure software developer production is not a new challenge. For years, lines of code was one of the most popular means of measuring individual value. Thankfully this has since been widely debunked ⎼ after all, a developer could write 10,000 lines of code a day and not produce a working product, while another developer writing 300 lines a day could have a successful working product that contributes to his or her company's bottom line.
But so much of the value software developers bring is in how they think about and approach solving a problem.
The most elegant, efficient, and technically sound code may be fewer lines or take longer to produce, but will serve a business much better in the long run. Not only will it likely function better and faster, but it will also likely produce far less technical debt. Bill Gates perhaps said it best when he stated,“Measuring software productivity by lines of code is like measuring progress on an airplane by how much it weighs.” Let’s please leave measuring KLOC as a thing of the past.
Much like measuring a developer’s KLOC, so many development metrics, despite their best effort incentivize the wrong behavior. Even the best intentioned developers may fall victim to changing their development style or approach to better fit a success metric handed down from leadership. This is why metrics such as bugs closed, defect rate, or ticket open and close rates can do more harm than good, and instill bad habits that can hurt the longevity of software products.
But What About Productivity Metrics?
Another common way companies measure their software developers is by attempting to track their productivity over time. This becomes an increasingly enticing option when organizations adopt certain frameworks such as agile, Scrum or KanBan. While these frameworks do contain some well thought out means of measuring and tracking efficiency gains over time, at the end of the day they focus more on velocity than productivity when it comes to developers ⎼ and measuring velocity is not equal to measuring productivity.
Measuring velocity metrics, such as the number of story points completed, ideal days or feature points in a sprint speaks more to the predictability of a developers work, rather than the productivity and value of it. Attempting to compare velocities between individuals on different teams is also futile, given that such measures are based on estimates that are not made objectively. Additionally, overemphasis on such metrics may incentivize counterproductive efforts, running the risk of encouraging your developers to distort value in the estimation and planning processes which can have large business repercussions.
Velocity metrics may offer assistance when it comes time to estimate projects, assign project loads to specific developers or teams, and identify process breakdowns, but it will not help distinguish your best workers from the rest.
So What Does That Leave?
It is hard to accept the fact that finding a proper metric in software development is near impossible. After all, we have complex tools to measure more than ever before, access to impressively robust data analysis, and revolutionary project frameworks, and yet this still alludes the top technology minds. As software development moves further away from waterfall and siloed production, and adopts continuous development and release cycles, other previously “revolutionary” metrics become obsolete as well, and factors such as speed to release hold less value. The uniqueness and complexity of software products continues to rise, making it increasingly difficult to find accurate sources of comparison across developers on different projects, teams, and companies.
It is for that very reason that we need to stop trying to find this “Holy Grail” of the industry.
Many have shifted towards focusing on overall business health metrics to assess the success of a developer or development team, happiness of the production team or customer satisfaction in place of traditional metrics. While this offers some more relevant insight, I would implore development managers to trust their instincts when it comes to assessing developer performance over borderline arbitrary metrics.
Developers and development managers know which developers are pulling their weight, contributing to the team, and writing good code.
The value of certain individuals will become apparent to other members of the product team as they observe the way developers talk about the product, bugs, possible solutions, and handle QA. So please forget attempting to learn function point analysis, using irrelevant metrics to rank developers, and measuring things just because we can. Use your extra headspace to instead pay attention to your developers job satisfaction, how quickly and consistently you are releasing quality functionality, and how your team is getting along. Because we’ve tried, and measuring individual developer value just doesn’t make sense.