Performance Metrics – Something to be Gamed? Part 1
In a recent post on Joel on Software, Joel wrote passionately about management consultants and their tendency to focus on performance metrics when analyzing software organizations. Simply put, he doesn't think that technical team performance can be effectively measured. In fact, he referenced a previous post from 2002 on the same subject–so this is a relatively long held belief on his part. Here's a snippet to share some of his thoughts:
Software organizations tend to reward programmers who (a) write lots of code and (b) fix lots of bugs. The best way to get ahead in an organization like this is to check in lots of buggy code and fix it all, rather than taking the extra time to get it right in the first place. When you try to fix this problem by penalizing programmers for creating bugs, you create a perverse incentive for them to hide their bugs or not tell the testers about new code they wrote in hopes that fewer bugs will be found. You can't win.
With all due respect, I have to disagree with Joel. I've heard this same argument in many forms before:
- The developers will quickly learn to game the metrics – so they will be useless
- The cost of collecting the metrics will outweigh all possible benefit
- The metrics will lead us towards a lack of overall creativity within the discipline
- The metrics will create measurement dysfunction where the measurements exceed the teams places on their product quality and value goals (the point in the above snippet)
And he touches on all four points across the two posts. However, I must point out that something important is missing from his discussion. Measurement and metrics in and of themselves don't DO anything. They simply give us data to interpret. What Joel seems to miss (and many miss this point by the way) is that we CHOOSE how we interpret and act on the metrics. They don't do it for themselves.
Can all of the above side-effects occur? Certainly! Does the lack of measurement preclude the dysfunction alluded to above? I don't think so. Bad leadership is bad leadership, no matter where they're getting their data—from measures or from perceptions. However, the potential actions or negative outcomes shouldn't be the sole argument against measurement.
But the biggest problem I have with Joel's argument is that he implies ALL measurement is bad. He doesn't really differentiate methods, types or approaches.
I assert that metrics provide necessary visibility into the workings of your software development teams. Visibility isn't bad—it's fundamental to managing a key business process. Whether it's visibility into productivity, quality, performance, or achievement of value—software is in the end about business performance and delivering value.
A lack of visibility can lead to poor recognition of performance characteristics—causing organizations to make critical project adjustments too late or not at all. One need only to look at the Agile Methods community to realize that visibility isn't bad. In fact, this is a group that pushes immensely on traditional software methods and project management techniques. However, when it comes to visibility, they're proponents of things like daily status meetings, daily progress burn-down charts, and frequent (real-time) adjustments based on a team's velocity and performance.
To me, it sounds like measurements and visibility are what you make of them. Wouldn't you agree?
Here's another take on this topic by Andrew Binstock for SD Times: http://www.sdtimes.com/article/column-20061215-03.html
This post is continued here...
Previous Post:
6th Sense Analytics Offers Solution To Open Source Projects and Academic Groups Free of Charge
