Making metrics meaningful
Mark Twain once reflected on the relative nature of numbers, popularizing the phrase: “There are three kinds of lies: lies, damn lies and statistics.”
Anyone whose ever commissioned or read the results of a research poll will know numbers can be sliced in different ways, to produce different meanings or interpretations. Twenty five percent of people like the taste of New Coke can be taken as quarter of consumers like the taste of New Coke or three quarters don’t like New Coke. (On this one, at least, we can say we know how consumers really felt.)
ComputerWorld’s Jerri Ledford has been ruminating on the nature of
meaningful metrics. Jerri’s conclusion is simple: the value of the metrics depends on the job at hand or the problem you are trying to solve. That’s not exactly rocket science, but its a useful check point.
According to one blogger responding to Jerri’s piece, to get meaningful metrics the metric definition process itself should be a journey not a destination. In other words, keep refining your metrics and re-modeling the benchmarks that use your numbers. He advises revisiting metrics quarterly or semi-annually noting the “relationship between information consumers and producers should not be taken lightly.”
That’s sound advice. Often, in software development, there’s a temptation to monitor and steer projects using benchmarks from third parties or by re-using metrics from other, older projects. No wonder things can, and do, go wrong. The reporting software might say everything is going well, yet you are experiencing slowdowns and the software is late. That’s no great surprise, if the metrics your system is calibrated on are old or irrelevant. It’s like navigating using a faulty compass.
For metrics to be meaningful and helpful in software projects, organizations should never assume they have all the metrics they need. Continue to harvest new data and refine the model that project assumptions and projections rely on. In short: never reach your destination, just enjoy the journey.
[metrics] [software] [development]
Previous Post:
6th Sense Analytics a Hit at CED Event
