Measuring Working Software
A few months ago at a conference I heard something like "The only thing true measure of a software project is working code." While I cannot recall the exact quote, this was the basic gist and is very inline with "Agilist" thinking.
Principle 5 of the Agile Manifesto [ Note: I don't think they are in any specific order ] is:
"Working software is the primary measure of progress."
I heard it again at a conference this week: "We measure working software."
"Working" is in the eye of the beholder
Twitter has been barraged with complaints about outages recently, and admittedly some of that critique is deserved. In a recent heavily maligned blog post a Twitter engineer described a scenario that posed some challenges for the application. Essentially the application degraded under heavy load and scale -- a reasonably discreet scenario.
Does Twitter work? Of course it does. It's been reported to handle tens of thousands of requests per second. Do users like it? They certainly did like it, but now the same piece of software is yielding different experiences. They need some metrics that incorporate load to truly measure their success.
At the Rational Developer's Conference this week I attended a BOF session on reporting. Several customers in the audience voiced challenges with an existing product in that area called Rational SoDA. The product manager inquired for specifics about the challenges and the audience member replied, "have you ever used it?"
There are many software applications that "work" but are almost impossible to use. Usability is admittedly an art-form but it is measurable. I've conducted user studies and heuristic evaluations in the past, and the results are always mind-opening. Incorporating usability metrics into success criteria is a far better measure than simply meeting a specification.
I rarely hear what the true definition of "working" really is. Here's a list of things I think are NOT good measures of "working":
- Meeting specifications. Often the specification have fatal flaws.
- Revenue. A market fluctuation affects revenue and is not a true measure of whether the software works.
- User surveys. Users often change their mind and the same piece of software under load can deliver different impressions.
If it were only that easy
So the next time you want to say "we measure working software" please clarify what is meant. I'd be very worried about teams that only measure one of the aforementioned potential proxies for "working" software. Sadly, software development is more complex than a binary measure like this. And although I appreciate the concept of simplicity, some things ( like measuring application development activities ) are not simple.
The measures an organization should use are more diverse and typically specific to it. I'll explore a more appropriate and realistic set of measures for an organization in a future post. In the mean time, be wary of development teams that just measure working software.
Previous Post:
Is Your Team Welded Shut?
Next Post:
6th Sense Analytics Announces Upgrades to its Award-Winning Solution
ankur_gupta10 ( 2008.6.06 4:26 am )