Seeing Traditional Metrics in a New Light
There is a traditional metrics model that intends to guide metrics definition. It is the GQM or Goal - Question - Metric model. The premise goes like this -- all metrics should start with a clearly defined goal, and subsequently a set of questions surrounding the support of the goal. Finally, metrics are defined to support measurement and attainment of the goal.
Here's an example:
- Goal: Reduce post System Test defects to 50% of the last project
- Question:
- How many defects were released (pre and post System Testing) in the previous project?
- How many defects should be found prior to start of system testing?
- How many defects are found in system testing for the current release?
- Metric:
- Number of defects per component in the previous release
- Number of components in this project / release
- Number of defects found pre, in and post System Test in this release
Keep in mind that GQM is basically an iterative process and it truly helps to focus metrics collection efforts more narrowly towards business goals. This addresses two of the most common problems with metrics efforts -- a lack of a clear connection to business goals and a habit of focusing on too many metrics at once.
From a 6th Sense Analytics perspective, I feel that GQM, while not necessarily bad, is too simple a model for effectively analyzing your data. Why? Because it can drive binary, metrics event triggered thinking when analyzing the data and driving subsequent action. In my experience, software development analysis can not be programmed or managed by triggering events from metrics. There is too much complexity and too many variables to boil it down to a simplistic "if we reach this metric, we are ok or are in trouble" position. Instead, I propose a modified method for 6th Sense Analytics data analysis.
Instead of Goal, define an Exploration phase where you focus on collecting finely grained data from yourself and your team in meaningful time frames. By meaningful, I'm implying sufficient depth and breadth across your project activity. Minimally, that's a 1-2 month sample of data per relevant project.
Next, I suggest a deep analysis step where you ask a lot of questions and truly drill into your data. This is a Discovery phase where you look for patterns at different levels of the data. You also look for internal and external influences as you search for root causes.
Finally, based on your discovery, you'll plan project and team level Adjustments to influence your project direction and guide it towards successful release. The focus should be on small, high priority adjustments that can clearly be observed for their effectiveness.
As in GQM, Explore - Discover - Adjust is a highly iterative and agile process. The emphasis is towards making small, high potential adjustments in real-time. Then analyzing their effect and continuously improving and adjusting as your projects evolve. It's analogous to game film analysis by sports coaches. They're looking for patterns leading towards requisite adjustments that will improve overall team performance. In fact, this analogy perfectly describes our analytics. They're your "game films" as you improve your personal or team performance to that of truly championship teams. While it might sound corny or trite, deep data insight has that capability.
Previous Post:
BarCamp RDU
Next Post:
6th Sense Analytics sponsors Gartner App Development Summit
