I often present at conferences on a variety of topics. A habit I've developed is polling the audience for informal surveys. One topic of interest for me is the Pareto Principle or the 80:20 Rule. What often surprises me is the number of software folks who've never heard of the principle, let alone use it as a tool within their project work.
Let's look at the principle outside of software first as a way of defining it. If you had a Toyota Prius warehouse that contained all of the cars parts, you would notice that 20% of the parts take up 80% of the space in the warehouse. You would also notice that 20% of the parts make up 80% of the overall vehicle cost.
That is the essence of the Pareto Principle or 80:20 Rule—that 20% of something has an 80% overall impact. Now let's bring that principle back into software projects.
Defect Clustering
There are a couple of ways to leverage Pareto in your software projects. The most obvious one involves defects. The principle clearly comes into play in defect clustering or the natural tendency for 20% of the components (or features or architectural layers) of any product to produce 80% of the defects. If you can clearly identify these fault prone components up-front, then you can focus your project efforts towards cleaning them up—thus reducing project risk and improving overall product quality.
From a pure testing perspective, you should focus a majority of your testing efforts on these areas. Initial testing should focus on partitioning defects and identifying the initial Pareto state for the application. Once you've determined your initial risk state—testing should be earlier, more frequent, and more thorough in these areas. You also need to monitor the Pareto distribution over time, as it will typically change as the application evolves.
From a process perspective, you can also influence more unit testing and inspections as a way to improve the overall quality in these areas from a development perspective. The point is though – focus your energies where they have the most overall impact to the product.
Code Construction
Pareto takes another turn in simple construction. It implies that 20% of the product will take 80% of the time as well. This then should affect your composition strategy and project planning for development. You'll probably want to front-load those 20% components early in the development schedule and clearly identify those tasks—throughout the team.
You might also want to assign some of your best software developers to work on those components.
Team Capabilities
Another area where Pareto rears its head, and it's probably the most controversial, is within your team capabilities themselves. If you truly buy into it, then 20% of your staff is performing 80% of the work. Plus or minus J I think we all realize that high impact developers are much more productive than others, but applying the 80:20 Rule to it really amplifies the importance.
Nonetheless, I'm convinced the rule works here as well. And teams need to become comfortable with this relationship and leverage it to their advantage. In this case, critical work assignments, both in Pareto area, overall scope and complexity, should be skewed toward the best resources.
So how do you implement Pareto analysis?
The challenge with Pareto in all of the above cases is early identification of "the 20%" and how to do that. Arguably the easiest application is defects. However, even there you need to identify the right number of Pareto categories, then correctly assign incoming defects into the categories before you can apply the technique. Only then do you gain insight into the distribution of defects over relevant component areas so that you can take action. It also takes additional time and effort to setup for analysis. I would argue its worth it, but it does take more time.
It gets much more hairy when you try and apply Pareto to your teams. Some of the challenges include:
- How do you identify your high impact performers?
- And identify the specific technology areas where they are the strongest?
- Then map them back to tasks on the project at-hand?
- While not demoralizing the remainder of your team?
- And assign them challenging work as well?
- That together drives the project towards optimum completion, while properly using your teams capabilities?
Applying Pareto within teams for work assignments requires a much more subtle hand and strong leadership experience. Again though, the payback can be tremendous as you effectively focus the skill evels of your team uniquely towards specific project challenges.
As it turns out, 6th Sense Analytics can help you explore Pareto in all of the above cases. You can view individual and team data as it applies to all of your projects. Active Time and Flow Time metrics will illustrate team capabilities towards contributed and produced project artifacts--including defects, construction and team performance. We amplify insight at 6th Sense and that insight can help you in Pareto Analysis as you make much more powerful adjustments within your teams. And as you're analyzing the data for Pareto, please keep in mind our EDA model as a framework for that analysis.
Proxy based estimation (PROBE) is a technique for estimating that is derived from the Personal Software Process (PSP) from the SEI. The idea is attractive because it provides guidance for how a developer, or a team for that matter, can categorize their historical estimate data so that it can be meaningfully used for future estimates.
The technique goes like this –
You analyze your work and categorize it into the units or types that you typically deliver your project features or other work in. I'll give you a simple example for my own activity. I do quite a few presentations in PowerPoint. I categorize my typical slides into simple text, table or data driven examples, high degrees of graphics or charts, and automation. Each area, for me at least, implies more complexity and thus more time to create.
These become my work proxies which drive my historical data collection.
Once you define your proxy index table, you start capturing estimate and actual data for the various categories of work that you typically perform. Obviously the data becomes more valuable as –
- You collect more data samples for each proxy for your work
- Refine the proxies in size and complexity to align with your work artifacts
- Your experience in interpreting the historical data—towards estimate formulation increases
Here is a tabular example representing my earlier proxy:
| Type of slide / Complexity |
Simple |
Moderate |
Complex |
| Simple Text |
< 5 lines |
< 10 lines |
> 10 lines |
| Moderate Research |
1 source |
2-5 sources |
> 5 sources |
| Objects, Tables & Graphs |
< 2 |
< 5 |
< 10 |
| Automation |
< 20% |
< 50% |
Near 100% |
If someone asked me to estimate an approximate 100 slide training class, I would first decompose the class into types of slides I would generally expect it to contain. For example, 20 simple slides, 60 of moderate complexity, and 20 with a lot of complexity and automation might be a reasonable decomposition for a standard 1 day workshop.
Next I would evaluate estimates and actuals from my historical data to gather some samples of my own execution. Using this data, I could extrapolate and make some reasonably accurate estimates on how long it will take me to pull together the class. As I completed the workshop, I would add the estimate + actual pairs into my database for future reference.
As you gather more historical data, you start to narrow the estimate uncertainty for any specific type of task. I hope you see how this relatively simple example can be extended into your individual and team software estimation.
But how does this map to 6th Sense Analytics?
You have finely grained Active Time data available at an individual and team level that accurately represents all work activity. First, Active Time becomes your actual measure for any estimate—easily captured via queries or status reports. You'll also want to start estimating in Active Time as well.
There are several advantages to this. First, Active Time naturally masks out all of the non-work activity such as meetings, sick time, etc. So you won't have to factor them into the estimates. Second, since 6th Sense Analytics is collecting your Active Time, either at an individual or group / project level, you can easily track progress to the initial estimate and gather actual effort upon completion.
If you do any research on PROBE you'll notice a lot more to it than I mentioned here. For example, there are some linear regression calculations that assist in narrowing the estimate uncertainty from the samples within your historical estimate and actual data sets -- this significantly increases the accuracy of your overall estimates.
Doug Schepers, our SVG guru/evangelist in residence at 6th Sense Analytics, will be giving three presentations in support of SVG in the Triangle area of North Carolina:
Karl Wiegers is a well know expert on software requirements, inspections and metrics and he's written several leading books on these topics. He's also written a paper entitled – Software Metrics: 10 Traps to Avoid that I thought would provide meaningful guidance to the 6th Sense Analytics crowd. His website is www.processimpact.com and the paper can be found here.
I do want to list the traps and provide some additional commentary or mapping to our data model. Here are the traps –
- Lack of Management Commitment
- Measuring Too Much, Too Soon
- Measuring Too Little, Too Late
- Measuring the Wrong Things
- Imprecise Metrics Definitions
- Using Metrics Data to Evaluate Individuals
- Using Metrics to Motivate, Rather than to Understand
- Collecting Data That is Not Used
- Lack of Communication & Training
- Misinterpreting Metrics Data
Disrupting the Cost Model of Traditional Metrics
There are a couple of the traps that I don't think apply as much to the 6th Sense Analytics approach, for example, #3 and #8. We have a finely grained model that enables you to collect as much data as possible for analysis and historical trending.
The pure cost of the data, beyond simply installing the desktop sensors, is virtually free, both cost to collect and cost per quantity of data. From this perspective, 6th Sense Analytics truly disrupts traditional software development metrics methods which typically have a high cost of collection and analysis. There are also the ongoing costs associated with evolving the metrics.
There are also a few traps whose cautions should be amplified; I've bolded these items in the above list. Since our collections model is so tightly coupled to detailed engineer work activity, it is truly incumbent on leadership to communicate the drivers behind introduction of the tool and their intent for usage. They need to honestly elaborate on:
- What are they going to collect and pay attention to?
- What are the drivers behind their decision to collect data?
- What are the overall short and longer term goals?
- What will they do with the metrics and, more importantly, what they won't do?
All these questions need to be proactively addressed and clearly communicated within the team.
We very much recommend starting analysis at a group or project level when first introducing 6th Sense Analytics. Only very slowly should analysis move down to the individual level from a leadership point of view and only then from the perspective of coaching or guiding the team towards improvement and not for performance improvement leading towards rewards or punishment.
A Community
Another important differentiator with 6th Sense Analytics is the fact that you're truly not alone with your metrics. We're there to help you in rolling the solution out within your team and in guiding your analysis. We also foster a growing community of users for you to learn from and collaborate with. You see, you'll gain insights not just from your own teams, but from a broad community of software developers who are being led by data and insights.
Tekrati's latest opinion piece has summarized the pain that sub-par market research departments cause their businesses: they "confuse managers, cloud decisions, hamper forceful execution and consume valuable resources."
Tekrati is talking about the practice of bringing in external IT analysts to advise on proposed new product and services. Tekrati, though, could easily be referring to the impact poor research and insight has on application development and on the growing practice of outsourcing large portions of the application lifecycle management chain.
Read More
In an earlier post I mentioned the book Slack as a reference point behind Flow Time. In this post, I want to expand on that a bit more because Slack Time is actually a construct that crosses Active & Flow Time and I think it's an important concept to consider within your teams.
In Slack, Tom DeMarco explores several aspects of productivity in modern technical teams. He speaks in terms of the following –
- The quality of work that we perform
- The time we have for "thinking"
- The absence or reduction of context switching
Quality of our Work
We all know that there is a sweet spot in our normal working patterns. If we work too little, we really don't meet project or company objectives and that doesn't satisfy the man. However, there is the counterintuitive position of working too hard and actually getting less done. For example, I've seen this demonstrated time and again within Endgames when making defect repairs. Typically, the most experienced engineers will take on far too many defects repairs in order to meet a release milestone. While they're very experienced, the overload and extended hours will cause them to introduce regressions, partial repairs and other side effects that will often derail the projects focus and schedule.
The key point here is that busyness doesn't always equate to effectiveness!
From a 6th Sense Analytics perspective, you not only want to monitor Flow Time, but also look for excessive periods of Active Time within the team. You want to correlate the two to insure that people are not overworking, burning out and beginning to regress in their effectiveness. Considering the very nature of Active Time, seeing an average of 25-30 hours of active time implies that the team is working 50-60 hours per week, which may lead in the long term to burnout.
Enabling "Think Time"
Think time maps to our notion of Flow Time. However, you also need to consider quantifiable aspects of Active Time by activity type. In software development think time is often related to reading and studying code and interfaces, reading online documentation, doing web research, and asking questions of other developers.
In real terms, you want to separate true code construction from code thinking as much as you can in Active Time to determine how you inpidually or at the team level are investing your time. Minimally, you should categorize view, design, and review time as think time and contrast it against Flow Time and other categories of Active Time. You're looking for balance in work effort that allows for thinking, while also achieving a sustainable pace.
The real point here is that you want to enable your teams to have time to consider alternatives and not be driving them towards 100% coding or excessive, heads down overtime. It's just not effective in gaining the creativity and innovation you need from yourself or your team.
Detecting & Reducing Context Switch Time
This is another area where DeMarco spends some significant time in discussion and where you can have a high impact within your teams. Multi-tasking of resources is one of the primary contributors to developer inefficiency. Each project context shift reduces inpidual efficiency by approximately 15%. That is from a raw Active Time perspective and the impact on flow is even greater.
You can detect multi project impacts by reviewing project or group Active Time and looking for inpiduals who are actively contributing to two or more projects. You'll want to reduce those supporting multiple projects as much as possible. If engineers must work on multiple projects, you'll want to establish a pattern of reduced context switches and immersion periods between the switches in order to maximize productivity.
Remember that context switches can also be driven by new hires, transfers and for team leaders. In all of these cases, the context switch impact can be detected, measured and should be reduced whenever possible.
