The process efficiency of executing a story on a Scrum team is the most important metric for team performance, because a team can easily double velocity in one sprint by driving process efficiency up over 50%. A team from India asked me what KPIs they should use and I told them just use process efficiency. They drove it to 80% within three days and on the fourth day had completed all work for a two-week sprint.
This phenomenon has been documented previously in an IEEE paper Scrum and CMMI - Going from Good to Great: Are You Ready Ready to Be Done Done C. Jakobsen and J. Sutherland, in Agile 2009, Chicago, 2009.
Process efficiency is defined as the real work time divided by the calendar time to get to done. The required data is easily available in any Scrum tooling. What we want to see is average process efficiency for stories completed in a sprint in real time. We want to abandon hours as a reporting tool for Scrum teams as data on over 60,000 teams in a Rally survey shows that the slowest teams use hours. The fastest teams use small stories, no tasking, and no hourly estimation. How can we estimate process efficiency for these teams?
Here is a simple way to calculate process efficiency for one story. If the velocity of the team is 50 and the story is 5 points then the real work time for the story can be estimated to be 5/50 of a sprint. If the story is started at the beginning of the sprint and finished at the end of the one week sprint then it uses 5 days of a 5-day sprint or 1 sprint. If we divide 5/50 by 1 we get 10%. Make that number more than 50% and you will double velocity.
Our Webside Scrum team is implementing this for our company and there are a few questions that come up:
- Do we just count business work periods in the denominator? What about weekends?
- We use an interrupt buffer? How do we handle interrupts.
- Webside process efficiency is over 100% because we have lots of small stories that get implemented really fast. Should we use a weighted average so that larger stories count more?
As we implement this I will update the blog on our approach. Even discussing how to implement a process efficiency metric has caused my Scrum team to introduce more discipline into backlog management and execution of stories.
In order for a team to improve process efficiency, they will have to stop multi-tasking and execute the Swarming: One-Piece Continuous Flow pattern. This is well known to radically increase the flow of story completion and is fundamental to lean production at Toyota.
Recently I met with Frank Verbruggen who is working on a Ph.D. thesis on the process efficiency of Agile teams. He wrote a nice blog on the topic after our meeting.
I am discussing with Lars Bachmann of http://www.demicon.de the option of creating a plugin for Jira to show real time process efficiency of story completion inside a sprint. This way Product Owners could tell if they are getting twice as many points!
That would be a great Plugin. Structures does something like that, but you have to update it manually
That will be really helpful!
Dr. Sutherland,
It was great to chat with you on this topic during the Minneapolis Scrum Gathering. The few minutes we spent at the vendor booth were incredibly informative and a highlight of the conference for me. As a result, I plan to explore the impact of highlighting process efficiency with my team.
Thanks much,
Matt Papenfuss
Interesting topic. A few additional questions…
1) Does the manner in which velocity is calculated matter? In other words are we using a 5-week moving average, yesterdays weather, average since inception…etc? What is recommended?
2) The example above walks through calculating process efficiency for an individual story, but how do we go about calculating it in aggregate? Should all completed points for a given sprint be used? Should process efficiency be calculated at the story level and then aggregated for all completed stories in a given sprint?
Hi Dennis,
Great questions:
For (1), we use a 3-week moving average. As long as you are consistent in your measurements, any reasonably velocity calculation would work to an extent, but I would steer away from “since inception”. You don’t want your teams general velocity improvement to drown out your process efficiency measurement.
For (2), we calculate it at the story and then do a weighted average by story points, but you could also just do a straight average and still get good results.
Finally we are working on a follow up to this topic with data and more explicit detail on the calculations. Hopefully this will be read mid or late Summer.