Update: See Process Efficiency – Adapting Flow to the Agile Improvement Effort the first of several papers to be published. By definition, Lean means that process efficiency is greater than 25%. The name Scrum derives from watching lean hardware teams. A new team can drive process efficiency to 80% in three days by implementing the Swarming pattern. There will be an iPhone and Android app available soon that will connect to Jira and display a Scrum team's average process efficiency per story.
The process efficiency of executing a story on a Scrum team is the most important metric for team performance, because a team can easily double velocity in one sprint by driving process efficiency up over 50%. A team from India asked me what KPIs they should use and I told them just use process efficiency. They drove it to 80% within three days and on the fourth day had completed all work for a two-week sprint.
This phenomenon has been documented previously in an IEEE paper Scrum and CMMI - Going from Good to Great: Are You Ready Ready to Be Done Done C. Jakobsen and J. Sutherland, in Agile 2009, Chicago, 2009.
Process efficiency is defined as the real work time divided by the calendar time to get to done. The required data is easily available in any Scrum tooling. What we want to see is average process efficiency for stories completed in a sprint in real time. We want to abandon hours as a reporting tool for Scrum teams as data on over 60,000 teams in a Rally survey shows that the slowest teams use hours. The fastest teams use small stories, no tasking, and no hourly estimation. How can we estimate process efficiency for these teams?
Here is a simple way to calculate process efficiency for one story. If the velocity of the team is 50 and the story is 5 points then the real work time for the story can be estimated to be 5/50 of a sprint. If the story is started at the beginning of the sprint and finished at the end of the one week sprint then it uses 5 days of a 5-day sprint or 1 sprint. If we divide 5/50 by 1 we get 10%. Make that number more than 50% and you will double velocity.
Our Webside Scrum team is implementing this for our company and there are a few questions that come up:
- Do we just count business work periods in the denominator? What about weekends?
- We use an interrupt buffer? How do we handle interrupts.
- Webside process efficiency is over 100% because we have lots of small stories that get implemented really fast. Should we use a weighted average so that larger stories count more?
As we implement this I will update the blog on our approach. Even discussing how to implement a process efficiency metric has caused my Scrum team to introduce more discipline into backlog management and execution of stories.
In order for a team to improve process efficiency, they will have to stop multi-tasking and execute the Swarming: One-Piece Continuous Flow pattern. This is well known to radically increase the flow of story completion and is fundamental to lean production at Toyota.
Recently I met with Frank Verbruggen who is working on a Ph.D. thesis on the process efficiency of Agile teams. He wrote a nice blog on the topic after our meeting.
Dr. Sutherland, could you share the JIRA plugin you mentioned here, as well as the latest formula that Scrum Inc. uses for calculating process efficiency? The original post mentions points/velocity for real work time and then divided by the # of sprints? Has this been proved to be reliable calculation? What if a story doesn’t take the entirety of the Sprint to be completed?
I think the conversation is about maximising throughput by minimising WIP and optimising capacity in tune with WIP and throughput. Swarming is an everyday process in agile teams when build breaks other team members subordinate their work towards breaking the bottleneck activity. But I’m not sure how process cycle efficiency could be improved by cosmetic changes to metrics. PCE is about maximising value addition and ruthlessly eliminating non value addition.
Do you have more information about this topic?
We have a few more items on it in progress, we recently coauthored this paper which is a great starting point for more information: https://34slpa7u66f159hfp1fhl9aur1-wpengine.netdna-ssl.com/wp-content/uploads/2018/08/process-efficiency-1.pdf
This document is great; is there a newer version of it yet? Also, did the Process Efficiency plugin for Jira that was mentioned in the earlier post ever happen?
Thanks!
Interesting topic. A few additional questions…
1) Does the manner in which velocity is calculated matter? In other words are we using a 5-week moving average, yesterdays weather, average since inception…etc? What is recommended?
2) The example above walks through calculating process efficiency for an individual story, but how do we go about calculating it in aggregate? Should all completed points for a given sprint be used? Should process efficiency be calculated at the story level and then aggregated for all completed stories in a given sprint?
Hi Dennis,
Great questions:
For (1), we use a 3-week moving average. As long as you are consistent in your measurements, any reasonably velocity calculation would work to an extent, but I would steer away from “since inception”. You don’t want your teams general velocity improvement to drown out your process efficiency measurement.
For (2), we calculate it at the story and then do a weighted average by story points, but you could also just do a straight average and still get good results.
Finally we are working on a follow up to this topic with data and more explicit detail on the calculations. Hopefully this will be read mid or late Summer.
Hi Alex,
Does 100% really mean the team just finishes the story in the exact way they expected without any unexpected wait or interruption time when doing the story work?
For example, if the team velocity is 10 and they forecasted 2 5-point stories for a 10-day sprint. For story 1, it takes 5 days to complete and PCE = ((5/10)/(5/10))=100%. On the remaining 5 days of the sprint, they finish story 2 and PCE is also 100%. Then the sprint ends.
So if there is a story less than 100% and the team can still finish both stories in the same sprint, when does that mean? What can cause this to happen? Is it that the effort for both stories are really not the same even they were both pointed at 5? If so, maybe we need to look at the way we estimated. Besides this, what can other factors be? Trying to learn more before introducing this because I found that none of the stories is 100% (ranging from 13% to 56%) in the sprint but the team finished what they have forecasted.
Thanks.
Two stories in a sprint means that one person cannot complete the story in half a sprint. For most teams this means the story has a high probably of not getting done. A good rule of thumb is story sizes small enough to be half a sprint for one person. So if a 5 person development team is doing two week sprints with a velocity of 50, half a sprint for one person is no more than 5 points.
Hi Alex:
Does below line in the article above mean make the average PCE of all the stories in the sprint more than 50%? Or it is about a story? If it is a story only for more than 50%, you will double velocity?
“Make that number more than 50% and you will double velocity”
Thanks.
The best paper documented doubling velocity for all teams that have Ready backlog is the IEEE Digitial Library paper Scrum and CMMI – Going from Good to Great: Are You Ready Ready to Be Done Done https://34slpa7u66f159hfp1fhl9aur1-wpengine.netdna-ssl.com/wp-content/uploads/2014/05/Scrum-CMMI-Going-from-Good-to-GreatAgile.pdf
This paper documents a “Flow” metric which is the average process efficiency per story.
Dr. Sutherland,
It was great to chat with you on this topic during the Minneapolis Scrum Gathering. The few minutes we spent at the vendor booth were incredibly informative and a highlight of the conference for me. As a result, I plan to explore the impact of highlighting process efficiency with my team.
Thanks much,
Matt Papenfuss
I am discussing with Lars Bachmann of http://www.demicon.de the option of creating a plugin for Jira to show real time process efficiency of story completion inside a sprint. This way Product Owners could tell if they are getting twice as many points!
That would be a great Plugin. Structures does something like that, but you have to update it manually
That will be really helpful!
We have implemented this metric in a way that doesn’t require any additional manual tracking (as long as team consistently updates statuses).
https://marketplace.atlassian.com/apps/1218809/quantify-timespent