Traditionally, management wants two things on any project, control and predictability. This leads to vast numbers of documents and graphs and charts. Months of effort into planning every detail in advance, so there will be no mistakes, no cost overruns, and things will be delivered on schedule.
The problem is that never actually happens. All that effort on planning, on trying to restrict change, on trying to know the unknowable is wasted. Every project involves discovery of problems and bursts of inspiration. Trying to restrict a human endeavor of any scope into color coded charts and graphs is foolish and doomed to failure. It’s not how people work, and it’s not how projects progress. It’s not how ideas reach fruition, or how great things are made.
Instead, it leads to frustrated customers not getting what they want. Projects delayed, over budget, and in many cases ending in abject failure. This is especially true for teams involved in the creative work of crafting something new. And many times management won’t learn of the glide path towards failure until millions of dollars and thousands of hours have been invested.
Scrum asks why does it take so long, and so much effort to do stuff? And why do we are we so bad at figuring out how long and how much effort things will take? The Cathedral at Chartres took 57 years to build. Aren’t you sure that the stonemasons looked at the bishop at the beginning of the project and said, “Twenty years, max. Probably be done in fifteen.”
Scrum embraces uncertainty and creativity. Because that is how people work. It places a structure around the learning process, enabling teams to assess both what they’ve created, and just as importantly, how they created it. The Scrum framework harnesses how teams actually work, and gives them the tools to self-organize and rapidly improve both the speed at which work is created, and the quality within it.
At its root, Scrum is based on a simple idea. Whenever you embark on a project, why not regularly check in, see if what you’re doing is heading in the right direction, and if it is actually what your customer wants? And check in on whether there are any ways to improve how you’re doing what you’re doing, any way of doing it better and faster, and ask, what is keeping you from doing that.
That’s what is called an ‘Inspect and Adapt’ cycle, better known as the Retrospective in Scrum. Every little while, stop doing what you’re doing, review what you’ve done, and see if it is still what you should be doing, and if you can do it better. (Learn more about a great Retrospective.)
Scrum got its name from a seminal 1986 Harvard Business Review article by Professors Hirotaka Takeuchi and Ikujiro Nonaka titled “The New New Product Development Game.” The two looked at the best teams across industries, automotive, photocopier makers, restaurants and printer manufacturers. They asked, what makes teams work? What separates great teams from everybody else? They said great teams have three characteristics, they are autonomous, they organize themselves, they are cross-functional, with all of the knowledge and skills necessary to develop and produce a product, and they are transcendent, they are seeking to do something greater than themselves. These cross-functional teams moved the design and production towards completion in a series of overlapping phases. They compared this to the ‘scrum’ formation in rugby, where the whole team "tries to go the distance as a unit, passing the ball back and forth".
The framework has grown out of decades of work and research in how people work together, how they think, and how organizations enable, demand and receive excellence, or crush creativity and wonder why. Scrum takes into account everything from why humans excel at comparative prediction, but are abject failures in absolute estimates, to the latest research on what motivates talented, sophisticated workers, to extensive investigation into what makes some teams excel and others wallow in mediocrity, to why multitasking hurts even as it feels like it helps. It’s based not in the way people think they work, but in actually how they do. It distills learning from teams at Toyota and Apple, the US Army and great sports teams.
The end result is that good teams can improve their performance by a factor of four. They also produce exactly what the customer wants, rather than wasting effort on things no one needs. It also leads to empowered, happier employees, who want to stay with their company, and who want to accomplish something truly remarkable.