The most common problem I’ve seen with software development teams is over commitment. Invariably individuals and teams are overly optimistic about what can be done in a certain time period. There are any number of reasons for this including arbitrary management deadlines and the team not pushing back, the developers desire to please, and just the fact that estimating in software development is hard.
Agile development teams are just as prone to this problem as any others. Every team I have helped transition to Agile has stepped into this bear trap almost immediately. And forewarning them doesn’t help. I now see stumbling into the trap as a valuable lesson and an essential step in the process of getting more mature about software development. I don’t mean maturity in the sense of the SEI’s Capability Maturity Model, I mean maturity in the sense of growing up, being realistic and accepting the limitations in themselves, their team and the organisation.
The difference about Agile, compared to traditional approaches to software development, is that Agile offers techniques to avoid the over commitment bear trap.
Agile Mechanisms for Controlling Overcommitment
The main Agile methods, as part of Agile Planning, all have a way to avoid overcommitment:
- Scrum – Team commits to output of a Sprint
- XP – Velocity in the Release Plan
- Kanban – Work in Progress (WIP) Limit
Scrum – Team commits to output of a Sprint
Overcommitment in Scrum is evident when a Scrum Team get to the end of a Sprint and they still have things on the Sprint Backlog that are not done.
In Scrum the Team commits to output of a Sprint (Schwaber & Beddle, 2002) although with the Scrum Update 2011 they now only "forecast" what they are going to deliver. The Team is meant to avoid overcommitment by using data from previous Sprints so the team has a sense of what is possible this time around. The Scrum Master is responsible for collecting this historical data and bringing it to the Sprint Planning session.
XP – Velocity in the Release Plan
Overcommitment looks similar in XP and Scrum but, in my opinion, XP has a more scientific approach to planning and hence to avoiding overcommitment.
Overcommitment in XP is obvious when an XP Team gets to the end of an Iteration and they still have things on the Iteration Plan that are not done.
XP’s measure for productivity is called "velocity" (Beck & Fowler, 2001). Velocity is the number of story points (or ideal days or whatever) delivered each iteration. The Release Plan shows user stories mapped to iterations using velocity as the constraint; an iteration should not have more user stories allocated to it than the team has demonstrated, with their velocity, that they can build in an iteration.
So the way to avoid overcommitment in XP is actively use velocity.
Kanban – Work in Progress (WIP) Limit
The Kanban Method (Anderson, 2010) comes at overcommitment from a completely different angle. Kanban works on the premise that higher Work in Progress (WIP) leads to longer Lead times (the time from order to delivery) and lower throughput (the amount done in a time period). Throughput is, like XP’s velocity, a productivity measure but it is not what is used to reduce overcommitment.
Overcommitment in Kanban is when a team has too much Work in Progress. There is either no Work in Progress limit or the limit is ignored or the limit is set at a high level.
The way to avoid overcommitment is to impose real Work in Progress limits, and the lower the better. The argument goes that with low Work in Progress, Lead Time reduces and Throughput increases. In other words do less at any one time to get more done over time. Personally I believe a team should also track Lead Time and Throughput just to be sure they going to get enough done in the time available.
Empirical Project Management
All of which I lump under the term empirical project management. "Empirical" because the planning is based on data not hope. In all cases it is about collecting data on previous performance to predict future performance. Beck and Fowler (2001) call this the principle of Yesterday’s Weather. I think of Work in Progress and Velocity as strong "Empirical" measures. Scrum is vague on the specific measure and just advise using historical data see if what the team commits to is feasible in a Sprint.
Unfortunately tracking empirical progress and using it effectively for planning are two distinct things. XPers are keen on talking about "courage" and I’m forced to use to the same term when encouraging teams to use the productivity data they have available. To use the data, rather than just collect it, the team need courage. The courage to admit to themselves what is really possible and the courage to give both management and the customer same message. Personally I happily pass on the real productivity data I’ve collected. Management and the customer might scream and shout at first but they’ll be grateful in the end; with better data they can make better decisions, particularly as it relates to the product.
References
Anderson, D. (2010). Kanban: Successful Evolutionary Change for your Technology Business. Blue Hole Press.
Beck, K, & Fowler, M. (2001). Planning Extreme Programming. Boston: Addison-Wesley.
Schwaber, K., & Beddle, M. (2002). Agile Software Development with Scrum. NJ: Prentice Hall.