Estimates are an essential part of Agile Project Planning.  Software estimation has always been problematic and people have proposed many different ways to do estimating.  Different methods are on a spectrum from formal to informal and from supposedly objective to seemingly subjective.  Different methods also get individuals estimate or groups.  And some methods estimate size and derive effort while others estimate effort directly.  Estimating in the Agile world has settled a certain approach which might be characterised as expert group estimation of size. this article covers Traditional Project Estimation, Agile versus Traditional Estimating, Estimating User Stories, Estimating Tasks, Contingency, and Agile Ballpark Estimates.

Traditional Project Estimation

Wikipedia: Software Development Effort Estimation has a pretty good summary of cost and effort estimation techniques. The more formal of the traditional approaches include COCOMO and Function Point Analysis (FPA), but in reality most estimates are based on expert judgement.

Expert Judgement

When using expert judgement the estimates are meant to be based on historical data of previous projects (if you have it) but normally the experts just rely on their memory.  It seems most people aren’t good at this. According to Wikipedia: Software Development Effort Estimation there is only a 60-70% chance of the estimate being right even if the person is 90% sure of the estimate.  Unfortunately there are many Project Managers out there estimating on behalf of their teams and I personally put more faith in the estimates from Technical Leads than the estimates from, often non-technical, Project Managers.

Estimating Tasks on the Work Breakdown Structure

Typically, as part of traditional Project Planning, somebody will estimate the effort required for each task in the Work Breakdown Structure (WBS).  The sum total of these becomes the total effort of the project. This is problematic for two reasons:

1. The expert’s estimate of each task is unlikely to be correct so the sum is going to be wrong too
2. The WBS is unlikely to include all the tasks needed for the project

Compounding this problem is that in traditional Project Monitoring & Control the project manager is trying to track actual effort against these estimates and considers any deviation an issue.

Function Point Analysis

There are other approaches which try to quantify the size of the software rather than estimate the effort directly.  Function Point Analysis involves analysing the required functionality and identifying Logical Transactions, Entities, and Data Elements.  Each of these is worth a certain number of Function Points.  The total number gives the size of the system. This analysis is useful because it is tying the estimate to the intended Project Scope and the process essentially validates the thinking on the Project Scope. From this you can estimate (or calculate) the effort required. For the Project Planning you’re back to doing a traditional Work Breakdown Structure (WBS) and then doing traditional Project Monitoring & Control against that. The people doing FPA need to be trained as function points in one organisation are meant to be comparable to function points in another.  Unfortunately it takes quite a lot of effort to do a function point count.

I’ve used Function Point Analysis in the past and have a summary of Mark II Function Point Analysis (FPA) on this site.  I used it to cross check the expert judgement of my Technical Leads. If my estimate corresponded to theirs then there was a good chance we’d be ok.

Ballpark Estimates

Management types often ask for estimates before they commission a project. For example:

Q: How long will it take to build a CMS for client XYZ? What’s your ball park figure for this?

Such estimates give management and other stakeholders a feel for the size of a project but they are intrinsically unreliable because they are:

• based on insufficient data
• usually done by a single person
• often done by somebody not in the development team (e.g. a Product Owner)

Management needs to understand the limitations of such an estimate. It is very important that the development team is not bound by any ball park estimates once the project starts. For example:

A: Roughly 10 weeks if we use the same team that did the original work.  Bear in mind this is a very, very rough estimate given our poor understanding of the requirements (a CMS could mean anything). Probably won’t be less than 8 weeks but could be 30 if they customer is demanding.  A proper Release Plan will give us more accuracy.

Despite these limitations ball park estimates are common, and useful, as they are the best that is available before a development team is formed.

Lines of Code (LOC)

Before moving on to Agile methods of estimating I’ve got to mention Lines of Code (LOC).  LOC is an intuitive and easy metric to capture for measuring software size. It was popular back the early days of computing but has lost favour amongst most practitioners.  None-the-less I was recently asked why I don’t advocate LOC as a metric (instead of those woolly Story Points) so I thought I’d address it here. Despite being intuitive and easy to measure LOC has some fairly major problems:

• LOC focuses on the coding aspect of the work rather than the total development effort.
• LOC is a poor productivity measure because one developer might be more productive than another despite using less code.
• Different development languages have a different syntax and will generate different LOC measures for the same functionality.
• Developers don’t write all the code.  This is a variation on the language thing and languages and the tools are tightly bound.  GUI Tools now generate a lot of the code.  If you count this code it seems the developer is being very productive.  If you don’t count it they seem very unproductive.
• It isn’t clear what to do if your system uses more than one development language.
• There is no standard definition of what a LOC is.
• A person’s behaviour is influenced by how you measure them, so if productivity means more LOC the developers will give you more LOC, even if they don’t need to. and more code is more complex to maintain, which is a bad thing.
• Last but not least … people aren’t very good at estimating LOC.

(See Wikipedia: Source Lines of Code if you want more detail on why not to use LOC).

Agile versus Traditional Project Estimation

DSDM and Crystal Orange have a fairly conventional approach to Agile Project Estimating. Project Manager and developers estimate effort derived from size. Crystal Orange uses estimated number of classes as the size measure, and DSDM typically uses Function Points. A Crystal Orange project manager might do a second estimate for comparison.

But increasingly the Agile community is using variations on the Agile Project Estimating from XP’s Planning Game.  The key features of this are:

• Estimate system size (i.e. Story Points, Ideal Days, etc) rather than effort
• Use expert judgement
• But group estimation provides the triangulation
• The people doing the work do the estimates
• Velocity is the measure of capacity and productivity (it measures functionality delivered each Timebox)
• Use Velocity to correlate size and effort
• Decide size of team, hence effort, hence intended Velocity, and schedule accordingly
• Measure actual Velocity to reconfirm schedule

Elements of this are familiar from the traditional methods. It estimates size rather than effort.  Like FPA it uses an abstract measure of size reflecting functionality rather than a concrete technical metric like LOC.  It is based on expert judgement but moderates the danger of this by using one of a variety of group estimation methods.  Unlike the traditional methods were the total effort estimate tells you how many people you need, in the Agile approach the you decide how many people to deploy and that tells you what you’ll deliver by the deadline. And finally in Agile the people doing the work do the estimates.

Estimating User Stories

User Stories on the Release Plan define the Agile Project Scope. The Agile team put estimates against the User Stories during the Release Planning Meeting of Agile Project Planning.

Story Points

Like many Agilists I’ve ended up using Story Points as a measure of User Story size.  Story Points are an abstract unit representing the difficulty of developing User Story.  It considers both bulk and complexity.  It is also a relative measure so a User Story of two Story Points is twice the size of a one Story Point User Story.

Story Points are similar to Function Points in that they are an abstract measure of size based on functionality.  But where characteristics of some functionality dictate how many Function Points are assigned it is the subjective opinion of the team which says how many Story Points are assigned.

When estimating a User Story the team has to work through:

• What is involved in the building this User Story?
• How big – considering both bulk and complexity – is this User Story relative to others the team has already developed?

As the team gets into development they will have more and more User Stories to compare to.  Initially, however, the team won’t have any existing User Stories to compare a new one to.  In this case the convention is to use Ideal Days to estimate the story (see below) and assume that one Ideal Day equals one Story Point.

Who Estimates User Stories

The team.  Estimating User Stories is a collective endeavour. There are different ways to do this including  triangulation, Planning Poker, paired comparisons, and voting.  I like Planning Poker (see below).

Velocity in Story Points

Agile processes promote a sustainable pace and one of the Principles Behind the Agile Manifesto refers to this:

Agile processes promote sustainable development.
The sponsors, developers, and users should be able
to maintain a constant pace indefinitely.

Velocity is the mechanism to find the sustainable pace.  Velocity is a measure of the teams capacity to delivery in a Timebox.  The Product Owner can’t cram more into a Timebox than the Velocity allows. The team’s Velocity for the Release Plan is measured in Story Points.

Velocity is the key to the empirical aspect of Agile Project Management.  The team’s actual Velocity is measured at the end of each Timebox by summing the the estimates for the User Stories completed in the Timebox.  “Completed” meaning “fully, totally, not even a little bit left undone, absolutely completed”.  This actual measure of Velocity is used to predict the Velocity for subsequent Timeboxes in the Release Plan.

See Agile Project Planning and Agile Project Monitoring & Control

Ideal Days

When XP first came out it promoted Ideal Days as the the estimating unit for User Stories (Beck, 2000). An Ideal Day is a work day without interruptions.  Imagine going into work on the weekend with nobody else around and no meetings to go to – that is the Ideal Day. In Release Planning developers decided how many Ideal Days it would take to develop each User Story.

I used Ideal Days for some time quite successfully, but they had one major drawback.  Stakeholders often got confused by the “day” bit of Ideal Day and started viewing these as actual day estimates.  The problem with this is it ignores the team’s capacity as measured by Velocity. The depressing thing is that often the people making that mistake were senor managers from my company who should have know better.  Story Points avoid this problem.

Ideal Days and Story Points are both abstract representations of software size. But the underlying question to get to the estimate differs; for Ideal Days it is “how long…?” and for Story Points it is “how big…?”.

None-the-less Ideal Days are useful when a new team starts a new project and hence doesn’t have existing User Stories to base their Story Point estimates on. Put another way, to get my first batch of Story Points I ask the team to estimate the Ideal Days.

Re-estimating User Stories

Normally you don’t re-estimate a User Story.  Changing the size of one User Story throws into doubt the relative size of other User Stories and may trigger a re-estimate of the entire Release Plan – something to be avoided.

I do on occasion re-estimate User Stories before they are developed, but only if the team’s understanding of the User Story has changed significantly. If your original understanding of the requirements was flawed and you get a new insight into the User Story then you can re-estimate.  Often this will result in a larger Story Point estimate.  For example, when one of my teams originally estimated the User Stories for the system they were working on they assumed the interface would be a fairly straight forward CRUD (Create Read Update Delete).  But it soon became apparent that the main data entry screen would involved some complicated processing that the team hadn’t been aware of.  We re-estimated the relevant User Stories based on the new understanding.

You shouldn’t, however, give a new Story Point value to a User Story that you’ve already started development of or finished.  Velocity is the mechanism to cope with under or over estimating.

Planning Poker

Planning Poker is a great way to facilitate estimating in Stories Points (Grenning, 2002). It is a variation on the Delphi method (Wikipedia: Delphi Method).  Basically the team members secretly propose an estimate for the User Story in question. The estimates are then compared.  If everybody agreed the estimate is accepted and assigned to the User Story.  If there is disagreement the team discuss the User Story and the upper and lower estimates before repeating the secret individual estimating. This can take several rounds before reaching a consensus.

You can read the original paper written by James Grenning on-line: Planning Poker or How to avoid analysis paralysis while release planning.  You can get packs of cards for Planning Poker from Mountain Goat Software.

Estimating Tasks

During Timebox Planning in Agile Project Planning the team identify and estimate Tasks to be done during the Timebox.

Ideal Hours

Whereas User Stories are estimated in Story Points, or sometimes Ideal Days, the Tasks are estimated in Ideal Hours.  Like Ideal Days this is uninterrupted time like when you work on the weekend.  Of course there is a short fall between Ideal Days and actual days.  You’ll only get between four and six Ideal Hours per working day.  The rest of the time is take up with email, meetings, answering the phone, managing other staff, staff development, etc.

Who Estimates Tasks

The simple answer is, the person doing the work.  But there are a couple of approaches depending on when people sign up for Tasks:

1. If people sign up for Tasks in the Timebox Planning Meeting, then the person who signed up for the task estimates the task during the meeting.
2. If people sign up for Tasks during the Timebox, the the entire Team estimates the task during the Timebox Planning Meeting.

The first approach promotes individual ownership and responsibility of Tasks and the second promotes team ownership and responsibility. I’m not sure whether either is better.

Velocity in Ideal Hours

You must track the team’s Velocity at the level of the Release Plan, i.e. in Story Points. You can also, if you want to, track the team’s Velocity at the Task level in Ideal Hours.  This can help if you think the team are over committing within the Timebox Planning Meeting of Agile Project Planning.

There are a couple of ways of doing this:

• Track the completed tasks in the same way the Velocity for User Stories is tracked.
• Let each team member just declare their Velocity for a Timebox based on how many hours they are available for work in that Timebox.

During the Timebox Planning Meeting you can use this data to validate how many Tasks the team is likely to be able to complete.

Re-estimating Tasks

Unlike User Stories, which are hardly ever re-estimated, Tasks are re-estimated often until the Task is complete, possibly daily.  Actually it is the work remaining that is re-estimated rather than the total work. Tracking work remaining on Timebox Burn Down Charts is a key element of Agile Project Monitoring & Control.

Contingency

You may have heard that there is no contingency in Agile projects.  It isn’t true.  Well not exactly.

The estimates for the User Stories created in Agile Project Estimating are meant to have no contingency. None. Zip. Nada. And I will repeat, no contingency. Agile estimates are meant to reflect the likely time to do the work. We expect that 50% of the time we’ll exceed the estimate and 50% we’ll run under (Cohn, 2006). On average we’ll be spot on. The lack of contingency on each piece of work means the Agile Team is less likely to dawdle.  However if you add up all those 50% probabilities that means we’ve only got 50% chance of being “on time” for the project as a whole, which is a problem. There are a few of ways to deal with this.

Low Priority User Stories as Project Contingency

All of the Agile methods put high priority requirements (i.e. User Stories) early in the Release Plan.  That means if the team runs out of time the high value items have already been delivered and it is only low priority items left.  In other words you don’t need contingency because the low value User Stories that don’t fit into the schedule are just dropped when the project ends.

MoSCoW: Low Priority User Stories as Timebox Contingency

DSDM uses a MoSCoW priority for all requirements.

M = Must Have
S = Should Have
C = Could Have
W = Would like to have but won’t get

In DSDM 60% of each Timebox is filled by Must Have items.  The remaining 40% of each Timebox has lower priority items assigned.  So if the team runs out of time within a Timebox it is clear which items have to get done (Must Have) and which are negotiable.

I’m not so keen on this approach.  The advantage is that the relative priority of all User Stories and Tasks is explicit.  The disadvantage of this method is that it deliberately schedules relatively low priority items early in the schedule thus risking Must Haves later in the project.

Project Buffer: 50% and 90% Estimates

The Project Buffer is described in Agile Project Planning and Cohn (2006). For larger projects you can consider doing two estimates for each User Story – a 50% and a 90% estimate – to derive a project level safety factor, the Project Buffer.

Agile estimates are normally pitched at the 50% chance, i.e. 50% of the time the actual time will be less and 50% of the time it will be more.  Traditional estimates include a contingency. Goldratt (1997) assumes there is a 90% chance that the work will be within these estimates. The difference between the 50% Agile estimate and the 90% estimate is the local contingency. You can use the 50% and 90% estimates to create a Project Buffer (see Agile Project Planning).

Personally I do add a Project Buffer but I don’t ask the team for two estimates because it adds overhead to estimating.

Agile Ballpark Estimates

Agile projects operate in the same context as traditional ones, so there are still management types asking for ballpark estimates.  The approach is just the same: give the estimate and caveat it heavily.  Beck (2000) suggested this kind of estimate be in Ideal Months rather than the Ideal Days of the Release Plan. Moving up a level of granularity helps highlight the imprecise nature of these estimates.  Personally I’ve used the same approach as described above and in Agile Project Planning but using a smaller number of technical people.

References

Agile Manifesto: Principles Behind the Agile Manifesto

Beck, K, & Fowler, M. (2001). Planning Extreme Programming. Boston: Addison-Wesley.

Cohn, M. (2006). Agile Estimating and Planning. Prentice Hall.

Goldratt, E. M. (1997). Critical Chain.

Grenning, J. (2002). Planning Poker or How to avoid analysis paralysis while release planning.
[Available on-line https://segueuserfiles.middlebury.edu/xp/PlanningPoker-v1.pdf.]

Wikipedia: Delphi Method

Wikipedia: Estimation in Software Engineering

Wikipedia: Software Development Effort Estimation

Wikipedia: Source Lines of Code