Agile Project Estimating

Estimates are an essential part of Agile Project Planning.  Software estimation has always been problematic and people have proposed many different ways to do estimating.  Different methods are on a spectrum from formal to informal and from supposedly objective to seemingly subjective.  Different methods also get individuals estimate or groups.  And some methods estimate size and derive effort while others estimate effort directly.  Estimating in the Agile world has settled a certain approach which might be characterised as expert group estimation of size. this article covers Traditional Project Estimation, Agile versus Traditional Estimating, Estimating User Stories, Estimating Tasks, Contingency, and Agile Ballpark Estimates.

Traditional Project Estimation

“Agile Estimating and Planning” by Mike Cohn of Mountain Goat Software is the best book on Agile estimating there is. If you read one Agile book then read this one.

Wikipedia: Software Development Effort Estimation has a pretty good summary of cost and effort estimation techniques. The more formal of the traditional approaches include COCOMO and Function Point Analysis (FPA), but in reality most estimates are based on expert judgement.

Expert Judgement

When using expert judgement the estimates are meant to be based on historical data of previous projects (if you have it) but normally the experts just rely on their memory.  It seems most people aren’t good at this. According to Wikipedia: Software Development Effort Estimation there is only a 60-70% chance of the estimate being right even if the person is 90% sure of the estimate.  Unfortunately there are many Project Managers out there estimating on behalf of their teams and I personally put more faith in the estimates from Technical Leads than the estimates from, often non-technical, Project Managers.

Estimating Tasks on the Work Breakdown Structure

Typically, as part of traditional Project Planning, somebody will estimate the effort required for each task in the Work Breakdown Structure (WBS).  The sum total of these becomes the total effort of the project. This is problematic for two reasons:

  1. The expert’s estimate of each task is unlikely to be correct so the sum is going to be wrong too
  2. The WBS is unlikely to include all the tasks needed for the project

Compounding this problem is that in traditional Project Monitoring & Control the project manager is trying to track actual effort against these estimates and considers any deviation an issue.

Function Point Analysis

There are other approaches which try to quantify the size of the software rather than estimate the effort directly.  Function Point Analysis involves analysing the required functionality and identifying Logical Transactions, Entities, and Data Elements.  Each of these is worth a certain number of Function Points.  The total number gives the size of the system. This analysis is useful because it is tying the estimate to the intended Project Scope and the process essentially validates the thinking on the Project Scope. From this you can estimate (or calculate) the effort required. For the Project Planning you’re back to doing a traditional Work Breakdown Structure (WBS) and then doing traditional Project Monitoring & Control against that. The people doing FPA need to be trained as function points in one organisation are meant to be comparable to function points in another.  Unfortunately it takes quite a lot of effort to do a function point count.

I’ve used Function Point Analysis in the past and have a summary of Mark II Function Point Analysis (FPA) on this site.  I used it to cross check the expert judgement of my Technical Leads. If my estimate corresponded to theirs then there was a good chance we’d be ok.

Ballpark Estimates

Management types often ask for estimates before they commission a project. For example:

Q: How long will it take to build a CMS for client XYZ? What’s your ball park figure for this?

Such estimates give management and other stakeholders a feel for the size of a project but they are intrinsically unreliable because they are:

  • based on insufficient data
  • usually done by a single person
  • often done by somebody not in the development team (e.g. a Product Owner)

Management needs to understand the limitations of such an estimate. It is very important that the development team is not bound by any ball park estimates once the project starts. For example:

A: Roughly 10 weeks if we use the same team that did the original work.  Bear in mind this is a very, very rough estimate given our poor understanding of the requirements (a CMS could mean anything). Probably won’t be less than 8 weeks but could be 30 if they customer is demanding.  A proper Release Plan will give us more accuracy.

Despite these limitations ball park estimates are common, and useful, as they are the best that is available before a development team is formed.

Lines of Code (LOC)

Before moving on to Agile methods of estimating I’ve got to mention Lines of Code (LOC).  LOC is an intuitive and easy metric to capture for measuring software size. It was popular back the early days of computing but has lost favour amongst most practitioners.  None-the-less I was recently asked why I don’t advocate LOC as a metric (instead of those woolly Story Points) so I thought I’d address it here. Despite being intuitive and easy to measure LOC has some fairly major problems:

  • LOC focuses on the coding aspect of the work rather than the total development effort.
  • LOC is a poor productivity measure because one developer might be more productive than another despite using less code.
  • Different development languages have a different syntax and will generate different LOC measures for the same functionality.
  • Developers don’t write all the code.  This is a variation on the language thing and languages and the tools are tightly bound.  GUI Tools now generate a lot of the code.  If you count this code it seems the developer is being very productive.  If you don’t count it they seem very unproductive.
  • It isn’t clear what to do if your system uses more than one development language.
  • There is no standard definition of what a LOC is.
  • A person’s behaviour is influenced by how you measure them, so if productivity means more LOC the developers will give you more LOC, even if they don’t need to. and more code is more complex to maintain, which is a bad thing.
  • Last but not least … people aren’t very good at estimating LOC.

(See Wikipedia: Source Lines of Code if you want more detail on why not to use LOC).


Agile versus Traditional Project Estimation

DSDM and Crystal Orange have a fairly conventional approach to Agile Project Estimating. Project Manager and developers estimate effort derived from size. Crystal Orange uses estimated number of classes as the size measure, and DSDM typically uses Function Points. A Crystal Orange project manager might do a second estimate for comparison.

But increasingly the Agile community is using variations on the Agile Project Estimating from XP’s Planning Game.  The key features of this are:

  • Estimate system size (i.e. Story Points, Ideal Days, etc) rather than effort
  • Use expert judgement
  • But group estimation provides the triangulation
  • The people doing the work do the estimates
  • Velocity is the measure of capacity and productivity (it measures functionality delivered each Timebox)
  • Use Velocity to correlate size and effort
  • Decide size of team, hence effort, hence intended Velocity, and schedule accordingly
  • Measure actual Velocity to reconfirm schedule

Elements of this are familiar from the traditional methods. It estimates size rather than effort.  Like FPA it uses an abstract measure of size reflecting functionality rather than a concrete technical metric like LOC.  It is based on expert judgement but moderates the danger of this by using one of a variety of group estimation methods.  Unlike the traditional methods were the total effort estimate tells you how many people you need, in the Agile approach the you decide how many people to deploy and that tells you what you’ll deliver by the deadline. And finally in Agile the people doing the work do the estimates.


Estimating User Stories

User Stories on the Release Plan define the Agile Project Scope. The Agile team put estimates against the User Stories during the Release Planning Meeting of Agile Project Planning.

Story Points

Like many Agilists I’ve ended up using Story Points as a measure of User Story size.  Story Points are an abstract unit representing the difficulty of developing User Story.  It considers both bulk and complexity.  It is also a relative measure so a User Story of two Story Points is twice the size of a one Story Point User Story.

Story Points are similar to Function Points in that they are an abstract measure of size based on functionality.  But where characteristics of some functionality dictate how many Function Points are assigned it is the subjective opinion of the team which says how many Story Points are assigned.

When estimating a User Story the team has to work through:

  • What is involved in the building this User Story?
  • How big – considering both bulk and complexity – is this User Story relative to others the team has already developed?

As the team gets into development they will have more and more User Stories to compare to.  Initially, however, the team won’t have any existing User Stories to compare a new one to.  In this case the convention is to use Ideal Days to estimate the story (see below) and assume that one Ideal Day equals one Story Point.

Who Estimates User Stories

The team.  Estimating User Stories is a collective endeavour. There are different ways to do this including  triangulation, Planning Poker, paired comparisons, and voting.  I like Planning Poker (see below).

Velocity in Story Points

Agile processes promote a sustainable pace and one of the Principles Behind the Agile Manifesto refers to this:

Agile processes promote sustainable development.
The sponsors, developers, and users should be able
to maintain a constant pace indefinitely.

Velocity is the mechanism to find the sustainable pace.  Velocity is a measure of the teams capacity to delivery in a Timebox.  The Product Owner can’t cram more into a Timebox than the Velocity allows. The team’s Velocity for the Release Plan is measured in Story Points.

Velocity is the key to the empirical aspect of Agile Project Management.  The team’s actual Velocity is measured at the end of each Timebox by summing the the estimates for the User Stories completed in the Timebox.  “Completed” meaning “fully, totally, not even a little bit left undone, absolutely completed”.  This actual measure of Velocity is used to predict the Velocity for subsequent Timeboxes in the Release Plan.

See Agile Project Planning and Agile Project Monitoring & Control

Ideal Days

When XP first came out it promoted Ideal Days as the the estimating unit for User Stories (Beck, 2000). An Ideal Day is a work day without interruptions.  Imagine going into work on the weekend with nobody else around and no meetings to go to – that is the Ideal Day. In Release Planning developers decided how many Ideal Days it would take to develop each User Story.

I used Ideal Days for some time quite successfully, but they had one major drawback.  Stakeholders often got confused by the “day” bit of Ideal Day and started viewing these as actual day estimates.  The problem with this is it ignores the team’s capacity as measured by Velocity. The depressing thing is that often the people making that mistake were senor managers from my company who should have know better.  Story Points avoid this problem.

Ideal Days and Story Points are both abstract representations of software size. But the underlying question to get to the estimate differs; for Ideal Days it is “how long…?” and for Story Points it is “how big…?”.

None-the-less Ideal Days are useful when a new team starts a new project and hence doesn’t have existing User Stories to base their Story Point estimates on. Put another way, to get my first batch of Story Points I ask the team to estimate the Ideal Days.

Re-estimating User Stories

Normally you don’t re-estimate a User Story.  Changing the size of one User Story throws into doubt the relative size of other User Stories and may trigger a re-estimate of the entire Release Plan – something to be avoided.

I do on occasion re-estimate User Stories before they are developed, but only if the team’s understanding of the User Story has changed significantly. If your original understanding of the requirements was flawed and you get a new insight into the User Story then you can re-estimate.  Often this will result in a larger Story Point estimate.  For example, when one of my teams originally estimated the User Stories for the system they were working on they assumed the interface would be a fairly straight forward CRUD (Create Read Update Delete).  But it soon became apparent that the main data entry screen would involved some complicated processing that the team hadn’t been aware of.  We re-estimated the relevant User Stories based on the new understanding.

You shouldn’t, however, give a new Story Point value to a User Story that you’ve already started development of or finished.  Velocity is the mechanism to cope with under or over estimating.

Planning Poker

Planning Poker is a great way to facilitate estimating in Stories Points (Grenning, 2002). It is a variation on the Delphi method (Wikipedia: Delphi Method).  Basically the team members secretly propose an estimate for the User Story in question. The estimates are then compared.  If everybody agreed the estimate is accepted and assigned to the User Story.  If there is disagreement the team discuss the User Story and the upper and lower estimates before repeating the secret individual estimating. This can take several rounds before reaching a consensus.

You can read the original paper written by James Grenning on-line: Planning Poker or How to avoid analysis paralysis while release planning.  You can get packs of cards for Planning Poker from Mountain Goat Software.


Estimating Tasks

During Timebox Planning in Agile Project Planning the team identify and estimate Tasks to be done during the Timebox.

Ideal Hours

Whereas User Stories are estimated in Story Points, or sometimes Ideal Days, the Tasks are estimated in Ideal Hours.  Like Ideal Days this is uninterrupted time like when you work on the weekend.  Of course there is a short fall between Ideal Days and actual days.  You’ll only get between four and six Ideal Hours per working day.  The rest of the time is take up with email, meetings, answering the phone, managing other staff, staff development, etc.

Who Estimates Tasks

The simple answer is, the person doing the work.  But there are a couple of approaches depending on when people sign up for Tasks:

  1. If people sign up for Tasks in the Timebox Planning Meeting, then the person who signed up for the task estimates the task during the meeting.
  2. If people sign up for Tasks during the Timebox, the the entire Team estimates the task during the Timebox Planning Meeting.

The first approach promotes individual ownership and responsibility of Tasks and the second promotes team ownership and responsibility. I’m not sure whether either is better.

Velocity in Ideal Hours

You must track the team’s Velocity at the level of the Release Plan, i.e. in Story Points. You can also, if you want to, track the team’s Velocity at the Task level in Ideal Hours.  This can help if you think the team are over committing within the Timebox Planning Meeting of Agile Project Planning.

There are a couple of ways of doing this:

  • Track the completed tasks in the same way the Velocity for User Stories is tracked.
  • Let each team member just declare their Velocity for a Timebox based on how many hours they are available for work in that Timebox.

During the Timebox Planning Meeting you can use this data to validate how many Tasks the team is likely to be able to complete.

Re-estimating Tasks

Unlike User Stories, which are hardly ever re-estimated, Tasks are re-estimated often until the Task is complete, possibly daily.  Actually it is the work remaining that is re-estimated rather than the total work. Tracking work remaining on Timebox Burn Down Charts is a key element of Agile Project Monitoring & Control.


Contingency

You may have heard that there is no contingency in Agile projects.  It isn’t true.  Well not exactly.

The estimates for the User Stories created in Agile Project Estimating are meant to have no contingency. None. Zip. Nada. And I will repeat, no contingency. Agile estimates are meant to reflect the likely time to do the work. We expect that 50% of the time we’ll exceed the estimate and 50% we’ll run under (Cohn, 2006). On average we’ll be spot on. The lack of contingency on each piece of work means the Agile Team is less likely to dawdle.  However if you add up all those 50% probabilities that means we’ve only got 50% chance of being “on time” for the project as a whole, which is a problem. There are a few of ways to deal with this.

Low Priority User Stories as Project Contingency

All of the Agile methods put high priority requirements (i.e. User Stories) early in the Release Plan.  That means if the team runs out of time the high value items have already been delivered and it is only low priority items left.  In other words you don’t need contingency because the low value User Stories that don’t fit into the schedule are just dropped when the project ends.

MoSCoW: Low Priority User Stories as Timebox Contingency

DSDM uses a MoSCoW priority for all requirements.

M = Must Have
S = Should Have
C = Could Have
W = Would like to have but won’t get

In DSDM 60% of each Timebox is filled by Must Have items.  The remaining 40% of each Timebox has lower priority items assigned.  So if the team runs out of time within a Timebox it is clear which items have to get done (Must Have) and which are negotiable.

I’m not so keen on this approach.  The advantage is that the relative priority of all User Stories and Tasks is explicit.  The disadvantage of this method is that it deliberately schedules relatively low priority items early in the schedule thus risking Must Haves later in the project.

Project Buffer: 50% and 90% Estimates

The Project Buffer is described in Agile Project Planning and Cohn (2006). For larger projects you can consider doing two estimates for each User Story – a 50% and a 90% estimate – to derive a project level safety factor, the Project Buffer.

Agile estimates are normally pitched at the 50% chance, i.e. 50% of the time the actual time will be less and 50% of the time it will be more.  Traditional estimates include a contingency. Goldratt (1997) assumes there is a 90% chance that the work will be within these estimates. The difference between the 50% Agile estimate and the 90% estimate is the local contingency. You can use the 50% and 90% estimates to create a Project Buffer (see Agile Project Planning).

Personally I do add a Project Buffer but I don’t ask the team for two estimates because it adds overhead to estimating.


Agile Ballpark Estimates

Agile projects operate in the same context as traditional ones, so there are still management types asking for ballpark estimates.  The approach is just the same: give the estimate and caveat it heavily.  Beck (2000) suggested this kind of estimate be in Ideal Months rather than the Ideal Days of the Release Plan. Moving up a level of granularity helps highlight the imprecise nature of these estimates.  Personally I’ve used the same approach as described above and in Agile Project Planning but using a smaller number of technical people.


References

Agile Manifesto: Principles Behind the Agile Manifesto

Beck, K, & Fowler, M. (2001). Planning Extreme Programming. Boston: Addison-Wesley.

Cohn, M. (2006). Agile Estimating and Planning. Prentice Hall.

Goldratt, E. M. (1997). Critical Chain.

Grenning, J. (2002). Planning Poker or How to avoid analysis paralysis while release planning.
[Available on-line https://segueuserfiles.middlebury.edu/xp/PlanningPoker-v1.pdf.]

Wikipedia: Delphi Method

Wikipedia: Estimation in Software Engineering

Wikipedia: Software Development Effort Estimation

Wikipedia: Source Lines of Code

8 thoughts on “Agile Project Estimating

  1. Hi,

    I’ve returned to your website a number of times over the last few years, since first using Agile on projects in 2008 while working in local government. I’ve always found your site really thorough, useful and interesting… so thanks for sharing all your experience!

    I was wondering if you had any words of wisdom (or recommended reading) on how to handle estimation at the sales stage in a digital agency environment, particularly where it’s necessary to commit to the delivery of a useful (and competitive) set of features for a fixed price?

    My instinct is to get into the nitty gritty with the client to define the minimum marketable feature set, estimate that at high level, add contingency (in some form), and propose a fee. Ideally that would itself be a costed exercise, but is there a better way?

    I would appreciate knowing your thoughts if you have time to share them.

    Thanks again for offering a great Agile resource.

    David.

  2. David, glad you like the site.

    I would follow your instinct re fixed price sales. The only addition I’d make is to (1) base your estimates on historical data, if you have it (2) get two estimates to compare (different method, group and/or person).

    Steven

  3. Great article, thanks a lot for sharing!

    However, I’m still unsure how a project can be even remotely accurate using Agile, given that a backlog is a living thing.

    At best, in sprint 0 (for example), a backlog will contain mostly high-level epics, which will likely be sliced into multiple stories when they get to sprint planning stages.

    Are you suggesting the team sit down and estimate (in story points or whatever method) the whole backlog (or at least for release 1 or MVP)? Because this would be difficult for two reasons:

    1) The epics will be high-level and probably lack detail and therefore be inestimable.
    2) It will be a massive time-killer sitting down estimating each high-level epic and probably a waste of time.

    So HOW do you get an ETA for an MVP at the beginning of a project?

    • Paul. Thanks for the positive comment.

      You cannot get an ETA for a MVP at the beginning of a project without estimation. Of course not all teams will need an ETA, but if you do, then you need to estimate. And estimate with relatively little detail.

      The problem of estimation accuracy is not limited to Agile projects. Software development is hard to estimate and all projects have a fluid scope whether or not the participants want to admit that. Agile acknowledges these realities and provides some simple tools to cope.

      Even though the backlog (scope) is a living thing, at any particular moment the scope is fixed. And much of it can be estimated. The scope might change next week but that just means a new estimate might be necessary.

      Just because an epic is high level does not mean you cannot estimate it. High level epics get high level estimates. Often the ball park estimates I mention in the main article. Of course some Epics cannot be estimated but I’ve found these to be relatively few and I put them at the end of the schedule to reduce risk or sponsor a spike to flesh them out.

      Estimating the entire backlog does not necessarily have to take a long time – either elapsed or effort. The backlog for my current programme covers the work of a team of 25 for 8 months. The estimation took only 30 minutes. We re-estimate as necessary.

      For me the benefit of more than one person is triangulation; I don’t need everybody to get that. With a small team I might include everybody but with larger teams I only involve a handful of people. The important thing is to use the same estimation approach/people for the duration of the project.

      The problem is not estimating the MVP at the beginning of the project. The problem is knowing what the MVP is at the beginning of the project. My experience is the the vision of the MVP evolves over time. So the conversation might start with, “when will I get my MVP?” That is a question from product owner to dev. But the question ends up as “What will the MVP be given we have only 2 months more?” That is a question the product owner asks themselves.

      • Thanks so much Steven!

        I’m learning so much from your blog. Please, please, please do keep it up!

        And I love that despite the original post being written in 2008, you are still actively responding to questions! Brilliant!

        Thanks again,

        Paul

  4. Hi, While studying for my DSDM Foundation exam I came across your site, which I find very informative on estimating; so thanks for sharing.

    I also have a question. In this article, within “MoSCoW Prioritisation” you say,
    “I’m not so keen on this approach. The advantage is that the relative priority of all User Stories and Tasks is explicit. The disadvantage of this method is that it deliberately schedules relatively low priority items early in the schedule thus risking Must Haves later in the project.”

    The first and last sentences in this quote are confusing me because I thought that all the “Must Haves” are scheduled first, thus making it an advantage and seem to me a pretty good reason to like MoSCoW… or did I misunderstand you in this section…

    Thanks again…

    • In DSDM each time box is meant to have about 60% of Must Have with lower priority items filling up the rest of the time box. This means there is a buffer that ensures that the Must Haves get done in the time box. Other items take their chances. That, for me, is a local optimisation. It guarantees deliver of the particular Must Have in the time box but risks the total set of Must Haves by diluting the effort available to deliver them.

      • Thanks for the very quick response.

        OK – got you. At least I think I am agreeing with you. I had assumed that in the early part of a project I would only plan to work on the high-level Must Haves and not touch the Shoulds/Coulds, until later, thus ensuring all the Must Haves are covered off first.

        Perhaps I could look at it like this: within each Must Have, there may be detailed elements that are Shoulds and Coulds, which may be dropped accordingly within that timebox. If any are dropped; once all the detailed Must Haves from each high-level Must Have are done I’d look at any dropped detailed Shoulds/Coulds from the high-level Must Haves before moving onto the high-level Should Haves.

        Anyway, thanks again, you answered my question.

Leave a Reply

Your email address will not be published. Required fields are marked *