Recently people at work have been asking my advice on how to run post implementation reviews of major programmes so I thought I’d write up my thoughts. I believe there are two types of post implementation review and I recommend doing both. The first type happens in Project Closure and the second happens after the project has finished, i.e. Post Project.

Reviewing the Project during Project Closure

Project Closure is just that. The time when the team is wrapping up and going home. This is an ideal opportunity to review how the project went. The process is a bit similar to retrospectives run during the project (see Agile Project Monitoring and Control) but there are two key differences:

• The scope of the review is the entire lifetime of the project rather than a subset (like the most recent timebox)
• The team won’t be around to implement any actions’

Basically get everybody together for a workshop with the following agenda:

1. Set the stage: Introductions and Agenda
2. Gather data: Timeline – What significant events happened when?
3. Generate insights: Things that went well, went badly, things we learned, novel ideas, and things to thank people for (“bouquets”)
4. Action plan: Specific actions to do to improve the process in future – Which ones to actually do – Who will do them – When
5. Close: Summarise actions, thanks, bye

The timeline is essential as it reminds everybody of the key events during the project. It includes when key people joined, when people left, the major milestones and/or releases. Whatever. It serves not only as a memory jogger but also an opportunity to vent.

The insights are in some ways a rehash of the timeline but with a focus on good stuff and bad stuff. Typically teams want to keep doing the good stuff and fix some, but not necessarily all, of the bad stuff.

The action plan refines this into specific action points. But most importantly it must include a filtering process to highlight the most important, significant, valuable actions. During a normal retrospective the team decides which of the action points they will implement in the near future. But during Project Closure the team has to decide which actions to recommend that the organisation undertakes, after the project team is gone. It is better to restrict the recommendations to a few key ones rather than everything. The fewer they are, the more likely they will get done.

For larger projects and programmes I recommend getting somebody outside the team to run the review as this is more likely to bring out any tensions between sub-teams. The reviewer will have to run several of the review workshops, more-or-less one for each sub-team. I have, when in the role of reviewer, supplemented the workshops with interviews of key stakeholders and mass questionnaires. Interviews can be formal or just "having coffee". I’m less keen on questionnaires. They have their place but don’t return as rich data as a workshop or other face-to-face interaction will. Questionnaires are also slightly problematic as responses are most likely from people with a chip on their shoulder. All of this material has to collated into a final report. Actually I find two reports to be better – one being a high level summary (often a powerpoint presentation) and leaving the gory details for a document. Where ever possible I quote the words of the participants in the reports to make them more real.

Reviewing Objectives Post-Project

The other type of review happens long after the project team are gone, very much Post Project. Often 3-6 months. The question is, did the project meet the original objectives? Basically go back to the Project Brief (Pre-project) and check that the promised outcomes of the initiative were actually met.

The specific objectives will vary from project to project, but in my field projects are often about delivering a product or significant product enhancement. So the review would focus on the product. Did it meet the business need? Did it achieve the return on investment? If a web based product, did it it get the target reach? The Product Owner is responsible for organising the review and accountable for the results.

As I mentioned in Pre-project, Marty Cagan of the Silicon Valley Product Group suggests we provide good answers to nine questions before pursuing an opportunity. These questios define both the objectives and the constraints of the project so it is worth reviewing how things panned out in reality. The nine questions of the Opportunity Assessment are:

1. Exactly what problem will this solve? (value proposition)
2. For whom do we solve that problem? (target market)
3. How will we measure success? (business metrics)
4. What alternatives are out there? (competitive landscape)
5. Why we are best suited to pursue this? (our differentiator)
6. Why now? (market window)
7. How will we deploy this? (gentle deployment strategy)
8. What is the preliminary estimated cost? (small/medium/large)
9. What factors are critical to success? (solution requirements)

Programmes focus more on the delivery of organisational benefits so that is what the review will concentrate on. Was the savings target realised? Was productivity increased? Are people collaborating more? What ever.