Quality Management, in a project context, is concerned with having the right processes to ensure both quality product and a quality project. This article describes Traditional Quality Management, Agile versus Traditional Quality Management, Agile Product Quality, Agile Project Quality, Agile Product Quality, Agile Quality Assurance and Control, and Agile Quality Improvement.

Traditional Quality Management

Project Quality Management is one of the nine project management knowledge areas in the Project Management Body of Knowledge (PMBOK) from the Project Management Institute (2004).  But Quality Management is a much wider initiative originating in Japanese manufacturing.

Quality Policy, Management, Systems, Assurance, Control, etc, etc

There are a lot of terms which include the word quality so I thought I’d start with some definitions from ASQ: Glossary to set the scene:

Quality policy: An organization’s general statement of its beliefs about quality, how quality will come about and its expected result.

Quality management (QM): The application of a quality management system in managing a process to achieve maximum customer satisfaction at the lowest overall cost to the organization while continuing to improve the process.

Quality management system (QMS): A formalized system that documents the structure, responsibilities and procedures required to achieve effective quality management.

Quality assurance : All the planned and systematic activities implemented within the quality system that can be demonstrated to provide confidence that a product or service will fulfil requirements for quality.

Quality control : The operational techniques and activities used to fulfil requirements for quality.

Quality plan: A document or set of documents that describe the standards, quality practices, resources and processes pertinent to a specific product, service or project.

Quality audit: A systematic, independent examination and review to determine whether quality activities and related results comply with plans and whether these plans are implemented effectively and are suitable to achieve the objectives.

Get the feeling this could all become rather burdensome?  It can, but it doesn’t have to.

Quality Management

Surprisingly ASQ: Glossary left out Quality Improvement from its definition of Quality Management.  Wikipedia: Quality Management fills in the gap:

Quality management is a method for ensuring that all the activities necessary to design, develop and implement a product or service are effective and efficient with respect to the system and its performance. Quality management can be considered to have three main components: quality control, quality assurance and quality improvement. Quality management is focused not only on product quality, but also the means to achieve it.

Subsequent sections will look at the main components (quality control, quality assurance and quality improvement) plus product and project quality.

Product Quality

Project Perfect: Project Quality Planning defines Product Quality as being:

‘fit for purpose’ … It covers things like how well it meets the user’s needs, and the total cost of ownership.

The reference to ‘fit for purpose’ comes from J.M. Juran, one of the TQM gurus, who believed quality was fitness for use as defined by the customer.  This definition applies equally as well to Project Quality as to Product Quality.

Contrast Duran’s definition to that from another quality guru, Philip Crosby, who described quality as “conformance to requirements” (Crosby, 1979). The problem with a literal reading of this definition is that the requirements may not fully represent customer expectations or needs. Crosby claimed he meant requirements in a wider sense but some people take it to literally mean documented specification – something that would appeal to a lot of traditional software people.

Project Quality

Notice that Quality Management is concerned with both Product Quality and “the means to achieve it” which Project Perfect: Project Quality Planning calls Project Quality. Project quality is the “things like applying proper project management practices to cost, time, resources, communication etc. It covers managing changes within the project”.  Interestingly this is pretty close to what is covered by Agile Project Management.

Quality Assurance

Crosby (n.d.) makes the analogy that quality assurance is like a person possessing a driver’s license. Possessing the driver’s licenses provides some confidence that the person is a safe driver … but we don’t know for sure.

The emphasis of traditional quality assurance is producing a quality plan. A good quality plan, like a driver’s license, offers confidence that quality will result. Project Perfect: Project Quality Planning suggests that in a project context the plan should answer the following questions:

• What needs to go through a quality check?
• What is the most appropriate way to check the quality?
• When should it be carried out?
• Who should be involved?
• What “Quality Materials” should be used?

Other key aspects of quality assurance is producing the quality materials themselves.  These include standards, guidelines, checklists, templates, procedures, process, user guides, example documents, and the methodology.

I was surprised to discover that the PMBOK has a limited view of Quality Assurance. It separates Quality Planning from Quality Assurance, thus leaving Quality Assurance as just the proof than quality processes are being followed.

Quality Control

Continuing Crosby’s (n.d.) analogy, if quality assurance is like a person possessing a driver’s license, then quality control is actually checking that that person is a safe driver.

In a project context quality control is about implementing the quality plan, i.e. doing the quality checks described in the plan. Normally just doing the check isn’t sufficient and there needs to be proof the quality check took place. In other words the results need to be documented somehow.

Quality Improvement

My observation is that traditional project management is not concerned about improvement just about delivering the specific objectives of the project. But there are organisational level quality improvement initiatives. Praxion: ISO 9001 Definitions Translated Into Plain English says “quality improvement refers to anything that enhances an organization’s ability to meet quality requirements”. It covers product improvement, process improvement and people based improvement. There are lots of formal big name quality improvement methods out there including ISO 9004:2000, Kaizen, Six Sigma, TQM, and Taguchi.  I was even involved in writing such a method in the mid-1990s for software process improvement: SPICE, the ISO framework for the assessment of Software processes (Wikipedia: ISO 15504).

---

Agile Versus Traditional Quality Management

I believe that the Agile process is in tune with Duran’s definition of quality, i.e. “Fitness for use”, whereas the traditional approach favours a literal interpretation of Crosby’s “conformance to requirements”. Agilists want to “satisfy the customer” and deliver “valuable software”. Traditionalists want to deliver software that implements the contracted requirements specification. Traditional software people ignore the fact that requirements in the form of a specification may not fully represent customer expectations or needs as they evolve.

Another difference in the traditional and Agile approaches to quality is the attitude to documentation.  The Agile Manifesto values:

Working software over comprehensive documentation

That means you’re unlikely to find detailed requirements specification or quality plans, and fairly unlikely to find quality records, in an Agile project. But the Agile methods do have ways to document their quality activities … more on this later.

---

Agile Product Quality

Agile is very much concerned about product quality in the sense of “Fitness for use” rather than “conformance to requirements”.  The first of the Principles Behind the Agile Manifesto is:

Our highest priority is to satisfy the customer
through early and continuous delivery
of valuable software.

I like this because it manages to include both satisfied customers and a valuable product, both very quality oriented aspirations. I believe the other element of this principle, continuous delivery, is the key feature of Agile and this principle is very clearly tying continuous delivery to customer satisfaction.

---

Agile Project Quality

Project quality includes “things like applying proper project management practices to cost, time, resources, communication etc. It covers managing changes within the project” (Project Perfect: Project Quality Planning).  Given that definition I believe Agile Project Quality is addressed by doing Agile Project Management including:

• Agile Lifecycle / Agile Heartbeat
• Agile Roles and Responsibilities
• Agile Project Initiation
• Agile Project Scope
• Agile Project Planning
• Agile Project Estimating
• Agile Project Execution
• Agile Project Monitoring & Control
• Agile Quality Management
• Agile Risk Management
• Agile Change Management
• Agile Project Closure

Agile sometimes suffers because people confuse a poor implementation of the method with a limitation of the method itself. A poor implementation will leave out certain key Agile Project Management practices and quality will suffer.

---

Agile Quality Assurance and Control

I paraphrase outrageously, but Quality Assurance is planning activities to demonstrate quality and Quality Control is implementing those plans. To assure quality a traditional project management project manager would, if they were being thorough, produce a quality plan for the project. Agile project managers don’t do this because the Agile process itself provides the quality assurance and control. Agile builds quality in to the product through a combination of practices from Agile Project Monitoring and Control and Agile Project Execution.

Product Owner in the Team

The team is trying to build software that meet’s the Product Owner’s intent, rather than what they wrote down, so in the Agile world the Product Owner becomes part of the team and guides development. Sometimes the Product Owner cannot devote 100% of their time to the project, but this is a risk that traditional projects also face.

Releasable Software every Timebox

Each Timebox is meant to result in releasable code.  Although meeting the Product Owner’s expectations is a priority it is not the only criteria. Releasable software:

• Meets the Product Owner’s expectations given features asked for and the Timeboxes completed so far
• Meets agreed coding standards
• Has to best design for the currently implemented features (via refactoring)
• Is easily maintainable (via refactoring)
• Has been tested to the satisfaction of the team and relevant stakeholders

Unscheduled Product Reviews

Because the Product Owner is part of team they get opportunities to informally review the product. Despite the fact these reviews are informal, I encourage teams to document the results. Assign a scribe, take notes, type them up, and email them out. I always send the email to the Product Owner and Copy To the Technical Lead and other members of the team working in that area. For me this is part of Agile Change Management.

Scheduled Product Reviews

The product is formally reviewed in the Timebox Review Meeting at the end of each Timebox. You can schedule more formal reviews during the Timebox, and for longer Timeboxes (say 3 or 4 weeks) this is a good idea. DSDM mandates the results of these reviews are documented. I think this is a good idea but I’m not too tied to review documents. I recommend documenting the reviews in the same was as I described in the Unscheduled Product Reviews, i.e. an email to the relevant stakeholders.

Frequent Status Meetings

The team monitors project progress in the Daily Team Meeting. It is also an opportunity to review whether the team is following the agree approach.

Automated Unit Tests

Automated unit tests express the internal behaviours of the software. The total suite of automated unit tests become regression tests and are used to verify that the internals of the software continue to function as designed after subsequent changes. Agile teams aim for 100% pass rate for automated unit tests.

More mature teams write the tests before they write the code in Unit Test Driven Development.

Acceptance Tests

In DSDM there are documented quality criteria for all work products (DSDM, 2002), but looser forms of Agile restrict this to the User Stories. User Stories are the high level description of the external behaviours and business rules of your software. Each User Story has at least one acceptance test. The acceptance tests elaborate the brief description provided by the User Story.  They define the scope of the story and clarify the Product Owner’s intent with concrete examples. This clarifies the Product Owner’s intent, points the team in the right direction, and confirm when the intent has been met.

Where possible Acceptance Tests should be automated.  As with the automated unit tests, the suite of automated acceptance tests become regression tests, validating that the customer’s intent continues to be met by the software after each change to the code.  You won’t automate all Acceptance Tests – it won’t be possible to automate some and others won’t be cost-effective to automate. But if you want the test repeated as part of a Regression Test then it is better to automate. If you can write a test so that a person can repeat the steps consistently then you can probably write a automated test and let the computer repeat the steps even more consistently.

More mature teams write the tests before they write the code in Acceptance Test Driven Development.

Test Driven Development

In Test Driven Development the tests become the specification. Because the tests are automated there is no ambiguity, the software either passes the test or it fails.

Regression Testing

A regression test is the repeat of an earlier test.  Usually that means Unit Tests and Acceptance tests. Regression tests ensure that changes to the software have not broken good code.  My experience is that if regression tests are manual they don’t happen.  It is with regression testing that the real value of test automation is shown.

Exploratory Testing

Exploratory Testing uses un-scripted tests to quickly identifying new types of problems with the software. Show stoppers (such as system crashes or unhandled errors) are usually fixed immediately. Less serious problems might be deferred.  This testing might also reveal new User Stories to be scheduled into later Timeboxes.

Specialist Testing

Extra testing activities like performance testing are scheduled in the same way as User Stories.

Code Review

The Two Pairs of Eyes approach provides a peer review of code to check it follows agreed coding standards, conforms to design guidelines and is easily understood by developers other than the author. This can be either through pair programming or more traditional code reviews/walkthroughs.

Code Metrics

Although not mandated by any of the Agile approaches, some Agile teams, like some traditional teams, collect metrics about the quality of the code. Examples are code-coverage (amount of code covered by unit tests), conformance to maintainability design principles (e.g. Lack of Cohesion of Services, Normalised Distance from the Main Sequence), and language-specific metrics.·

Continuous Integration

Continuous Integration is about maintaining quality all the time, throughout the project. It involves automatically integrating and running a regression test every time somebody does a check in. This is likely to happen several times a day. Running an automated regression test frequently means defects are highlighted soon after they are introduced (i.e. when the build goes Red, i.e. fails). The team’s top priority is to get the build Green again.

Informative workspaces

The Project’s Informative Workspace is the primarily place to show data on Project status. Typically it has Burn Down Charts, the Timebox Plan, perhaps the current build status, and anything else that might be a particular concern at the time (e.g. quality metrics such as test coverage). Essentially it is a way to monitor product and project quality on a daily basis.

Scheduled Project Reviews

The Retrospective part of the Timebox Review Meeting is a chance to review the project as whole. Problems are addressed as User Stories in subsequent Timeboxes.

---

Agile Quality Improvement

One of the Principles Behind the Agile Manifestois:

At regular intervals, the team reflects on how
to become more effective, then tunes and adjusts
its behavior accordingly.

Retrospectives are the mechanism most Agile teams use for reflection.  During a Retrospective the team looks at how well the Timebox went and what they can do different. The high priority changes become User Stories to go into the Release Plan for implementation. Often this is the process to implement new Agile processes.

---

References

Agile Manifesto: Principles Behind the Agile Manifesto

ASQ: Glossary

Crosby, P. B. (n.d.). Quality is Free – if you understand it. Philip Crosby Associates II, Inc. [Available on-line http://www.wppl.org/wphistory/PhilipCrosby/QualityIsFreeIfYouUnderstandIt.pdf.]

Crosby, P. (1979). Quality is Free. New York: McGraw-Hill.

The DSDM Consortium. (2002). Framework for Business Centred Development: Handbook for DSDM Version 4.1. Author.

Project Management Institute. (2004). A Guide to the Project Management Body of Knowledge (PMBOK Guide) [3rd Edition]. Author.

Project Perfect: Project Quality Planning

Wikipedia: Quality Management