What is Quality in PM?

Introduction

Source: Adam Farag, https://ecampusontario.pressbooks.pub/essentialsofprojectmanagement/part/chapter-10-project-quality-and-control/
This work is licensed under a Creative Commons Attribution 4.0 License.

Quality in PM

It's not enough to make sure you get a project done on time and under budget. You need to be sure you make the right product to suit your stakeholders' needs. Quality means making sure that you build what you said you would and that you do it as efficiently as you can. And that means trying not to make too many mistakes and always keeping your project working toward the goal of creating the right product.

Everybody "knows" what quality is. However, the way the word is used in everyday life is a little different from how it is used in project management. Just like the triple constraint (scope, cost, and schedule), you manage the quality of a project by setting goals and taking measurements. That's why you must understand the quality levels your stakeholders believe are acceptable, and ensure that your project meets those targets, just like it needs to meet their budget and schedule goals.

Customer satisfaction is about making sure that the people who are paying for the end product are happy with what they get. When the team gathers requirements for the specification, they try to write down all of the things that the customers want in the product so that they know how to make them happy. Some requirements can be left unstated. Those are the ones that are implied by the customer's explicit needs. For example, some requirements are just common sense (e.g., a product that people hold can't be made from toxic chemicals that may kill them). It might not be stated, but it's definitely a requirement.

"Fitness to use" is about making sure that the product you build has the best design possible to fit the customer's needs. Which would you choose: a product that is beautifully designed, well constructed, solidly built, and all-around pleasant to look at but does not do what you need or a product that does what you want despite being ugly and hard to use? You'll always choose the product that fits your needs, even if it's seriously limited. That's why it's important that the product both does what it is supposed to do and does it well. For example, you could pound in a nail with a screwdriver, but a hammer is a better fit for the job.

Conformance to requirements is the core of both customer satisfaction and fitness to use and is a measure of how well your product does what you intend. Above all, your product needs to do what you wrote down in your requirements document. Your requirements should take into account what will satisfy your customer and the best design possible for the job. That means conforming to both stated and implied requirements.

In the end, your product's quality is judged by whether you built what you said you would build.

Quality planning focuses on taking all of the information available to you at the beginning of the project and figuring out how you will measure quality and prevent defects. Your company should have a quality policy that states how it measures quality across the organization. You should make sure your project follows the company policy and any government rules or regulations on how to plan quality for your project.

You need to plan which activities you will use to measure the quality of the project's product. And you'll need to think about the cost of all the quality-related activities you want to do. Then you'll need to set some guidelines for what you will measure against. Finally, you'll need to design the tests you will run when the product is ready to be tested.

Quality and Grade

According to the International Organization for Standardization (ISO), quality is "the degree to which a set of inherent characteristics fulfill requirements." The requirements of a product or process can be categorized or given a grade that will provide a basis for comparison. The quality is determined by how well something meets the requirements of its grade.

For most people, the term quality also implies good value - getting your money's worth. For example, even low-grade products should still work as expected, be safe to use, and last a reasonable amount of time. Consider the following examples.

Statistics

Determining how well products meet grade requirements is done by taking measurements and then interpreting those measurements. Statistics - the mathematical interpretation of numerical data - are useful when interpreting large numbers of measurements and are used to determine how well the product meets a specification when the same product is made repeatedly. Measurements made on samples of the product must be within control limits - the upper and lower extremes of allowable variation - and it is up to management to design a process that will consistently produce products between those limits.

Instructional designers often use statistics to determine the quality of their course designs. Student assessments are one way in which instructional designers are able to tell whether learning occurs within the control limits.

Example: Normal Distribution

A refinery's quality control manager measures many samples of 87 octane gasoline, sorts the measurements by their octane rating into bins that are 0.1 octane wide, and then counts the number of measurements in each bin. Then she creates a frequency distribution chart of the data, as shown in Figure 10.1.

 

It is common to take samples - randomly selected subsets from the total population - and measure and compare their qualities, since measuring the entire population would be cumbersome, if not impossible. If the sample measurements are distributed equally above and below the centre of the distribution as they are in Figure 10.1, the average of those measurements is also the centre value that is called the mean and is represented in formulas by the lowercase Greek letter μ (pronounced mu). The amount of difference of the measurements from the central value is called the sample standard deviation or just the standard deviation.

The first step in calculating the standard deviation is subtracting each measurement from the central value (mean) and then squaring that difference. (Recall from your mathematics courses that squaring a number is multiplying it by itself and that the result is always positive.) The next step is to sum these squared values and divide by the number of values minus one. The last step is to take the square root. The result can be thought of as an average difference. (If you had used the usual method of taking an average, the positive and negative numbers would have summed to zero.) Mathematicians represent the standard deviation with the lowercase Greek letter (pronounced sigma). If all the elements of a group are measured, instead of just a sample, it is called the standard deviation of the population and in the second step, the sum of the squared values is divided by the total number of values.

Figure 10.1 shows that the most common measurements of octane rating are close to 87 and that the other measurements are distributed equally above and below 87. The shape of the distribution chart supports the central limit theorem's assumption that the factors that are affecting the octane rating are random and tend to offset each other, which is indicated by the symmetric shape. This distribution is a classic example of a normal distribution. The quality control manager notices that none of the measurements are above 88 or below 86 so they are within control limits, and she concludes that the process is working satisfactorily.

Quality Planning

High quality is achieved by planning for it rather than by reacting to problems after they are identified. Standards are chosen and processes are put in place to achieve those standards.

Measurement Terminology

During the execution phase of the project, services and products are sampled and measured to determine if the quality is within control limits for the requirements and to analyze causes for variations. This evaluation is often done by a separate quality control group, and knowledge of a few process measurement terms is necessary to understand their reports. Several of these terms are similar, and it is valuable to know the distinction between them.

The quality plan specifies the control limits of the product or process; the size of the range between those limits is the tolerance. Tolerances are often written as the mean value, plus or minus the tolerance. The plus and minus signs are written together, ±.

Defining and Meeting Client Expectations

Clients provide specifications for the project that must be met for the project to be successful. Recall that meeting project specifications is one definition of project success. Clients often have expectations that are more difficult to capture in a written specification. For example, one client will want to be invited to every meeting of the project and will then select the ones that seem most relevant. Another client will want to be invited only to project meetings that need client input. Inviting this client to every meeting will cause unnecessary frustration. Listening to the client and developing an understanding of the expectations that are not easily captured in specifications is important to meeting those expectations.

Project surveys can capture how the client perceives the project performance and provide the project team with data that are useful in meeting client expectations. If the results of the surveys indicate that the client is not pleased with some aspect of the project, the project team has the opportunity to explore the reasons for this perception with the client and develop recovery plans. The survey can also help define what is going well and what needs improvement.

Sources of Planning Information

Planning for quality is part of the initial planning process. The early scope, budget, and schedule estimates are used to identify processes, services, or products where the expected grade and quality should be specified. Risk analysis is used to determine which of the risks to the project could affect quality.

Techniques

Several different tools and techniques are available for planning and controlling the quality of a project. The extent to which these tools are used is determined by the project complexity and the quality management program in use by the client. The following represents the quality planning tools available to the project manager.

Cost-benefit analysis is looking at how much your quality activities will cost versus how much you will gain from doing them. The costs are easy to measure; the effort and resources it takes to do them are just like any other task on your schedule. Since quality activities don't actually produce a product, it is sometimes harder for people to measure the benefit. The main benefits are less reworking, higher productivity and efficiency, and more satisfaction from both the team and the customer.

Benchmarking means using the results of quality planning on other projects to set goals for your own. You might find that the last project in your company had 20% fewer defects than the one before it. You should want to learn from a project like that and put into practice any of the ideas they used to make such a great improvement. Benchmarks can give you some reference points for judging your own project before you even start the work.

Design of Experiments is the list of all the kinds of tests you are going to run on your product. It might list all the kinds of test procedures you'll do, the approaches you'll take, and even the tests themselves. (In the software world, this is called test planning.)

Cost of Quality is what you get when you add up the cost of all the prevention and inspection activities you are going to do on your project. It doesn't just include the testing. It includes any time spent writing standards, reviewing documents, meeting to analyze the root causes of defects, and reworking to fix the defects once they're found by the team: in other words, absolutely everything you do to ensure quality on the project. Cost of quality can be a good number to check to determine whether your project is doing well or having trouble. Say your company tracks the cost of quality on all of its projects; then you could tell if you are spending more or less than has been spent on other projects to get your project up to quality standards.

Control Charts can be used to define acceptable limits. If some of the functions of a project are repetitive, statistical process controls can be used to identify trends and keep the processes within control limits. Part of the planning for controlling the quality of repetitive processes is to determine what the control limits are and how the process will be sampled.

Cause-and-effect diagrams can help in discovering problems. When control charts indicate an assignable cause for a variation, it is not always easy to identify the cause of a problem. Discussions that are intended to discover the cause can be facilitated using a cause-and-effect or fishbone diagram where participants are encouraged to identify possible causes of a defect.

Monitoring for Active Control

When setting up monitoring and controlling systems for a new project, it's essential to keep in mind that not all projects are the same. What works for one project might not work for another, even if both projects seem similar. Also, the amount of monitoring and controlling required might vary with your personal experience. If you've never worked on a particular type of project before, the work involved in setting up a reliable monitoring and controlling  system will typically be much greater than the up-front work required for a project that you've done many times before. For projects you repeat regularly, you'll typically have standard processes in place that will make it easy for you to keep an eye on the project's overall performance.

Exactly which items you need to monitor will vary from project to project, and from one industry to another. But in any industry, you usually only need to monitor a handful of metrics. There's no need to over-complicate things. For example, when managing major construction projects for the Wisconsin Department of Transportation, Gary Whited, focused on these major items:

• Schedule
• Cost/budget
• Issues specific to the project
• Risk

He also recommends monitoring the following:

• Quality
• Safety
• Production rates
• Quantities

In other kinds of projects, you will probably need to monitor different issues. But it's always a good idea to focus on information that can serve as early warnings, allowing you to change course if necessary. This typically includes the following:

• Current status of schedule and budget
• Expected cost to complete
• Expected date(s) of completion
• Current/expected problems, impacts, and urgency
• Causes for schedule/cost overruns

As Whited explains, the bottom line is this: "If it's important to the success of your project, you should be monitoring it".

Note that measuring the percent complete on individual tasks is useful in some industries, where tasks play out over a long period of time. According to Dave Pagenkopf, in the IT world, the percent completion of individual tasks is meaningless: "The task is either complete or not complete. At the project level, the percent complete may mean something. You really do need to know which tasks/features are 100% complete. However sloppy progress reports can generate confusion on this point. 100% of the functions in a software product 80% complete is not the same as having 80% of the features 100% complete. A poorly designed progress report can make these can look the same, when they most definitely are not".

In addition to deciding what to monitor, you need to decide how often to take a particular measurement. As a general rule, you should measure as often as you need to make meaningful course corrections. For some items, you'll need to monitor continuously; for others, a regular check-in is appropriate. Most projects include major milestones or phases that serve as a prime opportunity for monitoring important indicators. As Gary Whited notes, "The most important thing is to monitor your project while there is still time to react. That's the reason for taking measurements in the first place".

Earned Value Analysis

Earned value analysis (EVA) is a monitoring and controlling process that compares project progress to the project baseline (original plan). EVA measures the performance of a project in terms of cost and schedule. It can tell the project team if a project is:

• Behind Schedule
• Ahead of Schedule
• Under Budget
• Over Budget

Before attempting the calculations involved in an earned value analysis of a project, it is important to understand the three basic inputs for EVA calculations. The three basic inputs are Planned Value (PV), Actual Costs (AC), and Earned Value (EV).

Planned Value (PV)

Planned Value - Refers to the expected cost that will be spent on the project over its lifetime.  For each activity, there is a total Planned Value (cost). More importantly, the amount that was going to be spent on each activity over time is also known.

Consider the information presented on Project Breakdown in Table 10.1. The amount that the project team thinks an activity will cost is called the planned value for that activity.

Actual Costs (AC)

The Actual Costs - this refers to the completed work - is the easiest of the inputs to understand. AC refers to any given activity cost at specific time. Actual costs don't reflect what was planned to be spent, but rather what was spent. This information is obtained from the accounting department and the data is based on invoices, paychecks and receipts related to the activity. While the project manager may have been planning to spend $78 on Activity 1 by the end of period one, the accounting department may inform him or her that the actual cost (AC) at the end of period one for Activity 1 is $50!!

However, the project manager still doesn't know if spending $50 on Activity 1 by the end of period one is good or bad, since he or she doesn't yet know how much work has been performed on Activity 1. The next basic input, earned value, will tell the project manager what percentage of the activity is completed and they will then know how well the project is progressing.

Earned Value (EV)

Earned Value - refers to the cost of work completed on an activity which can be found by multiplying the percentage of completed work for a given activity by the planned value for the same activity.

One thing to watch out for is that the calculation of EV is not time-dependent; it uses the total PV for an activity, not the value for PV at a certain point in time as found on a time-phased budget. For example, if Activity 1 is 100% complete at the end of period one, then EV = $50 × 100%, or EV = $50 On the other hand, if no progress has been made on this activity (0% complete), then $50 × 0%, or EV = $0.00

Cost Variance (CV)

CV is the first of two basic variances that can be calculated once EV, PV and AC have been determined for an activity or project. CV is simply the Earned Value minus the Actual Costs.

If CV is negative, that means that the project work is costing more than planned. If CV is positive, then the project work is costing less than planned. CV can be calculated for each activity, for segments of a project (for example a deliverable or sub-deliverable) or for the entire project. Watch the video: Calculating and Understanding Cost Variance for an explanation of how to calculate and interpret CV.

Schedule Variance (SV)

SV is the second of two basic variances that can be calculated once EV, PV, and AC have been determined for an activity or project. SV is simply the Earned Value minus the Planned Value.

If SV is negative, that means that less work has been performed than what was planned. If SV is positive, then more work has been done than planned.

Like CV, SV can be calculated for each activity, for segments of a project (for example a deliverable or sub-deliverable) or for the entire project. Watch the video: Calculating and Understanding Schedule Variance for an explanation of how to calculate and interpret SV.

<

Cost Performance Index (CPI)

While CV provides a dollar amount that reflects how much over or under the project is at a particular point in time, The Cost Performance Index (CPI) provides an indicator of the overall cost performance to date and a good idea of how the project work is trending with regard to cost performance. CPI is calculated as follows:

• A CPI that is < 1 means that the cost of completing the work is higher than planned.
• A CPI that is = 1 means that the cost of completing the work is right on plan.
• A CPI that is > 1 means that the cost of completing the work is less than planned.

Watch the video: Cost Performance Index (CPI) for a basic walk-through of CPI calculations and the interpretation of the results.

Schedule Performance Index (SPI)

While SV provided a dollar amount that reflected well the project is doing at turning dollars into completed activities on schedule, Schedule Performance Index (SPI) provides an indicator of the overall schedule performance to date.  Remember that there are some limitations on using money to measure time. Those limitations apply to SPI as well. To know whether a project is really behind or ahead of schedule, a project manager will also look at the planned start and finish dates, milestones, etc.

SPI is calculated as follows:

• An SPI that is < 1 means that the project is behind schedule.
• An SPI that is = 1 means that the project is on schedule.
• An SPI that is > 1 means that the project is ahead of schedule.

Watch the video: Schedule Performance Index (SPI) for a basic introduction to SPI calculations and the interpretation of the results.

Business Case

The Space Shuttle Challenger Disaster

In a detailed report by Jeff Forest(1996) at the Metropolitan State College, the factors that contributed to the Challenger disaster summed to environmental and human errors. The Space Shuttle Challenger 51-L was the 25th mission in NASA's STS program. On Jan. 28, 1986, STS 51-L exploded shortly after liftoff, destroying the vehicle and all of its seven crew members.

On the evening of January 27, 1986, Thiokol was providing information to NASA regarding concerns for the next day's planned launch of STS 51-l. Thiokol engineers were very concerned that the abnormally cold temperatures would affect the "O" rings to non-performance standards. The mission had already been cancelled due to weather, and, as far as NASA was concerned, another cancellation due to weather was unthinkable. Both parties were already aware that the seals on the SRB needed upgrading but did not feel that it was critical. Though the information provided by the Group Decision Support System  (GDSS) (with an associated expert system) showed that the "O" rings would perform under the predicted temperatures, Thiokol engineers questioned their own testing and data that were programmed into the GDSS. Thus, on the eve of the Challenger launch, NASA was informed that their GDSS had a flawed database.

At this point, NASA requested a definitive recommendation from Thiokol on whether to launch. Thiokol representatives recommended not to launch until the outside air temperature reached 53 °F. The forecast for Florida did not show temperatures reaching this baseline for several days. NASA responded with pressure on Thiokol to change their decision. NASA's level III manager, Mr. Lawrence Mulloy, responded to Thiokol's decision by asking, "My God, Thiokol, when do you want me to launch, next April?".

After this comment, the Thiokol representatives requested five minutes to go off-line from the GDSS. During this period the Thiokol management requested the chief engineer to "take off his engineering hat and put on his management cap," suggesting that organizational goals be placed ahead of safety considerations. Thiokol re-entered the GDSS and recommended that NASA launch. NASA asked if there were any other objections from any other GDSS member, and there was not.

First, Thiokol was aware of the "O" ring problem at least several months before the Challenger launch. However, the goal was to stay on schedule. NASA was made aware of the problem but it was "down-played" as a low-risk situation. Here is the first element of flawed information that was input into the GDSS. If NASA had been aware of the significance of the "O" ring situation, they probably would have given more credence to the advice of the Thiokol engineers' recommendations. However, the data transmitted during the GDSS meeting from Thiokol did say that it would be safe to launch for the forecasted temperatures. NASA was frustrated over the conflicting advice from the same source.

Second, the decision to delay a Shuttle launch had developed into an "unwanted" decision by the members of the Shuttle team. In other words, suggestions made by any group member that would ultimately support a scheduled launch were met with positive support by the group. Any suggestion that would lead to a delay was rejected by the group.

Finally, the GDSS was seriously flawed. As already mentioned, the database contained erroneous information regarding the "O" rings. Ideas, suggestions and objections were solicited but not anonymously.

The factors which led to the Challenger incident can be traced back to the inception of the shuttle program. NASA and Thiokol failed to maintain a quality assurance program through management support systems (MSS), as was initiated in the Apollo program, due to multiple source demands and political pressures. The GDSS used for the launch decision contained inaccurate data. Engineering members of the GDSS did not believe in the testing procedures used to generate the data components in the GDSS. And, the entire meeting was mismanaged.

Questions

1. What action should be taken to prevent this from happening again, in terms of management decision?
2. Discuss the constraints of this project and how quality is related to this constraint?

Key Terms

• Actual Costs: this refers to the completed work. Is the easiest of the inputs to understand.

• Benchmarking: means using the results of quality planning on other projects to set goals for your own.

• Cause-and-effect Diagrams: help in discovering problems based on variation.

• Control Charts: used to define acceptable limits.

• Cost of Quality: is what you get when you add up the cost of all the prevention and inspection activities you are going to do on your project.

• Cost Performance Index (CPI): provides an indicator of the overall cost performance to date and a good idea of how the project work is trending with regard to cost performance.

• Cost Variance (CV): is the first of two basic variances that can be calculated once EV, PV and AC have been determined for an activity or project. CV is simply the Earned Value minus the Actual Costs.

• Cost-benefit analysis: is looking at how much your quality activities will cost versus how much you will gain from doing them.

• Design of Experiments: is the list of all the kinds of tests you are going to run on your product.

• Earned Value: Refers to the cost of work completed on an activity which can be found by multiplying the percentage of completed work for a given activity by the planned value for the same activity.

• Earned Value Analysis (EVA): is a monitoring and controlling process that compares project progress to the project baseline (original plan). EVA measures the performance of a project in terms of cost and schedule.

• Planned Value: Refers to the expected cost that will be spent on the project over its lifetime.

• Quality: is "the degree to which a set of inherent characteristics fulfill requirements".

• Schedule Performance Index (SPI): provides an indicator of the overall schedule performance to date.

• Schedule Variance (SV): is the second of two basic variances that can be calculated once EV, PV, and AC have been determined for an activity or project. SV is simply the Earned Value minus the Planned Value.

• Statistics: the mathematical interpretation of numerical data - are useful when interpreting large numbers of measurements and are used to determine how well the product meets a specification when the same product is made repeatedly.

• Tolerances: are often written as the mean value, plus or minus the tolerance. The plus and minus signs are written together, ±.