Measuring the Value of Quality

I suspect many of you, like me, are always looking for the fact-based argument for quality. When pressed, I typically fall back on the cost-of-quality (poor quality) argument, and cite the statistics I’ve heard over the years.

Twenty cents of every dollar of revenue in manufacturing is lost to poor quality. Thirty cents of every revenue dollar in service is lost to poor quality. Seventy cents in healthcare, and I’ve never heard a number cited for government but everyone agrees it’s north of 70%. Often these numbers will capture the interest of the discussion enough that I can delve into a more elaborate explanation for what contributes to the cost of quality other than scrape and rework. I often end my epistle with something like, “And this 20% (30%, 70% or more) is available to every company without raising prices, or finding new customers. Twenty percent to the bottom line!”

I’m then troubled when lines don’t form at the door for more of this quality stuff. I just don’t get it.

I often wonder, then, what the unrealized benefit of quality is to society. Of course, I admit that I don’t have tools that are sophisticated enough to undertake the answer to the question and then move on to wondering who might have those tools. Do economists have tools that would make a compelling argument for the cost of poor quality at a societal level?

Well, short of an answer to my exact question, economists Albert N. Link, and John T. Scott (University of North Carolina, and Dartmouth College, respectively) used their economic tools to measure the net social value of the Baldrige Performance Excellence Program. It’s an interesting study and I recommend it to you.

And for two reasons. First, in the explanation of methodology they describe the “counterfactual evaluation method,” they used (pages 6 to 8), which goes along with the figures 1 and 2 (pages 23 and 24). Together the explanation and the figures offer a very interesting account of the market impact of improved quality performance. I’ve not seen this account before.

Secondly, the study describes the net social value of the Baldrige program. In 2001, when this study was first performed, the net social value had been found to be 207:1. Ten years later, with the expansion of the program to include healthcare, education, and the not-for-profit sectors, the value has grown to 820:1. Now, to be careful, this study is aimed at deriving the value of the Baldrige Program and the use of “public funds,” but the insights related to the larger question of the social value of quality are strong.

As governments and policy makers around the world look for approaches and tools to improve their economies – this study provides compelling evidence that quality is a known, and proven, contributor to improved and sustainable performance.

This entry was posted in Baldrige, Current Events, Quality and tagged , , , , , . Bookmark the permalink.

15 Responses to Measuring the Value of Quality

  1. Rodrigo Carrasco says:

    Thanks for your comments about the cost of Quality
    Actually, just now I’m working in analyzing such costs in my manufacturing site in Doumen China. However, so far even that I have included all scrap, and customer returns, FAE cost, Metrology maintenance, plus all the QA Manpower salaries, the total cost is no more than 2% of the revenue.
    I wonder if you can share with me an updated template of all related costs which we need to include in the analysis.

    I will appreciate your feedback and wish you a Happy New Year!

  2. Sam Khattab says:

    Deming told us about (PDCA) 50 years ago and Juran told us about cost of Quality (PVIEF*) 30 years ago. Management is still hard of hearing and would rather pay the “820″ later than the “1″ now!!!
    * (Prevent, Validate, Internal & External Failure)

    Q. professional need to find their “Tahrir Square” and camp out for a while!!!!

  3. Sam Khattab says:

    Deming told us about (PDCA) 50 years ago and Juran told us about Cost of Quality (PVIEF*) 30 years ago. Management is still hard of hearing and would rather pay the “820″ later than the “1″ now!!!
    * (Prevent, Validate, Internal & External Failure)

    Q. Professionals need to find their “Tahrir Square” and camp out for a while!!!!

  4. Kam Gupta says:

    This is a great reminder that the seeds of a good quality system are already sowed from the inception in almost every organization… the thing is that they are either not allowed to germinate well or not nourished well for a great harvest. This is what Baldrige companies do well. They nourish well for a great harvest.

    The studies have done a good job of linking historical data to a desired outcome, thus making a strong case for quality.

    On a deeper level though, somehow, we seem to need someone else to tell us how great the quality effort can be, if done right. Sometime scientific approach to real and recurring quality benefits are overlooked in favor of ‘what is out there’.

    Journey back to around 1985:
    Taguchi, in his book “Offline Quality Control’ – talked about two core things – the quality loss (QL) function (loss to the society due to variable quality characteristics), and parameter design – creating environmentally immune designs to minimize the QL. Leadership that embraces these two fundamental beliefs – robust designs and consistent products – create far superior product and customer delight more sustainably, and at a much lower overall cost, than those which have ‘programs’ or strategies driven by external criteria, not necessarily focused on robust design. In my experience with the review, conversations, case studies, and site visits to over 200 companies including many ‘Baldrige’ and similar companies, very few organizations pay attention to the concept of design robustness – whether product, process or service. A robust process design without a robust product design is virtually ineffective in bringing high quality to the market and customer delight.

    I see a far greater possibility and profitability in combining the two approaches. If with a great quality program, a company can benefit to the tune of, say, 30 percent, imagine how much more can it mine by combining the scientific and strategically focused approaches.

    As we move to more technology integration with the ‘conventional’, I see a greater probability of the two techniques coming together in the near future.

    Kam Gupta/ Executive Coach and Consultant

  5. In my book QUALITY QUANTIFICATION THEORY , (Economy Goodtime Measurement System) the Qulity concept is ultimately defined.
    To me the quality of something is the concept that my minde measures to finde out how much good is that thing.
    I think that this is tru for everybody, is’t it?
    The unet measurement of quality is “1 good”…..

  6. Like a well designed product and / or process, it is desirable that attitude of ‘first time right’ is ingrained right from the formative years.
    This would neccessiate far less intervention of ‘external programs’ when a person reaches a stage in the life where he /she is also ‘produces’ various ‘products’ or ‘services as well.

  7. Great post, thank you for writing it. I find a similar phenomenon happens when we talk about rework on IT projects (now accounting for up to 60% of project labor costs!) and yet management is so short term focused that they would rather, as one earlier commenter stated, pay the $1 now, and the $820 later.

    Good communication also suffers from a similar fate – the benefits are usually seen as major to a project AND intangible (which means “fluff” to engineers and technical professionals.) Somehow we have to wake up our industry to the costs of non-communication and non-quality in quantifiable terms as you have presented here, but then again you can only “lead a horse to water, you cannot make him drink.” Perhaps management simply has life too good to take notice of the inordinate (x820) costs of poor quality – and prefers instead to spend money rather than investing in a quality product. Sad.

    • Azieb says:

      A college rhtsoiy prof once taught me to look at each year (or each decade in rhtsoiy) in terms or the word PERSIA. List the Political, Economic, Religious, Social, Intellectual, and Aesthetic sectors of the culture (at the city, state, country, profession, industry, or regional level) to get full perspective. 2011 generated change in all sectors at all levels as the idea of debt vs austerity was raised just as the world was coming slowly out of the newly modernized banking system crash and bailout of 2008. And US is still dealing with the housing crisis (also Europe, and perhaps next in China as their bubble bursts or is massively adjusted by government). Since privatization of everything is promoted as the answer, the Baldridge subsidies are now gone. High cost of Healthcare in US is due mainly in my opinion to private insurers setting prices by state, city, industry negotiating by divide and conquer. Perhaps that problem will be addressed over next decade.Lean Six Sigma teams in Healthcare are fixing the hospitals as we speak, and just starting into the Parma industry with the Design for Quality initiatives, but they are totally ignoring the private insurance mess that pays for procedures instead of results. Perhaps the continuing Political and Economic turmoil will genereate some Intellectual change in 2012 in the Social sector of Healthcare cost-benefit improvment? Aesthetic and Relious sectors are changing faster as well, as technology pushes entertainment to help us deal with the depressive too-frequent bad news, and religions increase help for their own locals at the expense of the foreign missionary support. And of course PERSIA is also a word still used to discuss the turmoil in Middle East in historial perspective as religious feuds vie with tribal feuds in creating instability. I predict that 2010-2020 decade will provide historians with massive data analyzed badly until it becomes clear that correlation does not prove causation. Perhaps eventually the exobytes of data will be used more for enlighenment than advertising. That makes me an optimist, I am told. Happy Holidays to all.

      • Minh says:

        I don’t think the goal of the case study is to present a model of what a work stseym should look like, nor should it be. What we don’t want examiners to do is judge whether an organization has appropriately designed their work stseyms. What we do want examiners to do is look for a description of the process for designing their work stseym(s). That is the approach to the case study we are expecting, I believe. The whole reason for designing work stseyms is to ensure the organization’s assets are working together to most efficiently and effectively produce the organization’s goods and services, is it not? I believe examiners should be looking specifically for the process of deciding the scope, purpose, boundaries and elements of the key work stseym(s), all in alignment with that organization’s strategies. They should be looking for how those elements are integrated in such a way that if there are changes planned within the work stseym that they are able to determine the impact on all other elements within that work stseym. For example, if there is a planned change to the supplier, is there a process for assessing what impact that would have on other elements in the work stseym, such as the equipment being used, the procedures being followed, the work process itself, technologies required, and so forth? After a site visit, the team might determine gaps in the way work stseyms were designed that might lead to better work stseyms, based on the Criteria and the integration scoring dimension. But that is a comment after site visit, if it turns out to be appropriate.

  8. Anilkumar says:

    We need to focus on product quality,process quality and Project quality at the same time for long term sustainability.

  9. David Walker says:

    This reminds me. How could we (the society) kill the ECQ program? The Economic Case for Quality is at the very core of our existance as a society (ASQ). To kill this program 3 years ago sends a message that all we want to do is preach, not prove.

    Who would like to band to gether to get this program back in and on track?

  10. Joe Dunn says:

    “The Malcolm Baldrige program my foot!”

    Considering how long the Malcolm Baldrige program has been around, its proponents do a poor job of clearly and compellingly stating the evidence for its value. I’m in search of some claim of its benefits with clearly stated proof or evidence of that claim.

    I’m grateful for all attempts to state these supposed benefits, including Link’s. But there is a need for real improvement and clarity.

    When I start reading what’s out there, I fall asleep. But since this is such an important question – reported benefits of hundreds to one on investment! So I keep forcing myself to try to find and read something readable so I can discern and restate simply (1) what is being claimed for Baldrige, (2) what evidence is being offered to support the claim, and (3) what basic assumptions are behind the claim.

    Although I’ve been in manufacturing and in ASQ for several years, I can’t really confidently say what Baldrige is and how it works, in just a few common-language phrases. Except maybe, “The Baldrige program method is to ask organizations a lot of questions, which is supposed to lead to improvement in performance and bottom line.”

    I thought maybe Link’s economic evaluation paper would be a good place to finally get on board.

    Like most academic papers, this one starts out with no indication of its intended audience. People smarter than you or me, you assume.

    The paper starts, “Using a counterfactual method…”. The paper repeatedly mentions the “counterfactual method” as key. I searched in vain within the first two or three pages for a definition of this said method to no avail.

    Again, this is the typical style of academic papers for decades, nay centuries. And since using highly specialized terms without conveniently defining them has been the academic thesis style for ages, that must be right … right? That must be real helpful to somebody somewhere … right?

    And since the dictionary defines “counterfactual”, it should be obvious to me what Link’s “counterfactual method” is, right?

    Get the hay down out of the loft, on to the barn floor where us cows can get it.

    I was hopeful when I saw the section title “Evaluation Methodology”. Yay! They’re gonna tell how they did the study! Hm. OK, the holy grail definition of the counterfactual method is found on page 9. (It’s already been a long haul across pages 1-8.)

    But how is that definition supported as a valid method of evaluating Baldrige’s value benefit? Keep reading. “Consider Figure 1 …” Scroll up, up, up …. Scroll down, down, down … down … down, down. Scroll back up-up-up … What the … I finally found said Figure 1 just fourteen blinking pages past the reference to it.

    “Producer … differentiated … product … triangle ABP* …”


    “Figure 2 depicts the annual effect resulting when the firm implements a performance excellence program using the Baldrige Criteria.”

    Really? That’s what results from Baldrige? But how does Link know? How were those numbers derived? Which paragraph, which section of this paper hides the solid gold that shows that Baldrige is the salvation of the world?

    Ah! Page 12. Here’s how the data was collected (ahem … ready?): They asked “firms and organizations” how much they would have had to spend to achieve their current performance level without Baldrige.


    Link’s paper is pretty good considering that …

    Uh-oh. There’s a supplier quality containment needed in the plant. Gotta leave my analysis of academic papers and writing opinions behind for now.

    I’m a good communicator. I bring good tools to my plant. But I can’t bring Baldrige’s alleged benefits to my plant because no one anywhere anytime has clearly presented valid evidence I can translate into a drive that will benefit operators, managers, supervisors and fellow quality personnel in my plant.

    Prove me wrong. Although I’m not a research academic, I can read, have a degree and ASQ certifications.

    Till next paper, regards.

  11. Leodegardo M. Pruna says:

    Thank you for referring to the paper written by economists Link and Scott on their study of the Baldrige Performance Excellence Program. The study and your incisive commentary strengtened the position of the nature of “Quality” and its role in the productivity and sustainability of any enterprise when practiced and institutionalized in the firm.

  12. What we measure effects what we do.
    Now that we know what the quality it is and how we can measure it in time evrything is going to change for better!

  13. Ben Linders says:

    Good to give attention to the high costs of bad quality, an how to measure them. As Carol Dekkers already mentioned, many IT systems still have high quality costs for reparing software defects, and maintaining the software.

    It is possible to measure the quality of software during development, and to take action to prevent IT failure costs. You can also do Cost of Quality measurements, which support the business case to take actions to improve quality, by preventing that defects are made.