Commentary by Jack West, American Society for Quality
May 16, 2006
Download the Report (PDF, 158 KB)
What Do Quality Scores Mean?
Differences Between Service Quality and Product Quality
This report on quality is based on data from the American Customer Satisfaction Index (ACSI), a key economic indicator and the nation’s leading measure of customer satisfaction. It offers further analysis and commentary by ASQ experts on the perceived quality component of ACSI
QUALITY IN THE UTILITIES, TRANSPORTATION, INFORMATION, HOSPITALS, AND ACCOMMODATION & FOOD SERVICES SECTORS: HIGHLIGHTS FROM THE FIRST QUARTER 2006 ACSI
During the first quarter of 2006, customers of a broad variety of service providers were interviewed for the American Customer Satisfaction Index. The interviews took place at a transitional time in the U.S. economy. Consumer confidence was still high, although poised to take a steep decline in May. The economy maintained signs of strength and consumers continued to shop, although there were some first signs that high energy costs may be causing consumers to be more cautious.
Against this background, the rising fuel costs appear to be taking a toll on customer perceptions of quality in the energy utilities and airline industries. Wireless phone service providers and hospitals are the only groups showing any significant improvement in ACSI perceived quality scores during the first quarter of 2006 compared to the first quarter of 2005. The gains in satisfaction with the quality of wireless service brought up the score for the information sector as a whole.
Among the 30 energy utilities tracked by the ACSI, ten experienced significant changes in their perceived quality scores this quarter compared to the same period last year. Of these, six declined (Centerpoint, Entergy, FPL, National Grid, Northeast, and TXU) and four (Allegheny, Edison International, First Energy, and Xcel) showed gains in perceived quality. TXU and FPL—whose quality scores dropped—were also among the utilities posting the highest price increases last year. Duke Power and Xcel—whose perceived quality scores rose—rank among the lowest-priced utilities as measured by price charged per kilowatt-hour.
Rising fuel costs, aggravated by disruptions in energy supplies as a result of last season’s record number of hurricanes, factor prominently into the way in which customers rate the quality of energy utilities.
The price of electricity in the U.S. rose by an average of nearly 11 % between April 2005 and April 2006. Consumers heating their houses with natural gas saw their bills increase 17% last winter to a record high of $867 on average nationwide. Deregulation has not brought about a consumer dividend: the highest electric costs occur in states that have deregulated retail electricity markets (according to NUS Consulting Group of Park Ridge , NJ ).
More consumers are complaining about their utility bills, and some are charging price fixing; eighteen communities in the Midwest have filed an antitrust lawsuit against natural gas producers.
Compared to the first quarter of 1995, when the ACSI first measured customer satisfaction with utilities, the perceived quality rating of the industry as a whole has declined significantly (3.5 index points).
Among the beleaguered U.S. airlines, the only one to show a significant gain in perceived quality was US Airways, which had the lowest perceived quality score this quarter last year. But even that was not good news, as US Airways still managed only to pull up into a last-place tie with Northwest Airlines. Northwest, whose bankruptcy protection filing occurred on September 14, 2005, also experienced turbulence in the form of a strike by mechanics, janitors, and airplane cleaners; the carrier dropped precipitously in customer quality ratings to join US Airways at the bottom of the group.
The airline industry receives low overall quality marks—the lowest of any industry group measured in the ACSI. And just when they thought things couldn’t get any worse, the airlines continued to get hammered with persistently rising fuel costs.
Southwest Airlines bucks the trend—the only American network carrier to capture perceived quality ratings matching the overall national quality index for all industries.
The airline industry lost $25 billion between 2001 and 2005, during which time there were also 20 airline bankruptcies. As losses continue to mount, carriers have been taking a host of actions that dissatisfy customers: slashing payrolls; eliminating some amenities and charging passengers extra for others; reducing capacity; and trying to increase load factors through scheduling less frequent service on many routes and by flying smaller jets. Northwest and its regional partners, for example, have eliminated 11% of capacity, much of it coming by ditching its aging and fuel-guzzling DC-9 jets.
From February 2005 through February 2006, the major U.S. network airlines reduced their full-time-equivalent headcount by 26% (96,000 FTEs). The decline at US Airways was 42% and at Northwest 29%. During that time, Northwest slipped from fourth largest to sixth largest in terms of FTE number of employees.
Pay and benefits for employees who have remained on the payroll have been repeatedly cut, resulting in demoralized workers. As part of its bankruptcy recovery plan, Northwest targeted $1.35 billion from cuts in payrolls and benefits.
The total number of passengers traveling on U.S. air carriers rose over 4% in 2004, while the number of flights remained the same. That means more passengers per flight. And the FAA says passenger volume will continue to grow 3-4% for the foreseeable future. Passenger volume has recovered since the low point following Sept. 11; but capacity has fallen; load factors are now at an all-time high with about 80 percent of seats filled for the industry overall.
But passengers shouldn’t expect things to get better—they’re bound to get worse. U.S. domestic airlines aren’t ordering many new planes, so capacity is not expected to increase substantially between now and 2009, according to a new report from Bear Stearns & Company. The author of that report says strong demand for airline seats could push ticket prices up 8.1% this year and 3.9% next year. That means passengers will be paying more to sit in crowded planes with fewer comforts.
Customers love their pizza but don’t have so many good things to say about burger joints.
The only limited-service restaurants to record gains this quarter compared to the first quarter last year are Little Caesar’s, Domino’s, and Pizza Hut. Furthermore, both Pizza Hut and Little Caesar’s have enjoyed substantial gains in perceived quality since the initial ACSI measurement of this category in 1995.
Wendy’s is the only one of the three measured hamburger restaurants with perceived quality ratings that come close to those given to the pizza restaurants; Burger King is in the middle of the pack, and McDonald’s is all alone far behind all other measured limited-service restaurants in perceived quality.
The national hamburger chains are struggling to break away from their reputation as providers of commodity burgers. New sandwich offerings are marketed using terms intended to increase their quality appeal, such as “select,” “hand-carved,” “made to order,” and “slow roasted.” Other new product introductions such as salads for adults and yogurt, deli sandwiches and low-fat milk for kids are designed to appeal to health-conscious diners. But most of their customers aren’t really looking for healthy fare; the real appeal of these places still seems to be chopped meat and cheese—the more the better, and never mind the calories and cholesterol.
Papa John’s is the clear quality leader among the major pizza restaurants. This chain touts the quality of its pizza ingredients—including proprietary cheese and flour, and dough delivered fresh to its restaurants from central production facilities which they call quality control centers—as a main component of its business plan.
Starbucks has the highest perceived quality rating among limited-service restaurants. This is the first time Starbucks has appeared in this rating category.
Since hospital care is a locally-delivered service, there are no individual hospitals that receive sufficient mentions in the ACSI interviewing process to warrant their listing individually by name. As a group, however, hospitals increased their perceived quality score by nearly four percentage points this quarter compared to the first quarter of 2005.
For a number of years hospitals have been adopting and adapting improvement approaches that have proven effective in other sectors. These efforts have been given some urgency as a result of recent reports from the Institute of Medicine detailing a crisis in American healthcare. Many more hospitals have been applying proven quality management techniques such as Six Sigma methodology and other approaches, including an emphasis on measurement of outcomes that are meaningful from the patient point of view. The Malcolm Baldrige criteria for healthcare are becoming a widely accepted framework for improvement that many hospitals are adopting.
The healthcare industry’s high level of interest in quality improvement techniques is confirmed by recent ASQ studies. These studies show that healthcare organizations have the highest level of commitment to a performance management approach among all industries surveyed. They also have the highest percentage of respondents that are actually measuring results of performance management approaches and the highest recognition of customer satisfaction as a performance management tool. Furthermore, recognition of Six Sigma and Baldrige was higher in healthcare than in any other industry.
Increased transparency and public reporting of quality measures, such as the information provided by the Hospital Compare website provided by the Centers for Medicare and Medicaid Services, as well as the growing adoption of pay-for-performance plans, are likely to be having an impact on improving quality. The effect of these initiatives is to raise healthcare consumers’ awareness of quality.
This growing level of interest may now be reaching the point where it is having a noticeable effect on consumers’ evaluations of the quality of care they are receiving from hospitals.
Even though determination of quality is a complex and highly subjective calculus involving the simultaneous processing of many factors inside the mind of the consumer, that does not mean it can’t be quantified.
The ASQ Quality Report relies on the tested methodology of the ACSI to help quantify the subjective evaluations of the goods and services acquired and consumed in the United States . Interviews with many customers probe multiple facets of quality such as product or service attributes, price, and market fit to measure the subjective evaluations of the goods and services acquired and consumed in the United States . Data derived from these interviews are used as inputs to the ACSI’s econometric model, which combines numerous proxy measures (reflecting the consumer’s overall consumption experience) to arrive at an index number on a 0 to 100 scale. This is not a percentage. Unlike the output of many familiar consumer surveys, an ASQ Quality Report score of 80, for example, does not mean that 80% of consumers who were interviewed have high regard for the quality of the particular product or service in question.
The unique methodology used to calculate the ASQ Quality Report’s quality scores has the advantage of allowing for cross-industry and cross-company comparison. For example, it allows us to say with assurance that consumers have higher regard for the quality of beer than they have for the quality of banking services, or that consumers think Heinz products have better quality than Dole products.
To better understand this quarter’s quality report results, it helps to have some insight into the differences between quality of services and quality of products.
Perceived quality for the service-based industries measured during the fourth quarter is noticeably lower than perceived quality for manufactured goods (see Table 2). Why is this?
One of the main enemies of quality is variation. Most products are produced with highly defined, predictable and controlled processes. On the other hand, most services are the result of an interaction between two people. Since interactions among people are inherently more variable than manufacturing processes, there are more opportunities for defects to occur in the delivery of a service than in the production of a product.
Because of this variability, many service providers operate in a response mode. The conditions change constantly as the service transaction progresses. As a result, it is difficult to predict and script in advance all of the scenarios that may occur in this type of environment.
Furthermore, relatively few service organizations have employed advanced quality systems like Lean, Quality Function Deployment or Six Sigma--techniques that have been a mainstay in manufacturing for decades. Thus the typical manufacturing organization is far ahead of the typical service organization in the application of advanced quality practices.
Based on this reasoning, one would expect that those manufactured products that also have a service component will have a lower perceived quality score for the service component than for the manufactured component. This expectation is confirmed by the ACSI results for virtually all products that entail both manufactured and service components, such as automobiles and major appliances.
This report on quality offers further analysis by ASQ experts and is based on a key economic indicator and the nation’s leading measure of customer satisfaction, the American Customer Satisfaction Index. Produced by the University of Michigan in partnership with the American Society for Quality and CFI Group, the ACSI is produced quarterly, measuring more than 200 companies in 41 industries. The index has been issued and supported by the partnership for more than 10 years.
The ACSI uses two primary criteria to define the consumer’s quality experience:
ACSI data collection occurs through thousands of quarterly interviews with customers who have purchased and used specific products or services within defined time periods. It treats satisfaction with quality as a cumulative experience rather than a most-recent-transaction experience.
Customer interviews that formed the basis of the overall ACSI and its quality component occurred during the period of January through March of this year. Customers of companies in the utilities, transportation, information, hospitals, and accommodation & food services industries were interviewed.
Each quarter a different sector is measured, with the fresh data being used to update the national ACSI score quarterly on a rolling basis. ASQ plans to issue Quarterly Quality Reports, with analyses on these sectors, based on the latest ACSI scores.
About the Author
Jack West is a past president of the American Society for Quality and a quality expert. A Six Sigma Master Black Belt instructor and consultant, West has held various engineering and management positions at Westinghouse Electric Corporation and Northrop Grumman. He holds a doctorate in business administration and a master’s degree in management, both from The George Washington University. John Ryan, ASQ public policy analyst, also contributed to this report.
About the American Society for Quality (ASQ)
The American Society for Quality is the world’s leading authority on quality. With more than 92,000 individual and organizational members, the professional association advances learning, quality improvement, and knowledge exchange to improve business results and to create better workplaces and communities worldwide. As champion of the quality movement, ASQ offers technologies, concepts, tools, and training to quality professionals, quality practitioners, and everyday consumers, encouraging all to Make Good Great®. ASQ has been the sole administrator of the prestigious Malcolm Baldrige National Quality Award since 1991. Headquartered in Milwaukee, Wisconsin, the 60-year-old organization is a founding partner of the American Customer Satisfaction Index (ACSI), a prominent quarterly economic indicator.
Other information about ASQ and the ACSI can be found on the following Web sites: www.asq.org/quality-report/index.html