Validating Key Results Linkages in the Baldrige Performance Excellence Model

April 2003
Volume 10 • Number 2

Contents

Validating Key Results Linkages in the Baldrige Performance Excellence Model

by James R. Evans, University of Cincinnati, and Eric P. Jack, University of Alabama, Birmingham

The Malcolm Baldrige National Quality Award Criteria for Performance Excellence represent a framework for high-performance management systems. Although the criteria have evolved based on the collective wisdom of quality experts and management practitioners throughout the United States, little empirical research has been performed to validate the criteria and its core concepts and values. One of the key linkages in the criteria is the relationship between external results and internal performance metrics, which reflect the processes and management systems that drive results in an organization. The authors use a large empirical data set to explore these linkages, and apply canonical correlation to uncover significant relationships between variables, such as between employee satisfaction and process performance, and between work system improvement and productivity. These results: 1) validate some of the key linkages; 2) support long-standing beliefs and anecdotal evidence by practitioners of the relationships among endogenous and exogenous results for business performance; and 3) lend credibility to causal hypotheses that improving internal management practices leads to improvements in external results. As such, they provide new evidence of the validity of the Baldrige model and its examination/self-assessment process that seeks to validate strong business results as an outcome of high-performance management practices.

Key words: Baldrige, canonical correlation, information and analysis, performance results

INTRODUCTION

The Malcolm Baldrige National Quality Award Criteria for Performance Excellence represent an integrated performance management model. The model has evolved in parallel with high-performance management practices and experiences of winning companies and experts in quality management. Evans (1997) proposed causal linkages among criteria elements—results and management practices—based on an input-output analysis and logical modeling. However, little empirical research to either explore or validate criteria linkages has been performed. In this article the authors use a comprehensive database to explore and validate some of the anecdotal beliefs that underlie the Baldrige framework. Specifically, they focus on the relationships among key external (exogenous) results and internal (endogenous) performance metrics.

PERFORMANCE MEASUREMENT FRAMEWORKS AND SUPPORTING LITERATURE

Performance measurement has long been studied at the operational level of organizations, particularly in the context of manufacturing (Maskell 1991). The two most influential frameworks for measuring organizational performance are the balanced scorecard (Kaplan and Norton 1992; 1996), and the Malcolm Baldrige National Quality Award Criteria for Performance Excellence.

The balanced scorecard was proposed in response to the limitations of traditional accounting measures, its purpose being to translate strategy into measures that uniquely communicate an organization’s vision. The original Kaplan-Norton version of the balanced scorecard (the term has taken on a generic meaning in practice) consists of four perspectives:

  • Financial perspective: Measures the ultimate results that the business provides to its shareholders. This includes profitability, revenue growth, return on investment, economic value added, and shareholder value.
  • Internal perspective: Focuses attention on the performance of the key internal processes that drive the business. This includes such measures as quality levels, productivity, cycle time, and cost.
  • Customer perspective: Focuses on customer needs and satisfaction as well as market share. This includes service levels, satisfaction ratings, and repeat business.
  • Innovation and learning perspective: Directs attention to the basis of a future success—the organization’s people and infrastructure. Key measures might include intellectual assets, employee satisfaction, market innovation, and skills development.

A balanced scorecard approach helps in identifying the right measures by aligning them with the organization’s vision and strategy. This provides a means of setting targets and allocating resources for short-term planning, communicating strategies, aligning departmental and personal goals to strategies, linking rewards to performance, and supplying feedback for organizational learning. Kaplan and Norton (1996) provide several examples that demonstrate the effective use of the balanced scorecard. Some empirical research in the manufacturing sector that examines the relationship between competitive priorities and performance are studies by Miller and Roth (1994), Noble (1995), and Kathuria (2000), which found a correspondence between performance measures used and the manufacturing priorities emphasized by firms. This research provides some empirical evidence of the value of aligning measures with strategy. For a more general discussion on this issue and the design of performance measurement systems, see Feurer and Chaharbaghi (1995).

The Malcolm Baldrige National Quality Award Criteria for Performance Excellence provide a similar, though slightly different, framework as the balanced scorecard. The business results category groups performance measures into five major categories:

  • Customer
  • Financial and market
  • Human resource
  • Supplier and partner performance
  • Organizational effectiveness

Basically, the difference between the Kaplan-Norton framework and the Baldrige Award criteria is a matter of semantics; the internal perspective includes most of the measures from the Baldrige organizational effectiveness item, while innovation and learning encompasses human resources measures. In both models, the focus is on a set of measures that provides a comprehensive perspective on organizational performance, and any measure an organization might use can be assigned to an appropriate category in either framework.

Although there has been considerable research on measuring the dimensions of quality in organizations (for example, Flynn, Schroeder, and Sakakibara 1994; Saraph, Benson, and Schroeder 1989; and Ahire, Golhar, and Waller 1996), little formal research addresses the actual selection of organizational performance measures. Most published literature examines this issue from practitioners’ perspectives (for example, Letza 1996; Axson 1999; Hepworth 1998; Carlin 1999; Frigo and Krurnwiede 2000; Kueng 2000; and Rousseau and Rousseau 1999/2000). The anecdotal evidence suggests that a good balanced scorecard contains both leading and lagging measures and indicators. Lagging measures (outcomes) tell what has happened; leading measures (performance drivers) predict what will happen. For example, customer survey results about recent transactions might be a leading indicator for customer retention (a lagging indicator); employee satisfaction might be a leading indicator for turnover, and so on. These measures and indicators should also establish correlational or cause-and-effect relationships across perspectives. Kaplan and Norton (1996) note the importance of establishing and validating causal relationships as a basis for managing and validating an organization’s strategy. These are also cited as critical elements of the Baldrige framework, particularly in validating the impact of management practices in assessments.

Many empirical studies have examined relationships between managerial practices or dimensions of quality and business performance. Adam (1994) studied alternative quality improvement approaches and operating and financial performance, finding a strong relationship between the approach used and performance quality. Samson and Terziovski (1999) examined the relationship between total quality management (TQM) practices and operational performance, focusing on the relationship between performance outcomes and award categories, and demonstrated that leadership, people management, and customer focus were particularly strong predictors of performance. Choi and Eboch (1998) found some paradoxical relations among TQM practices, plant performance, and customer satisfaction. Flynn, Schroeder, and Sakakibara (1995) examined reasons for differences in high- and low-quality plants. Curkovic, Vickery, and Droge (1999) examined relationships between competitive dimensions of quality. Sinclair and Zairi (1995) analyzed results from 22 organizations, focusing on relationships among performance measures and strategic goals and improvement areas to investigate their alignment. Their results suggested a gap between goals and performance measurements that are implemented. Other similar studies include Anderson and Sohal (1999), Chapman, Murray, and Mellor (1997), Dow et al. (1999), Kannan et al. (1999), Forza and Filippine (1998), Ittner and Larcker (1997), Cua, McKoone, and Schroeder (2001), and Germain, Droge, and Christensen (2001).

Some empirical studies have addressed cause-and-effect linkages or correlations among organizational performance measures. These include Norreklit (2000), who examined the assumptions and cause-and-effect chain in the balanced scorecard; studies of the relationship between customer satisfaction, value and loyalty, and financial performance (Brandt 2000; Anderson, Fornell and Lehmann 1994; Bernhardt, Donthu, and Kennett 2000; Edvardsson et al. 2000); relationships between employee attitudes and customer satisfaction (Tornow and Wiley 1991; Hallowell, Schlesinger, and Zornitsky 1996); relationships between work environment and customer service as related to financial performance (Wiley 1991; Borucki and Burke 1999); and relationships between customer attitudes and market share/financial performance (Naumann and Hoisington 2000).

One approach to assessing the relationship between measures is correlation. Kaplan and Norton (1996, 254) note the value of measuring correlations between two or more measures as a means of validating hypothesized cause-and-effect relationships and validating strategy. Hoisington and Huang (2000) describe an empirical study conducted at IBM’s AS/400 Division in Rochester, Minn., a 1990 winner of the Baldrige Award. Using 10 years of data and 50 key measurements, they demonstrated strong correlations and causal effects between market share, customer satisfaction, productivity, warranty cost, and employee satisfaction. They conclude by stating the need to take an enterprise view of measurements and understand the impact of one measurement on another, noting that most organizations are functionally and hierarchically oriented, which typically inhibits such analyses.

Hardie (1998) summarizes the findings of 43 case studies, experiments, opinion surveys, and correlational surveys that study the effects of quality on business performance, and develops a theoretical framework that suggests a causal chain of impacts from improving quality leading to increased profitability. Much of the work examined by Hardie that involves correlation studies stems from the marketing literature and focuses on loosely defined aggregate measures of quality and such outcomes as market share; these include Phillips, Chang, and Buzzell (1983), Wagner (1984), Jacobson and Aaker (1987), Buzzell and Gale (1987), Anderson, Fornell, and Lehmann (1994), and Danaher and Rust (1996).

SURVEY DATABASE

Ghosh and Handfield, with the help of the American Society for Quality, developed a database using a sample of 1469 directors from manufacturing firms in a variety of industries. A total of 307 usable surveys (21 percent) were produced. Table 1 provides a breakdown of the respondents by industry and firm size (number of employees). The firms are located in all 50 states, and include organizations from aerospace and defense, automotive, chemicals, computers and electronics, construction, medical devices and equipment, telecommunications, utilities, and several others. Using number of employees as a surrogate for the firm size (small: less than 500; medium: 500 to 1000; and large: greater than 1000), the data consist of 57 percent small, 15 percent medium, and 28 percent large firms. These data have been used in several studies (Jayaram, Handfield, and Ghosh 1997; Handfield, Ghosh, and Fawcett 1998). This database also contains a variety of results measures and indicators, which were clustered into logical categories. These are summarized in Table 2. (Aliases specified for the indicator variables will be used in the discussion of the results.)

HYPOTHESIZED LINKAGES

One of the key management practices that has been a part of the Baldrige Award criteria for many years is the use of business results to analyze and subsequently improve organizational performance. According to the criteria, “Analysis includes trends, projections, comparisons, and cause-effect correlations intended to support performance reviews and the setting of priorities for resource use. Accordingly, analysis draws upon all types of data: customer-related, financial and market, operational, and competitive.” In the results category of the Baldrige Business criteria (prior to 2001), the first two items, customer-focused results and financial and market results, represent exogenous results, while the remaining, human resource results, supplier and partner results, and organizational effectiveness results, are generally endogenous. This implies a logical linkage among results indicators.

The Baldrige Award criteria framework suggests clear linkages between practice and results, directly between the human resource focus and process management categories, and indirectly from the others. For example, Evans (1997) proposed an influence diagram model of specific results linkages, suggesting areas of strong correlation and implying causality, which forms a basis for using results for improvement. Handfield and Ghosh (1995) used structural equations modeling to empirically test the linkages, but their results showed little evidence of the relationship between quality practices and business results. Wilson and Collier (2000) also used structural equations modeling, and their results suggest a strong relationship between quality practices and organizational performance. Pannirselvam and Ferguson (2001) showed that internal practices such as human resource management and information management had significant effects on performance. Yet, despite these efforts, several researchers have continued to call for more research to validate these linkages (Kannan et al. 1999; Wilson and Collier 2000; Flynn and Saladin 2001). In this study, the authors address this gap by testing and validating some of these linkages. For example, in Figure 1, they present a model with 20 hypotheses that were tested in this study. The first 10 hypotheses represent linkages among the endogenous variables as follows:

  • H1: Employee satisfaction has a positive impact on process performance.
  • H2: Work system improvement has a significant impact on productivity.
  • H3: Work system improvement has a significant impact on employee satisfaction.
  • H4: Work system improvement has a significant impact on process performance.
  • H5: Process performance has a significant impact on productivity.
  • H6: Employee satisfaction has a significant impact on service quality.
  • H7: Employee satisfaction has a significant impact on product quality.
  • H8: Process performance has a significant impact on service quality.
  • H9: Process performance has a significant impact on product quality.
  • H10: Supplier performance has a significant impact on product quality.

The next 10 hypotheses evaluate the direct linkages between the endogenous variables and the exogenous results as follows:

  • H11: Employee satisfaction has a significant impact on market performance.
  • H12: Service quality has a significant impact on customer satisfaction.
  • H13: Product quality has a significant impact on customer satisfaction.
  • H14: Product quality has a significant impact on financial performance.
  • H15: Supplier performance has a significant impact on financial performance.
  • H16: Process performance has a significant impact on financial performance.
  • H17: Productivity has a significant impact on financial performance.
  • H18: Customer satisfaction has a significant impact on market performance.
  • H19: Market performance has a significant impact on financial performance.
  • H20: Customer satisfaction has a significant impact on financial performance.

METHODOLOGY AND RESULTS

In this study, the authors use canonical correlation to evaluate the hypotheses shown in Figure 1. Canonical correlation analysis is a multivariate statistical technique that is used to analyze the relationship between sets of multiple independent and dependent variables (Hair et al. 1998). As opposed to simple linear correlation, which measures the strength of the relationship between two simple observable variables, canonical correlation measures the strength of relationships between two latent (unobservable) variables, each of which consists of multiple (observable) indicators. Canonical correlation assumes that each set can be given some theoretical meaning, at least to the extent that one set can be defined as the independent variables, and the other as the dependent variables.

In this study, the Baldrige results measures are the canonical variates, and the authors’ hypothesized model of results linkages provides the theoretical context. For example, hypothesis 11 in Figure 1 suggests a relationship between employee satisfaction (ES) and market performance (MP). In Table 2, the authors use three observable indicators to measure employee satisfaction: employee absenteeism, employee grievances, and employee turnover; and four measures of market performance: market share, average annual market share growth, average annual sales growth, and growth/decline in customer base. In canonical correlation, each latent variable is represented as a linear combination of the indicator variables in the sample data. Thus,

Canonical correlation finds a set of weights that maximizes the correlation between the two linear combinations (latent variables). The resulting correlation is called the first canonical correlation coefficient. Because customer complaints and competitive leadership had only one indicator variable, they were excluded from the analysis.

The authors used SAS version 6.02 to complete the canonical correlation analysis. They began the analysis with the 307 cases from the Ghosh-Handfield database. However, some data were missing for a few of the observable variables, which caused the actual number of observations used in the canonical analysis to be reduced from 307 cases to 279 cases. The authors then used CANCORR procedure in SAS to analyze the hypothesized relationships between the dependent and independent latent variables.

Table 3 summarizes the first canonical correlations and significance levels between each latent variable. Also, since they used aliases to identify the observable variables in SAS, they provide (in the appendix) a list of these aliases along with a description of these variables and the names used in the actual database.

Hair et al. (1998) suggest using three criteria for determining how canonical functions should be interpreted: 1) the level of statistical significance; 2) the magnitude of the canonical correlation; and 3) the redundancy measure for the percentage of variance accounted for from the two sets. As is generally accepted practice, the authors used a .05 significance level as the threshold. However, no generally accepted guidelines have been established regarding the magnitude of the canonical correlations. Hair et al. (1998) suggest using common guidelines for significance of factor loadings (a minimal level of ±0.30 for practical significance). Similarly, no generally accepted guidelines for a minimum acceptable redundancy measure have been established. The authors have selected an index of 0.10 or greater as the threshold.


Table 4 shows the results sorted by descending values of the redundancy index. Multiple significant orthogonal canonical relationships are indicated by (1) and (2). All of the canonical correlations shown meet the guideline for practical significance. The shaded rows signify those that meet both the significance test and redundancy index threshold for the dependent variable. The authors also observe that higher redundancy index values generally are accompanied by higher canonical correlation values. Table 4 also shows the canonical R2, which represents the percentage of the variance in the dependent canonical variate that can be explained by the independent variate, and the shared variance in the dependent variable set included in the dependent canonical variate. (The redundancy index is the product of these two values.) Figure 2 shows a graphic summary of these results. The linkages shown by the solid arrows were validated by both the significance level test and the redundancy tests. The linkages shown by the dashed arrows are validated only by the significance level tests.

If the authors compare the hypothesized linkages (see Figure 1) with those validated by the canonical correlation analysis (see Figure 2), they note that seven of the 10 hypotheses about the linkages among endogenous variables were validated. Also, all 10 of the linkages between the endogenous variables and the exogenous results were significant, and five of these met all three criteria to validate a causal relationship (Hair et al. 1998).

DISCUSSION OF RESULTS

Three methods have been proposed for interpreting canonical variates when the relationship is statistically significant and the correlation and redundancy index are acceptable:

  1. Examining the magnitude and signs of the standardized coefficients of the canonical weights
  2. Canonical structure correlations—the simple linear correlations between an original observed variable and the set’s canonical variate
  3. Canonical cross-loadings—correlations between the observed dependent variables directly with the independent canonical variate and vice versa. Table 5 shows these values for each of the shaded rows in Table 4.

Hair et al. (1998) recommend cross-loadings as the preferred method, since they provide a more direct measure of the dependent-independent variable relationships. The standardized coefficients, however, can provide insight into the nature of the relationships, and the authors will interpret these when appropriate. To illustrate this, consider the relationship between employee satisfaction and process performance. The standardized canonical coefficients define the canonical variate. In this case, the canonical variates are:

EMP_S1 = .6891 ES1 + .2366 ES2 + .2342 ES3
Where EMP_S1 = Employee satisfaction
ES1 = Employee absenteeism
ES2 = Employee grievances
ES3 = Employee turnover

and

PROCESS1 = .8037 PP1 + .3422 PP2
Where PROCESS1 = Process performance
PP1 = production equipment downtime
PP2 = number of causes of process variation

These weights optimize, in a predictive sense, the ability of employee satisfaction indicators, as defined by the canonical function, to predict the composite of process performance. All signs are positive, indicating direct relationships that would be expected from the definition of the variables. In the first canonical function, employee absenteeism (ES1) contributes the most to the variate employee satisfaction, while equipment downtime (PP1) contributes the highest to process performance. The canonical correlation of 0.684 suggests that about 47 percent of the variance in the linear composites can be explained. Because the weights are designed to optimize the canonical correlation, however, the canonical loadings and cross loadings provide more appropriate interpretation.

The canonical structure correlations show that all three employee satisfaction indicators have relatively high loadings on the composite resulting in a moderate shared variance (0.65), suggesting that these three measures are appropriate indicators of satisfaction. Similarly, the process performance indicators have a shared variance of 0.68, again, indicating a relatively strong indication of process performance. Thus, the indicators provide a fairly well-defined dimension for representing the latent variables.

The canonical cross-loadings show that employee satisfaction indicators have moderate correlations with the dependent variate, process performance, and therefore are reasonable predictors of process performance. The square of these values (.42, .26, and .25, respectively) indicate the percentage of the variance explained by the composite function. While these results are not particularly high, considering the variety of firms included in the sample data, they do appear to have some practical significance. Similarly, employee satisfaction is moderately correlated with the process performance indicators. Analyzing process performance as the independent variate, however, does not provide useful interpretative results from a causal modeling perspective.

Although each pair of variates may be interpreted this way, in the appendix the authors will examine each dependent variate in relation to all other variates associated with it in Table 4 as an approach to justifying the causal relationships suggested in Figure 2. To facilitate this process, they have re-sorted Table 4 first by dependent variate and then ordered by the canonical correlation. The result is listed in Table 6.

SUMMARY AND CONCLUSIONS

These empirical results support long-standing beliefs and anecdotal evidence by practitioners about the relationships among endogenous and exogenous results for business performance, and lend credibility to causal hypotheses that improving internal management practices leads to improvements in external results. For example, consider the relationships among endogenous variables in the Baldrige results category, such as between employee satisfaction and process performance, and between work system improvement and productivity. Strong correlation among these latent variables suggests the importance of many fundamental management practices that are embedded in the Baldrige requirements, such as a focus on employee well being and motivation, and attention to the design of work systems and their linkage to other categories, such as process management. By strengthening the practices that lead to improved levels of internal performance, the analysis indicates that improved performance of production/delivery processes will likewise occur. Second, high levels of the endogenous variables are correlated with exogenous performance results as measured by market share, customer satisfaction, and financial performance. This provides evidence that improving the performance of endogenous variables will positively impact the most important external business performance measures. Thus, this research provides new evidence of the validity of the Baldrige model and its examination/self-assessment process that seeks to validate strong business results as an outcome of high-performance management practices.

The main shortcomings of this study are inherent in the limitations of the statistical method used to test and validate these hypotheses. While the authors have shown that canonical correlation can be used to validate the relationship between two sets of latent variables, more sophisticated methods, such as structural equation modeling, are needed to account for measurement error and to concurrently validate the causal linkages within the entire model.

APPENDIX: EXAMINATION OF DEPENDENT VARIATES
Employee Satisfaction

Employee satisfaction is correlated significantly with process performance and product quality as an independent variate. The standardized canonical coefficients and canonical structure correlations show relatively strong and consistent relationships between each of the observable indicators (ES1, ES2, and ES3) and the employee satisfaction variates. Also, the relatively high degree of shared variance suggests that the three observed indicators are reliable measurements of the latent variable, employee satisfaction. The canonical cross-loadings with both process performance and product quality are also relatively high, suggesting a causal relationship between the variates. This is consistent with observations from many Baldrige Award winners that increased employee satisfaction leads to higher performance.

Process Performance

Process performance is correlated significantly with employee satisfaction as a dependent variate, and with product quality and market performance as an independent variate. The standardized canonical coefficients and canonical structure correlations show relatively strong and consistent relationships between the indicators PP1 and PP2 and their respective canonical variates. Canonical cross-loadings also show moderate correlations among the process performance indicators and employee satisfaction, product quality, and market performance variates. The cross-loadings are weakest for market performance. However, the authors note that market performance is further in distance from process performance in the causal chain model, so the relationship would not be as direct as with the other two variates.

Customer Satisfaction

Customer satisfaction is correlated significantly as a dependent variate with product quality, service quality, and work system improvement. The canonical functions suggest a bit of an anomaly:

CUST_SAT = .1914 CS1 + .8815 CS2 (with product quality)
CUST_SAT = -.3122 CS1 + 1.1361 CS2 (with service quality)
CUST_SAT = -.09292 CS1 + 1.0145 CS2 (with work system improvement)

One would expect that both customer retention (CS1) and customer satisfaction (CS2) would have the same signs in each canonical function, as both should be positive indicators of satisfaction. As the canonical structure correlations are all positive, customer retention (CS1) acts as a supressor in the second (service quality) and third (work system improvement) variates. The authors, however, observe that the canonical structure correlations for customer satisfaction (CS2) are 0.97 and higher in all three cases, and that the standardized canonical coefficients dominate customer retention (CS1). This suggests that customer satisfaction (CS2) is a more significant indicator of satisfaction than customer retention (CS1). Higher cross-loadings also support this notion.

The apparent anomaly can also be explained by a comment made by Patrick Mene, vice president for quality at the Ritz-Carlton Hotel Company, a two-time Baldrige Award recipient, at the 2000 Quest for Excellence Conference in Washington, D.C.: “Satisfaction is an attitude; loyalty is a behavior.” Customers may indeed be satisfied but still switch allegiance based on other factors. Thus, customer retention (CS1) is not necessarily a reliable indicator of satisfaction. The authors have searched the marketing literature to find explanations for these results. They note that researchers have found mixed results in their investigations of the relationship between customer satisfaction and customer loyalty. Several studies have indeed found satisfaction to be a leading factor in determining loyalty (Anderson et al. 1994). Researchers in the other camp advocated by Reichheld (1993) suggest that customer satisfaction is not a surrogate for customer loyalty, and thus increasing customer satisfaction does not necessarily lead to increased customer loyalty to a product, service, or organization. More recent research (Devaraj, Matta, and Conlon 2001) suggests that customer loyalty is driven by factors such as product or service quality, price, and company or brand image.

Product Quality

Product quality is correlated with the variates employee satisfaction, work system improvement, and process performance as a dependent variate, and with customer satisfaction and financial performance as an independent variate. There is considerable variation as to the contribution of individual indicators: customer rejection of product (PQ1), overall product quality (PQ2), defect rates/cost (PQ3), and frequency of field repair/service (PQ4) to the different canonical variates. Overall, the canonical structure correlations show that PQ1, PQ2, and PQ4 are most strongly indicative of the product quality latent variate. The canonical cross-loadings show the strongest relationship with financial performance and the weakest relationship with work system improvement. Again, this is consistent with the authors’ prior observation of the distance between the latent variates in the causal model.

Service Quality

Service quality is correlated significantly only with customer satisfaction. On-time delivery (SQ2) dominates the relationship in terms of its contribution to the canonical variate and its canonical structure correlation.

Work System Improvement

Work system improvement is correlated significantly as an independent variate with product quality, customer satisfaction, and financial performance. Although employee suggestions (WS1) and quality problems resolved (WS2) contribute about equally to the customer satisfaction variate, the contribution of WS1 is essentially nil for product quality and financial performance.

Financial Performance

Financial performance is correlated significantly as a dependent variate with productivity, market performance, work system improvement, and product quality. From a practical perspective, this suggests that quality-related initiatives do have a significant impact on financial performance as many studies have shown (for example, Hendricks and Singhal (1997) and the National Institute of Standards and Technology’s (NIST) continuing study of Baldrige winners’ performance [www.quality.nist.gov]). The contributions of the individual observations differ among the variates. For example, cost of quality (FP4), prevention cost (FP5), and warranty cost (FP6) are the major contributors to the variate associated with productivity and product quality, whereas return on total assets (FP1) and growth in ROA (FP2) are the major contributors to market performance, which makes sense intuitively. All indicators except cost of quality (FP4) contribute to the work system improvement variate. The cross-loadings are consistent with the canonical structure correlations.

Productivity

Productivity is correlated significantly with financial performance. Only rework (P2) and scrap (P3) contribute strongly to the relationship. Again, this makes sense because the number of quality inspectors (P1) would not likely affect long-term financial performance.

Market Performance

Finally, market performance is correlated significantly with process performance and financial performance. Although the relationship with process performance is significant, the correlations suggest an inverse relationship with market share (MP1). As process performance is far removed from market performance in the causal chain, the authors cannot give reasonable credibility to this result. The canonical cross-loadings are relatively weak, suggesting a spurious correlation. On the other hand, market share growth (MP2) and average sales growth (MP3) are strongly correlated to financial performance, as one might expect.

ACKNOWLEDGMENT

The authors are grateful to Professor Soumen Ghosh of Georgia Tech for providing the database for this research.

REFERENCES

Adam, Everett, E. Jr. 1994. Alternative quality improvement practices and organization performance. Journal of Operations Management 12, no. 1: 27-44.

Ahire, S. L., D. Y. Golhar, and M. A. Waller. 1996. Development and validation of TQM implementation constructs. Decision Sciences 27, no. 1: 23-56.

Anderson, E. W., C. Fornell, and D. R. Lehmann. 1994. Customer satisfaction, market share, and profitability: Findings from Sweden. Journal of Marketing 58, no. 3: 53

Anderson, M., and A. S. Sohal. 1999. A study of the relationship between quality management practices and performance in small businesses. International Journal of Quality & Reliability Management 16, no. 9: 859-877.

Axson, David A. J. 1999. The fastest route to right answers. Strategy & Leadership 27, no. 3: 6-10.

Bernhardt, K. L., N. Donthu, and P. A. Kennett. 2000. A longitudinal analysis of satisfaction and profitability. Journal of Business Research 47: 161-171.

Biggs, M. 1999. Decision-support products track leading performance metrics, application usage. InfoWorld 21, no. 32: 54.

Borucki, C. C., and M. J. Burke. 1999. An examination of service-related antecedents to retail store performance. Journal of Organizational Behavior 20: 943-962.

Brandt, D. R. 2000. Linking measures of customer satisfaction, value, and loyalty to market and financial performance: Basic methods and key considerations. In Proceedings of ASQ’s 54th Annual Quality Congress. Milwaukee, Wisconsin: ASQ Quality Press.

Buzzell, R. D., and B. T. Gale. 1987. The PIMS principles: Linking strategy to performance. New York: Free Press.

Carlin, T. 1999. Simplifying corporate performance measurement. Australian CPA 69, no. 11: 48-50.

Chapman, R. L., P. C. Murray, and R. Mellor. 1997. Strategic quality management and financial performance indicators. International Journal of Quality & Reliability Management 14, no. 4: 432-448.

Choi, T. Y., and K. Eboch. 1998. The TQM paradox: Relations among TQM practices, plant performance, and customer satisfaction. Journal of Operations Management 17, no. 1: 59-75.

Cua, K. O., K. E. McKoone, and R. G. Schroeder. 2001. Relationships between implementation of TQM, JIT, and TPM and manufacturing performance. Journal of Operations Management 19, no. 6: 675-694.

Curkovic, S., S. K. Vickery, and C. Droge. 1999. Quality and business performance: An empirical study of first-tier automotive suppliers. Quality Management Journal 6, no. 2: 29-40.

Danaher, P. J., and R. T. Rust. 1996. Indirect financial benefits from service quality. Quality Management Journal 3, no. 2: 63-75.

Devaraj, S., K. F. Matta, and E. Conlon. 2001. Product and service quality: The antecedents of customer loyalty in the automotive industry. Production and Operations Management 10, no. 4: 424-439.

Dow, D., D. Samson, and S. Ford. 1999. Exploding the myth: Do all quality management practices contribute to superior quality performance? Production and Operations Management 8, no. 1: 1-27.

Edvardsson, B., M. D. Johnson, A. Gustafsson, and T. Strandvik. 2000. The effects of satisfaction and loyalty on profits and growth: Products versus services. Total Quality Management 11, no. 7: 917-927.

Evans, J. R. 1997. Critical linkages in the Baldrige award criteria: Research models and educational challenges. Quality Management Journal 5, no. 1: 13-30.

Feurer, R., and K. Chaharbaghi. 1995. Performance measurement in strategic change. Benchmarking for Quality and Management & Technology 2, no. 2: 64-83.

Flynn, B. B., R. G. Schroeder, and S. Sakakibara. 1994. A framework for quality management research and an associated measurement instrument. Journal of Operations Management 11, no. 4: 339-366.

Flynn, B. B., R.G. Schroeder, and S. Sakakibara. 1995. Determinants of quality in high- and low-quality plants. Quality Management Journal 2, no. 2: 8-25.

Flynn, B., and B. Saladin. 2001. Further evidence of the validity of the theoretical model underlying the Baldrige criteria. Journal of Operations Management 19: 617-652.

Forza, C., and R. Filippini. 1998. TQM impact on quality conformance and customer satisfaction: A causal model. International Journal of Production Economics 55, no. 1: 1-20.

Frigo, A. L., and K. Krurnwiede. 2000. The balanced scorecard. Strategic Finance 81, no. 7: 50-54.

Germain, R., C. Droge, and W. Christensen. 2001. The mediating role of operations knowledge in the relationship of context with performance. Journal of Operations Management 19, no. 4: 453-470.

Hagen, M. R. 2000. Quality awards listing. Quality Progress 33, no. 8: 64-74.

Hair, J., R. Anderson, R. Tatham, and W. Black. 1998. Multivariate data analysis, 5th edition. Upper Saddle River, N.J.: Prentice Hall.

Hallowell, R., L. A. Schlesinger, and J. Zornitsky. 1996. Internal service quality, customer, and job satisfaction: Linkages and implications for management. Human Resource Planning 19, no. 2: 20-31.

Handfield, R., and S. Ghosh. 1995. An empirical test of the linkages between the Baldrige criteria and financial performance. In Proceedings of the Decision Sciences Institute. Atlanta: Georgia State University.

Handfield, R., S. Ghosh, and S. Fawcett. 1998. Quality-driven change and its effects on financial performance. Quality Management Journal 5, no. 3: 13-30.

Hardie, N. 1998. The effects of quality on business performance. Quality Management Journal 5, no. 3: 65-83.

Hendricks, K. B., and V.R. Singhal. 1997. Does implementing an effective TQM program actually improve operating performance? Empirical evidence from firms that have won quality awards. Management Science 43, no. 9 (September): 1258-1274.

Hepworth, P. 1998. Weighing it up—A literature review for the balanced scorecard. Journal of Management Development 17, no. 8: 559-563.

Hoisington, S. H., and T. Huang. 2000. IBM Rochester correlation on measurements of employee satisfaction cost of quality, productivity, customer satisfaction, and market share. In E. Naumann and S. Hoisington, eds., Customer Centered Six Sigma. Milwaukee, Wisconsin: ASQ Quality Press.

Ittner, C. D., and D.F. Larcker. 1997. The performance effects of process management techniques. Management Science 43, no. 4: 522-534.

Jacobsen, R., and D. A. Aaker. 1987. The strategic role of product quality. Journal of Marketing (October): 31-44.

Jayaram, J., R. Handfield, and S. Ghosh. 1997. The application of quality tools in achieving quality attributes and strategies. Quality Management Journal 5, no. 1: 75-100.

Kannan, V. R., K. C. Tan, R. B. Handfield, and S. Ghosh. 1999. Tools and techniques of quality management: An empirical investigation of their impact on performance. Quality Management Journal 6, no. 3: 34-49.

Kaplan, R. S., and D. P. Norton. 1992. The balanced scorecard—Measures that drive performance. Harvard Business Review (January/February): 71-79.

Kaplan, R. S., and D. P. Norton. 1996. The balanced scorecard. Boston: Harvard Business School Press.

Kathuria, R. 2000. Competitive priorities and managerial performance: A taxonomy of small manufacturers. Journal of Operations Management 18, no. 6: 627-641.

Kueng, P. 2000. Process performance measurement system: A tool to support process-based organizations. Total Quality Management 11, no. 1: 67-85.

Letza, S. R. 1996. The design and implementation of the balanced business scorecard. Business Process Re-engineering and Management Journal 2, no. 3: 54-76.

Maskell, B. H. 1991. Performance measurement for world-class manufacturing. Portland, Ore.: Productivity Press.

Miller, J. G., and A. V. Roth. 1994. A taxonomy of manufacturing strategies. Management Science 40, no. 3: 285-304.

Naumann, E., and S. H. Hoisington. 2000. Customer centered six sigma. Milwaukee, Wisconsin: ASQ Quality Press.

NIST. 2000. Baldrige quality award program, criteria for performance excellence. Washington, D.C.: National Institute of Standards and Technology (NIST).

Noble, M. A. 1995. Manufacturing strategy: Testing the cumulative model in a multiple country context. Decision Sciences 26, no. 5: 693-721.

Norreklit, H. 2000. The balance on the balanced scorecard—A critical analysis of some of its assumptions. Management Accounting Research 11: 65-88.

Phillips, L. W., D. R. Chang, and R. D. Buzzell. 1983. Product quality, costs position and business performance: A test of some key hypotheses. Journal of Marketing (spring): 26-43.

Pannirselvam, G., and L. Ferguson. 2001. A study of the relationship between the Baldrige categories. International Journal of Quality and Reliability Management 18, no. 1: 14-34.

Reichheld, F. F. 1993. Loyalty-based management. Harvard Business Review 71: 64-73.

Rousseau, Y., and P. Rousseau. 1999/2000. Turning strategy into action in financial services. CMA Management 73, no. 10: 25-29.

Samson, D., and M. Terziovski. 1999. The relationship between total quality management practices and operational performance. Journal of Operations Management 17, no. 4: 393-409.

Saraph, G. V. P., G. Benson, and R. G. Schroeder. 1989. An instrument for measuring the critical factors of quality management. Decision Sciences 20, no. 4: 810-829.

Sinclair, D., and M. Zairi. 1995. Benchmarking best-practice performance measurement within companies using total quality management. Benchmarking for Quality Management & Technology 2, no. 3: 53-71.

Tornow, W. W., and J. W. Wiley. 1991. Service quality and management practices: A look at employee attitudes, customer satisfaction, and bottom-line consequences. Human Resource Planning 14, no. 2: 105-115.

U.S. Department of Commerce. 2000. 2000 Criteria for Performance Excellence. Washington, D.C.: U.S. Department of Commerce.

Wagner, H. M. 1984. Profit wonders, investment blunders. Harvard Business Review (September/October): 121-135.

Wiley, J. W. 1991. Customer satisfaction: A supportive work environment and its financial cost. Human Resource Planning 14, no. 2: 117-127.

Wilson, D., and D. Collier. 2000. An empirical investigation of the Malcolm Baldrige quality framework. Decision Sciences 31, no. 2: 361-390.

BIOGRAPHIES

James R. Evans is professor of quantitative analysis and operations management, and director of the Total Quality Management Center in the College of Business Administration at the University of Cincinnati. He received bachelor’s and master’s degrees from Purdue in 1972, and a doctorate in industrial and systems engineering from the Georgia Institute of Technology in 1975. Evans has published more than 80 refereed journal articles and is author or coauthor of over a dozen books in the areas of quality management, statistical process control, management science, simulation and risk analysis, applied statistics, operations management, creative thinking, and mathematics. His current research interests include organizational impacts of information and analysis and conceptual theories underlying the Baldrige criteria. He serves on several editorial boards of professional journals including ASQ’s Quality Management Journal. He has served as an examiner, senior examiner, and alumni examiner for the Malcolm Baldrige National Quality Award from 1994-2001 and is a member of the board of trustees for the Ohio Award for Excellence.

Eric P. Jack is an assistant professor of Management at the Department of Management, Marketing and Industrial Distribution, University of Alabama at Birmingham. His research interests include operations strategy with special emphasis on flexibility, quality management, and supply chain management. His research has been published in Journal of Operations Management, Production and Operations Management, and in a 1999 text, Quantitative Models for Supply Chain Management. In addition, his recent work is under consideration at several journals including International Journal of Operations and Production Management, Health Care Management Review, Medical Care Research and Review, and Production and Operations Management. His work experience includes 21 years as a U. S. Air Force officer where his responsibilities at various international locations involved facility planning, design, construction, and maintenance. He is a member of the American Production and Inventory Management Control Society, the Academy of Management, the Decision Sciences Institute, and the Production and Operations Management Society.

 

Return to top

Featured advertisers

Article
Rating

(0) Member Reviews

Featured advertisers





ASQ is a global community of people passionate about quality, who use the tools, their ideas and expertise to make our world work better. ASQ: The Global Voice of Quality.