Use What You Know
Leveraging engineering knowledge in life data analysis
by William Q. Meeker, Necip Doganaksoy and Gerald J. Hahn
Engineers and Managers often must make important decisions in situations in which there is substantial uncertainty because of limited data. In some of those cases, the data analyses can be bolstered by incorporating engineering knowledge and experience. The following is a simple method for doing this. A more formal statistical approach involving Bayesian methods will be covered in a future Statistics Roundtable column.
To illustrate the data analyses, consider an example dealing with aircraft engine bearing cage failures, first offered in the Weibull Analysis Handbook1,2 (see the sidebar "The Importance of Design for Reliability" for current thinking on averting such problems).
The Importance of Design for Reliability
The example of the bearing cage reliability evaluation, conducted about 30 years ago, was based upon analysis of field data.
Although, unfortunately, there is still a need to scrutinize field life results today (and recently developed technology makes it easier to do so), the major emphasis has shifted to ensuring high reliability during product design.
Thus, the bearing cage reliability problem would, hopefully, be identified and addressed before releasing the product into the field. This would require early product testing, which could also call for incorporating engineering knowledge into the analysis of the resulting data.
—W.Q.M., N.D. and G.J.H.
The reliability design goal for this product required 90% of the units to exceed a life of 8,000 hours—that is, fewer than 10% failures by 8,000 hours. Based on early field results, there was some question as to whether this goal was being met. The available data needed to be closely examined. If the resulting analysis indicated an inability to meet the goal, an expensive redesign would be required.
The life data on 1,703 field units are shown in an event plot in Figure 1. In this plot, failures and unfailed (or censored) units are indicated by asterisks and arrows, respectively. The number of unfailed units at each exposure time is shown in the right-hand margin of the plot. Running times vary because different groups of units had different starting times. Such data are referred to as "multiple right censored."
Figure 1 shows that:
- Six of the 1,703 units, or 0.4%, failed at times ranging from 230 to 1,510 hours.
- The 1,697 unfailed units had running times that varied from 50 to 2,050 hours; only 21 units had exposure times exceeding 1,500 hours.
Figure 2 is a Weibull probability plot of the life data. The plotted points are nonparametric estimates—estimates obtained without making any assumptions about the form of the underlying distribution—of the population fraction failing at each of the failure times. The Kaplan-Meier method3 was used to obtain these estimates.
The solid line shows the Weibull distribution maximum likelihood (ML) estimate of the fraction failing as a function of time. The ML parameter estimates are n∧ = 11,792 hours for the Weibull distribution characteristic life (approximately the 63rd percentile) and β∧ = 2.305 for the Weibull distribution shape parameter. The dashed lines show the all-important pointwise 95% confidence intervals (based on inverting likelihood ratio tests4) on the fraction failing at different times.
The plotted points appear to be scattered randomly around the solid line, suggesting a Weibull distribution provides a reasonable fit—at least within the range of the data. The estimated 10th percentile of the distribution for time to failure, often referred to as B10 life, is 3,903 hours—considerably below the goal of 8,000 hours or more. A 95% confidence interval on B10 is 2,093 to 22,144 hours (the upper bound of this interval is beyond the range of Figure 2).
Thus, you could optimistically argue that B10 could be as high as 22, 144 hours. In contrast, a pessimist would point out that B10 also could be as low as 2, 093 hours. In short, you may be in great shape or in deep trouble. Or, using more refined language, conventional analysis of the available data provides poor precision for estimating B10, leaving much statistical uncertainty.
Though disappointing, this result is hardly surprising. Even though the data involved 1,703 bearing cages, only six failed. And, most importantly, even the longest running time (2,050 hours) was barely more than one-quarter of the desired minimum design life of 8,000 hours. You would have to wait considerably longer than 2,050 hours before being able to make a decision. The resulting delay in taking action could have serious consequences and may be unacceptable to management.
The preceding results were based exclusively on analysis of the available field data from the new design. But suppose there was extensive operational experience and data from similar past bearing cage designs. Can this knowledge be leveraged to sharpen your analysis? The answer may be "yes."
Let’s suppose that engineering knowledge of the phenomenon underlying bearing cage failure pointed to a wear-out type of failure mode, as would be expected from fatigue failures, implying the hazard function5 is increasing over time.
When the time-to-failure data can be represented by a Weibull distribution, an increasing hazard function is evidenced by the shape parameter (β) of this distribution exceeding 1. When β = 2, the hazard function increases linearly with time, and when β > 2, the hazard function increases at an increasing rate. Thus, the higher the value of β, the more pronounced the increase in the hazard function over time.
Our expectations, based on engineering knowledge, may be supported by past data. Let’s suppose, for example, that analysis of life data on various past bearing cage designs suggested the times to failure could be fitted to Weibull distributions with estimated values of β consistently between 2 and 2.5. Thus, based on a combination of engineering and empirical knowledge, project engineers may be fully comfortable asserting that the times to failure for the new design follow a Weibull distribution with a shape parameter that lies within the broadly chosen range from β = 1.5 to β = 3.
In his article, "Weibull Analysis of Reliability Data With Few or No Failures,"6 Wayne Nelson suggests using such knowledge in your evaluations, especially for situations in which there are few failures, as in our example. This approach calls for assuming different credible values of β, fitting Weibull distributions for each and comparing the results. Fitting a Weibull distribution with a given shape parameter can be readily accomplished using most standard software programs for fitting a Weibull distribution to time-to-failure data.
Weibull distributions with given values of β = 1.5, β = 2 and β = 3, respectively, were fitted to the bearing cage life data. The results are shown in the probability plots in Figures 3, 4 and 5. These analyses leave little room for optimism.
If β = 2 or 3, the data clearly show that the reliability goal of fewer than 10% failures by 8,000 hours has not been met because the upper 95% confidence bound on B10 (provided by the right-hand side dashed curves) is below 8,000 hours in both cases. Only if β = 1.5 does there appear to be some chance that the reliability goal has been met.
But even in this case, the ML point estimate of B10 (denoted by the solid line in Figure 3) is 6,465 hours, considerably less than the desired 8,000 hours. In totality, therefore, the new analyses clearly suggest the need for a redesign.
Comparing Figures 3, 4 and 5 with Figure 2 shows that in this example, which requires considerable extrapolation, assuming a value for β appreciably shrinks the width of the confidence interval and dramatically reduces the statistical uncertainty. It converts an essentially uninformative analysis into a highly informative one.
All of this seems almost too good to be true—yet the mathematics are undisputable. At the same time, you need to scrutinize the underlying assumptions. In our example, the analysis involved extensive extrapolation (from 2,050 to 8,000 hours). Such extrapolation is always dangerous. The assumed Weibull distribution may no longer hold in the extrapolated region.
For example, the observed failures might not be due to fatigue or other wear-out, but rather to a newly introduced manufacturing defect. This could cause early failure in a small proportion of the units, leading to a decreasing hazard function over time and a more favorable value of B10. This might be suggested by a Weibull β—estimated from the data—that is less than 1 or, possibly, from a probability plot in which time to failure does not seem to fit a simple Weibull distribution; but this possibility also needs to be assessed from physical evaluation of the failed units.
As always, you also must determine the degree to which the data used in your analysis accurately reflects what can be expected in the future. For example, you need to ensure the past operating environment and maintenance policies will not be changed in any way that would affect the time-to-failure distributions of the bearing cages. The added uncertainty due to any such changes is not reflected by the statistical confidence intervals. Finally, your findings clearly depend on the credibility of the assumed values of β.
An alternative approach is to describe your uncertainty about β by a statistical distribution called a prior distribution. You can combine this prior distribution with the observed data by a formal statistical method known as a Bayesian analysis. This leads to a posterior distribution on time to failure that, in turn, allows you to estimate a single value of B10 and its associated uncertainty based on a combination of your prior knowledge of β and the observed data. How to use this method will be explained in a future Statistics Roundtable column.
References and Notes
- Robert B. Abernethy, J.E. Breneman, C.H. Medlin and G.L. Reinman, Weibull Analysis Handbook, Air ForceWright Aeronautical Laboratories Technical Report AFWAL-TR-83-2079, 1983, http://handle.dtic.mil/100.2/ada143100.
- Data and further discussion on this example are covered in example 8.16 of William Q. Meeker and Luis A. Escobar’s Statistical Methods for Reliability Data, John Wiley & Sons, 1998.
- See the bibliographic notes in chapter 3 and section 6.4.2 of Meeker and Escobar’s Statistical Methods for Reliability Data, reference 2.
- For more information on inverting likelihood ratio tests, see section 8.3.3 of Meeker and Escobar’s Statistical Methods for Reliability Data, reference 2.
- See chapter 2 of Meeker and Escobar’s, Statistical Methods for Reliability Data, reference 2.
- Wayne Nelson, "Weibull Analysis of Reliability Data With Few or No Failures," Journal of Quality Technology, Vol. 17, 1985, pp. 140–146.
William Q. Meeker is professor of statistics and distinguished professor of liberal arts and sciences at Iowa State University in Ames, IA. He has a doctorate in administrative and engineering systems from Union College in Schenectady, NY. Meeker is a fellow of ASQ and the American Statistical Association.
Necip Doganaksoy is a principal technologist-statistician at the GE Global Research Center in Schenectady, NY. He has a doctorate in administrative and engineering systems from Union College in Schenectady. Doganaksoy is a fellow of ASQ and the American Statistical Association.
Gerald J. Hahn is a retired manager of statistics at the GE Global Research Center in Schenectady, NY. He has a doctorate in statistics and operations research from Rensselaer Polytechnic Institute in Troy, NY. Hahn is a fellow of ASQ and the American Statistical Association.