Reliability Assessment By Use-Rate Acceleration
by Necip Doganaksoy, Gerald J. Hahn and William Q. Meeker
Statistical evidence is often needed to show that a proposed product meets or exceeds its reliability goals. Many times, such evidence must be obtained in a compressed time period. For instance, the manufacturer of a newly designed washing machine might want its product to operate failure-free for five years or more, but might have only six months to prove this.
For products that are not used continuously, the desired validation can often be achieved by use-rate acceleration—that is, by running the product more frequently than under normal usage.
For example, by operating a toaster 100 times daily, you can simulate 10 years of operation in about 73 days, assuming a twice-a-day use rate by purchasers. Accelerated use-rate testing might be appropriate in testing other products such as photocopiers, printers, bicycles and laptop computers.
Such testing assumes the increased cycling rate will excite the failure modes seen in normal operations; for instance, failures that result directly from product operation and not, say, chemical change over time. This assumption is reasonable for many, but not all, failure modes. Failures due to corrosion provide a counter example; these are likely to be dependent on elapsed time, rather than usage.
Accelerated use-rate testing might be performed on the component, subsystem or system level. It requires selecting appropriate units for testing and establishing a plan that provides useful data for statistical analysis.
Washing Machine Motor Example
A new model motor had been built for use in washing machines.1 The motor was completely redesigned to reduce noise and improve reliability. Skilled design engineers used top quality materials and state-of-the-art methods to correct reliability problems on previous designs. They also performed short highly accelerated life tests (HALTs), subjecting components and a few prototype motors to intensive temperature cycling, vibration and overvoltage conditions to discover, understand and remove potential failure modes.
The new motor’s reliability was required to be at least 97% after 10 years of operation. Some motor bearings could wear out, causing failures, but this was felt to be unlikely during the first 10 years.
The engineers were confident they had developed a highly reliable motor. But had they? Would 97% of the motors last 10 years? In light of the extensive design changes, experience and engineering judgment provided only baseline estimates. To really find out, the engineers had six months before product release to conduct an appropriate life test on new motors at conditions that simulated customer use.
The testing was intended to quickly identify and eliminate any remaining reliability problems. It also was meant to demonstrate, with 95% statistical confidence, that the desired reliability goal would be met.
How could we obtain the equivalent of 10 years of field experience in six months? We did it by running a sample of motors continuously at stresses that simulated their operation in washing machines and shutting down for only brief cooldowns between periods of continuous running. (Such cooldowns were to activate potential failure modes due to temperature cycling.
This strategy allowed us to run 24 cycles daily, simulating
of field operation in each month of testing, assuming a use rate of four washes per week. The testing was conducted on prototype motors using special equipment that subjected them to mechanical loads simulating those encountered during a typical washing cycle. The underlying assumption that failures depend on motor running time and shutdowns—independent of elapsed time—seemed reasonable from engineering considerations.
The Test Plan
Statistical evaluations2 showed that testing 66 motors for six months would provide an appropriate sample size, balancing precision of information gained against cost. The motors used for the test needed to reflect, as closely as possible, the variability expected in large scale production. Thus, the 66 motors were built at three different times, using multiple lots of materials for each of the key assembled parts. Detailed records of the motors’ histories and test performance were maintained.
Results after one month: To everybody’s surprise and consternation, after one month of testing (728 test cycles or 3.5 years of customer use), four of the 66 motors had failed. The resulting estimated reliability of 94%—(62/66) x 100—in 3.5 years was far short of the goal of 97% in 10 years. (This shows how it is easier to prove inability to meet reliability goals than to demonstrate high reliability.)
Physical evaluation indicated that a manufacturing defect was the root cause of the four failures. In fact, all these failures occurred during the first week of testing. Fortunately, it was easy for manufacturing to fix the process to avoid such failures on future motors, and the test was continued with the four failed motors replaced by new ones.
Results after three months: After three months of testing (2,184 test cycles or 10.5 years of customer use) four more failures occurred. One bearing failure took place at 1,989 cycles, or the equivalent of 9.6 years of customer use.
There were three additional failures (at 4.5, 6.7 and 7.5 years) due to malfunction of a plastic part. This failure mode also needed to be eliminated to meet the reliability goal. The design team felt confident it could correct this malfunction by changing the geometry of the plastic part. A separate program was initiated to validate the resulting design fix.
Assuming the other two identified failure modes (manufacturing defect and plastic part malfunction) are successfully corrected, only the bearing failure mode remained applicable in assessing the reliability of future motors. This resulted in an estimated 10.5-year reliability of 98.3% (58 out of 59 surviving motors), which exceeded the 10-year reliability goal of 97%.
However, this estimate, based upon testing a relatively small sample of motors, was subject to much statistical uncertainty. To account for this, a lower 95% confidence bound on reliability (based on the binomial distribution3) was calculated as 92.2%. Generally speaking, this meant we could claim with 95% confidence that the 10.5-year motor reliability was at least 92.2%. This value was appreciably less than the 10-year goal of 97%.
Although the three-month results appeared favorable, assuming successful fixes of the two identified problems, continued testing was required to narrow the gap between the estimated reliability and the lower confidence bound to a level that would demonstrate the desired reliability statistically.
Results after six months: After six months of testing (4,368 test cycles; 21 years of customer use), seven bearing failures and four additional plastic part failures occurred, as shown in Table 1. Note the seven failures due to the plastic part malfunction are taken to be censored observations in this tabulation (for example, all that we know about their failure times is that these exceeded their observed survival times), under the assumption that this malfunction would not occur on future motors.
Analysis of Results After Six Months
Weibull and lognormal distributions were fitted to the data.4,5,6 Both models are frequently used in such applications and seemed reasonable on both theoretical and empirical grounds. The findings from the two analyses were similar. Therefore, only the Weibull distribution results are shown in Figure 1.
The solid line in Figure 1 is the Weibull distribution estimate (using maximum likelihood) of the percentage of devices failing as a function of years in service. The dotted curve provides approximate upper 95% confidence bounds on these failure probabilities. Even though Figure 1 shows only the seven bearing failures, the line was fitted taking all data into consideration.7,8
The estimated 10-year reliability was now 99.4% (failure probability of 0.006), with a 95% lower confidence bound of 96% (failure probability of 0.04). The plotted points in Figure 1 (p. 75) scatter around a straight line, supporting the Weibull distribution model assumption within the range of the data.
The 95% lower confidence bound of 96% on 10-year reliability just missed providing the desired demonstration of 97%. But the 97% demonstration can be made with 92% confidence—and this was judged sufficient for production start-up.
Further Evaluation and Testing
All failed motors, plus a sample of the unfailed ones, were taken apart and evaluated to obtain information to improve future product reliability.
Ten surviving motors were selected randomly for another 4,000 cycles of testing to obtain more precise reliability estimates. In addition, 25 units built during a one-month period and 25 units randomly sampled from the first week of high volume production—all incorporating the two fixes—were tested for varying times. The results confirmed that the earlier problems had been successfully resolved.
Finally, to check for possible new process problems, five motors are selected randomly each week from production. Four are tested for one week and one for three months.
In this washing machine example, 10 years of normal operation was simulated in three months by accelerating the use rate. For products that operate continuously, such as refrigerators and power generation equipment, this type of accelerated test is not possible. In such cases, testing might be conducted under more severe environments, such as higher temperature or humidity, to accelerate the physical or chemical degradation process that causes certain failures, such as the weakening of an adhesive bond. Accelerated testing also might involve exposing the units to increasing amounts of stress, such as voltage or pressure.
Statistical concepts can be used in both cases to develop a life test of one or more of the accelerating variables at various conditions. The resulting data are used to predict time to failure at normal operating conditions, with statistical confidence bounds, based on a physically appropriate mathematical model that relates the accelerating variable to time to failure.9
- Necip Doganaksoy, Gerald J. Hahn and William Q. Meeker, “Assuring Product Reliability and Safety,” Statistics: A Guide to the Unknown, fourth edition, Duxbury Press, 2006.
- William Q. Meeker, Gerald J. Hahn and Necip Doganaksoy, “Planning Reliability Assess-ment,” Quality Progress, June 2005, pp. 90-93.
- Gerald J. Hahn and William Q. Meeker, Statistical Intervals: A Guide for Practitioners, Wiley, 1991, section 6.2.
- Necip Doganaksoy, Gerald J. Hahn and William Q. Meeker,
“Product Life Data Analysis: A Case Study,” Quality
Progress, June 2000,
- William Q. Meeker and Luis A. Escobar, Statistical Methods for Reliability Data, Wiley, 1998.
- Wayne B. Nelson, Applied Life Data Analysis, Wiley, 1982.
- Necip Doganaksoy, William Q. Meeker and Gerald J. Hahn, “Divide and Conquer: Reliability Analysis by Individual Failure Modes,” Quality Progress, June 2002, pp. 47-52.
- Meeker and Escobar, Statistical Methods for Reliability Data, see reference 5, section 6.4.
- Gerald J. Hahn, William Q. Meeker and Necip Doganaksoy, “Speedier Reliability Analysis,” Quality Progress, June 2003, pp. 58-64.
NECIP DOGANAKSOY is a statistician and Six Sigma Master Black Belt at the GE Global Research Center in Schenectady, NY. He has a doctorate in administrative and engineering systems from Union College in Sche-nectady. Doganaksoy is a senior member of ASQ and a fellow of the American Statistical Assn.
GERALD J. HAHN is a retired manager of statistics at the GE Global Research Center in Sche-nectady, NY. He has a doctorate in statistics and operations research from Rensselaer Polytechnic Institute in Troy, NY, where he is also an adjunct faculty member. Hahn is a fellow of ASQ and the American Statistical Assn.
WILLIAM Q. MEEKER is professor of statistics and distinguished professor of liberal arts and sciences at Iowa State University in Ames, IA. He has a doctorate in administrative and engineering systems from Union College in Schenectady, NY. Meeker is a fellow of ASQ and the American Statistical Assn.