First audit roadmap
Q: I recently earned my lead auditor certification, and I’m in the process of setting up a third-party audit of an organization that’s seeking an ISO 9001:2015 certification. Can you give me some advice or steps to follow? Can you also tell me what pay rate I should expect?
Sienna Plantation, TX
A: Congratulations on receiving your lead auditor certification. Your first step is to contact the auditee and let the organization know that you have been directed by your client to set up an audit. Decide on a mutually convenient date for the audit, and clearly establish or convey the audit’s objective and scope.
For example, the objective of the audit may be: "To verify compliance of the auditee’s quality management system (QMS) with ISO 9001:2015 requirements" or "To assess the readiness of the auditee to go through a registrar’s ISO 9001:2015 certification audit." The scope may include a specific plant or facility, or a product or production line at a plant or facility in which the audit will be performed. If these agreements are made over the phone, put them in writing, and follow up with the auditee.
If you are going to lead this audit, select and notify other auditors who will accompany you—that is, unless your client is selecting who will be on your team.
After you have set an audit date, you must prepare for the audit just as you learned in ISO 9001 lead auditor training, and create an audit plan.
Request a copy of the quality manual from the organization you are going to audit. Review it carefully, make notes and develop a feel for the auditee’s QMS. You can develop your own checklist or use a canned checklist. While going through the quality manual, decide what clarifications and verifications you want to accomplish. Decide who you might want to see or interview, what questions you will ask them and what information you will look for. You also can tentatively decide which departments you want to visit and how much time you want to spend in them. Before your first meeting, you should have a good idea of what you want to accomplish and how to do it.
After these steps, share this information with the auditee to ensure the organization has some understanding of your action plan. You also could do this at the opening meeting and give the auditee an opportunity to ask you questions about the audit.
If you are the lead auditor for this audit, you can expect to be paid $600 to $800 per day plus expenses. If you are an auditor, you can expect $400 to $600 per day plus expenses. To better gauge whether your payment expectations are in line with what is available, I suggest you read the 2015 QP Salary Survey1—which contains information about quality professionals’ salaries based on their education, level of experience, industry or geographical location. Good luck on your first audit.
Pradip V. Mehta
Mehta Consulting LLC
- Max Christian Hansen, "A Winning Hand?" Quality Progress, December 2015, pp. 16-21.
Q: What method of statistical sampling do automotive airbag manufacturers use? My understanding is that airbags go through destructive tests. Hence, testing lots of samples would seem very costly. Specifically, I want to learn:
- How do manufacturers select their sample for a given lot?
- What happens if a sample is identified as defective?
- Do they discard the whole batch based on the result of one failed sample?
In addition, are there any combination of destructive and nondestructive testing that are performed on airbags before a batch is released?
A: You have asked some specific questions that should be answered by someone who works in the airbag manufacturing industry. I suspect that anyone who currently works in the industry, however, is bound by confidentiality—especially today because of the massive recall underway in the United States and around the world. According to the National Highway Traffic Safety Administration (NHTSA), more than 28 million vehicles equipped with Takata airbags have been recalled.1 The recall involves 14 vehicle manufacturers and at least 24 makes (BMW, Chevrolet and Chrysler) for models produced between 2000 and 2015.
Investigations have revealed that the airbag inflator may rupture during deployment and eject metal shrapnel into the passenger compartment, which could cause injury or death. There have been at least 10 deaths and 139 injuries in the United States so far related to this recall.2 Investigators now believe that there are three factors that can collectively result in catastrophic failure of the inflator mechanism:
- The use of ammonium nitrate as the propellant.
- Inadequate structural design of the inflator housing.
- Prolonged or repeated exposure to high heat and humidity. Over time, moisture can penetrate the inflator canister and make the ammonium nitrate propellant more explosive.3, 4
About 7 million airbags have been replaced so far. Some of the returned units have been tested to simulate a crash by sending an electrical signal to the inflator. As of October 2015, 115,000 returned units have been tested, and 450 of those tests (about 0.4%) resulted in a ruptured inflator.
The data show that older units and units from regions with high humidity (such as the Gulf Coast) are up to 10 times more likely to rupture during deployment. This is important because it reveals the latent nature of this failure mode. The likelihood of catastrophic failure increases over time, but probably could not have been detected using typical sampling and test methods at the time of manufacture. Even if Takata was performing destructive tests under normal conditions, the failure rate was probably too low to be detected.
I contacted a colleague who worked in the airbag industry for many years. He said that the industry has long been aware of the detrimental effect of moisture on airbag propellants. Ideally, manufacturers should select a propellant that is stable over time and not prone to absorb moisture. (It is worth noting that Takata was the only major airbag manufacturer who selected ammonium nitrate as the propellant.) Airbag manufacturers also may add a desiccant to the canister to absorb moisture and extend the life of the product. Some manufacturers inject inert gas into the canister to displace moisture-laden ambient air just before sealing. They also conduct 100% nondestructive leak testing, and a single leak would result in rejecting the batch. Whether Takata used these precautions is unknown.
Is it possible to detect latent and time-dependent failure modes? The usual approach is accelerated testing. When engineers develop a new design, they cannot afford to wait for many years to verify that the design will meet the reliability requirements. So, they subject the design to higher levels of stress, expecting the products to fail more quickly. They can use this data to extrapolate and estimate the product’s life under normal operating conditions.
These accelerated life test models require a high level of engineering knowledge and statistical expertise. Some models use constant stress, and some models use step-stress. Without getting too technical, let’s assume that the selection of the stress model makes a huge difference. In the case of the Takata inflator assembly, it is possible that thermal cycling plays an important role. The unit heats up during the day and cools off at night. After the unit accumulates thousands of thermal cycles, the metal canister and seals may start to fatigue, and micro-cracks could allow moisture to enter.
Now consider destructive testing in general across industry. Destructive testing is expensive, so large sample sizes are undesirable and perhaps not even feasible. Is it necessary to test to failure? In some cases, the product can be stress tested beyond the normal operating conditions, and if the unit survives the test, it can still be sold. For example, airplane wings are tested by applying a force that is 1.5 times greater than the maximum lift force exerted against the wing during flight.
Testing to failure using attribute sampling plans (pass or fail) is inefficient because it requires large sample sizes. But if we can obtain variable data during the destructive testing, we can model the distribution to predict the likelihood of failure—and reduce the sample size by 90% or more, provided the likelihood of failure is small. We also can reduce the testing time and total cost by using reliability testing with the right censored data.
Suppose I want to test the reliability (life) of a circuit board design. I select 10 units, subject them to a test, and after four or five units have failed, I can stop the test and predict the life of the population based on the failed and censored units.
With respect to your other questions about sampling, the samples should either be randomly selected or representative of the population. What does "representative" mean? Studies have shown that for some processes, the probability of failure changes throughout the production run, so it is actually better to sample systematically, such as once per hour. While this sampling approach is not random, it is more likely to accurately estimate the distribution of the population.
If any of the samples fail, the batch should be held and an investigation should be initiated to determine the root cause. The investigation should consider the severity of the failure mode and the likelihood of additional potential failures in the population. If the underlying root cause is easy to detect, it may be acceptable to perform 100% inspection to sort out the nonconforming units. But if the risk is too high, the batch should be scrapped. If the failure mode is linked to a design flaw, the batch should be rejected and production should be discontinued.
Master Black Belt
Director of quality systems
NSF Health Sciences Pharma Biotech
- Safecar.gov, "Recalls Spotlight: Takata Air Bag Recalls," www.safercar.gov/rs/takata/takatalist.html.
- Clifford Atiyeh and Rusty Blackwell, "Massive Takata Airbag Recall: Everything You Need to Know, Including Full List of Affected Vehicles," Car and Driver, March 2, 2016, http://tinyurl.com/car-and-driver-takata.
- Scott Yon, "Coordinated Remedy Program Proceeding: What is the Status? (Part II)," remarks, U.S. Department of Transportation National Highway Traffic Safety Administration, Oct. 22, 2015, http://tinyurl.com/takata-remarks.
- Scott Yon, "Summary of Takata Ballistic Testing" presentation, U.S. Department of Transportation National Highway Traffic Safety Administration, Oct. 22, 2015, http://www.safercar.gov/rs/takata/pdfs/Yon-Status-Presentation.pdf.
For more information
Nelson, Wayne B., Accelerated Testing—Statistical Models, Test Plans and Data Analysis, Wiley and Sons, 2009.