Tips for Automotive Auditors
by R. Dan Reid
ISO 9000 has taken more than its fair share of criticism, largely due to the variation in international third-party conformity assessment.1 A significant difference in auditor and registrar auditing competence is caused by factors such as education, training and experience.2
There are also systemic problems. The oversight function3 design of the international third-party conformity assessment process has not consistently provided customer organizations with confidence in the overall process.
The system design with respect to the revenue stream contributes to the problem. Companies contract with registrars for auditing and certification services so are viewed as customers.
Another group of key customers are the companies purchasing your organization’s product. They are the direct customers of the quality system certification status of their suppliers. Yet, because they are out of the revenue stream, registrars usually don’t view them as customers of the audit process or results.
Initiatives To Reduce Variation
These were key concerns of Chrysler (now DaimlerChrysler), Ford and GM, who I’ll refer to as original equipment manufacturers (OEMs) in the rest of this article, when they were designing QS-9000 and their recognized third-party certification process.
Based on responses from a survey of registrars from around the world, QS-9000 Appendices B, G and H were developed and made a condition of achieving OEM recognition of the registrar’s certificates. Despite much OEM work to reduce variation in the third-party conformity assessment process (such as auditor qualification) used to support the launch of QS-9000, additional measures became necessary.
One of these measures was an auditor recertification process. This was designed first to test the auditors’ knowledge of the QS-9000 requirements and their ability to properly apply their knowledge in a variety of given scenarios.
Where the testing revealed insufficiency, the process design required auditors to take additional OEM sanctioned classes to supplement or update their competence to continue as OEM recognized QS-9000 auditors. This measure required recertification every three years.
In today’s world, change is one of the only constants, so it is critical for auditors and registrars to make concerted efforts to keep their knowledge and skills up-to-date.
Examples of Required Auditor Skills
To add value, auditors should be agents for positive change. In addition to the requirement of bringing specific industry experience to the audit process, it is helpful if they are skilled in process engineering or reengineering.4
An auditor should be able to readily identify the inputs, outputs and requirements of any process and determine or confirm the key characteristics,5 inputs and outputs of the process.
These characteristics can be either variable or attribute. The OEMs published a common statistical process control (SPC) reference manual to provide common methodology for understanding variation, process capability or performance, control and improvement.
Since its release in 1992, a strong bias toward the use of variable data has resulted in an underutilization of process controls for attribute data. While OEM and supplier quality has continued to improve, many supplier caused OEM quality problems today result from attribute characteristics.6 Chapter three of the SPC manual is dedicated to attribute control charts and should be familiar to QS-9000 auditors.
Auditors must also review the OEM customer quality feedback reports of the auditees.7 Information in these reports can and should be a major consideration in determining the effectiveness of the auditee’s processes and system.
One of the OEM survey questions asked of auditors in designing the QS-9000 third-party certification requirements was whether the auditors evaluated the effectiveness of the actions taken by auditees to address a requirement.
Some indicated they did not because they were only to verify whether the auditees (the experts) were doing what they said they were going to do. Others indicated they made some evaluation of the effectiveness of the actions taken. The OEMs then made the latter a QS-9000 Appendix B requirement to increase confidence in the eventual audit results.
Auditors must be able to evaluate effectiveness on at least three levels (see Figure 1, p. 72). They need to determine whether the:
- Actions taken to address a requirement are effective in meeting or exceeding the intent of the standard.
- Operation and control of processes are effective in meeting planned results and meeting or exceeding the intent of the standard.
- The management system is effective in meeting planned arrangements and meeting or exceeding the intent of the standard.
Process controls are widely used in industry today. But process design leading to a process capable of producing the desired result all the time must come before process control or there will be inherent problems in the process with no planning for mitigating potential downstream effect.
Process capability from a statistical point of view is a function of tolerance spread. The auditor should determine whether characteristic tolerances were based on statistical methodology. If not, process capability or performance index (Cpk or Ppk) values may not be much help in determining whether the process (at the characteristic level) is, in reality, capable.
Regardless, the auditor should look to downstream process results (scrap, rework and customer complaints, for example) to determine whether the system is capable at a higher level. Automotive OEM quality expectations are at high level, less than 25 parts per million for example, at a part number level. Wherever possible, error proofing methods should be in place rather than relying on detection and correction of problems after they occur.
Measurement Systems Analysis
The amount of measurement variation as a percentage of the total tolerance spread must be understood to determine process capability. Auditors must have a basic understanding of measurement bias, linearity, stability and gage repeatability and reproducibility studies.
Measurement system analysis comprehends operator-to-operator variation and same operator variation over time. The OEMs published a common measurement systems analysis manual for suppliers in 1990. Auditors should understand the key detailed guidance in this manual, which goes well beyond the ISO 9001 requirements for monitoring and measuring processes or product.
Know the Terminology
With regard to process capability, auditors must know the difference between process capability, stability and targeting. With regard to process control, they must know the difference between special and common cause variation. They should know and be able to determine proper application of variable and attribute control charts and how the charts can be used for process improvement. Guidance is provided in the OEM’s SPC manual.
Effective communication depends on a good understanding of the definitions of terms. This is especially critical in standards work, as evidenced by ISO 9000, a separate standard in the ISO 9000 series devoted only to terminology, and by other documents—ISO 8402 for example.
The OEMs recognized this need during the global launch of QS-9000, when many questions led to the publication of QS-9000 sanctioned interpretations.
There are now 85 automotive sector specific definitions in QS-9000 to supplement the ISO 9000 terminology documents. Auditors must be students of the ISO 9000/QS-9000 language to be relevant and add value to audits. A review of this terminology should be a core component of continuing education programs for automotive auditors.
The OEMs published a common reference manual for failure mode and effects analysis (FMEA) in 1993. FMEA did not originate in the automotive sector, but it has proven to be an effective risk management tool for a number of industries over time. There are several key points for automotive auditors to understand about FMEAs.
To be effective, the FMEA must be a live document, updated with information fed back from the field, including items such as warranty and customer complaints. It is important the FMEA be worked from the left to right sides of the form, which means initial high risk priority numbers (RPN) are addressed to mitigate the effects of a potential failure, and the RPNs are recomputed and reprioritized on the right side of the form based on these initial actions.
Attempts should be made in the process design stage to error proof tooling, machinery and equipment. Where the RPN or severity rating remains high on the right side of the FMEA form, these characteristics should be designated as critical by the supplier on the FMEA form and carried forward to the part number control plan and appropriate work instructions to mitigate the effects of a potential failure downstream—in manufacturing or assembly.
Opportunities for Improvement
The OEMs require auditors of QS-9000 to identify opportunities for improvement in their audit report—a significant departure from the third-party conformity assessment expectations of ISO 9000 auditors. This adds value and benefits the auditee’s customers.
Many third-party auditors are excellent and dedicate themselves to exceeding their client’s expectations. As with law enforcement officers, however, there is a tendency over time for auditors to view themselves, rather than the standard, as the authority.
Registrars must ever be on the alert to prevent this mind-set in their auditors. The future success of the international third-party conformity assessment process in part depends on it.
NOTES AND REFERENCE
- The National Institute of Standards and Technology Guidance on Federal Conformity Assessment, 15 CFR Part 287, effective Aug. 10, 2000, says conformity assessment means any activity concerned with directly or indirectly determining requirements are fulfilled. It includes sampling and testing; inspection; supplier’s declaration of conformity; certification; quality and environmental management system assessment and registration; accreditation; and recognition. Conformity assessment activities may be conducted by the supplier (first party) or by the buyer (second party) either directly or by another party on the supplier’s or buyer’s behalf, or by a body not under the control or influence of either the buyer or the seller (third party).
- In this article, “registrar” is synonymous with “certification body.”
- A national accreditation body, typically a government agent, provides oversight of the registrar process for registrars who have contracted with it for its service.
- The minimum knowledge for registrar auditors for QS-9000 and ISO/TS 16949 is specified by the OEMs and is the basis for auditor certification testing and training. These OEM requirements are not addressed in their entirety in this article, and some of this article’s content goes beyond the OEM specification.
- While an auditor is prohibited from being a consultant to the same organization, he or she should have this same ability. See ISO DIS 10019:2003 Guidelines for the Selection of Quality Management System Consultants and Use of Their Services, clause 188.8.131.52.
- R. Dan Reid, “Characteristic Management,” Quality Progress, November 2003.
- The auditee is the company being audited, not just the employee to whom a question is addressed.
R. DAN REID, an ASQ Fellow and certified quality engineer, is a purchasing manager at GM Powertrain. He is co-author of the three editions of QS-9000; ISO/TS 16949; the Chrysler, Ford and GM Advanced Product Quality Planning With Control Plan; Production Part Approval Process; and Potential Failure Mode and Effects Analysis manuals; the current version of ISO 9001; and ISO IWA 1. He was also the first delegation leader of the International Automotive Task Force.