BACK TO BASICS
Take Another Look
The link between audit performance and process effectiveness
By Daniel J. Le Saux
The use of several strategies to monitor and improve the overall effectiveness of a quality management system is included in ISO 9001, paragraph 8, "Measurement Analysis and Improvement."1 Two of these strategies are often used independently and the strong interaction between them is overlooked. They are process monitoring and measurement (8.2.3) and internal audit (8.2.2).
An audit can be defined as a comparison of things as they are to an objective standard of what they should be. As such, audit performance—or process compliance—is the degree to which actual practice matches expected practice. A high level of compliance provides confidence the results, good or bad, are repeatable. On the other hand, a low level of compliance provides little confidence the results will repeat.
Metric performance, or process effectiveness, is a measure of whether quality objectives—such as planned results—are being achieved.
Analyzing the interactions between audit performance and metrics performance can trigger systematic actions that improve overall results.
It is important to compare the performance metrics most closely associated with the process being audited. The possible interactions between audit and metrics performance, shown in Figure 1, are explained below.
Noncompliant and ineffective: This is the worst-case scenario. There is no indication the metric results can be improved by adhering to the documented process because there is no evidence showing the process is actually capable of producing valid results.
The initial approach should drive compliance and then closely monitor metrics performance. If the metrics show a positive trend, the documented process is valid. If the metrics show a negative trend, the documented process is invalid, and efforts must be made to improve the process.
Actions must also be taken to understand why process documentation is not being followed, and behavioral changes must be deployed to ensure it will be going forward.
Compliant but ineffective: The audit results show actual practice matches expected practice; however, planned results are not being achieved. This is an easier problem to solve because the documented process is already being followed. Changes to the process documentation can be made with a high degree of confidence it will be followed. Improving the process and its associated documentation will improve the metrics performance.
Noncompliant but effective: This indicates the process owners have concluded that the documented process is ineffective and they have already made the corrections in practice without documenting the changes. In this case, the documented procedures should be modified to match what individuals are already doing.
Further actions should be taken to investigate why the process owners did not modify the procedure to reflect the new process. Allowing process owners to arbitrarily modify a process without updating documentation allows for excessive variation in the process, leading to unpredictable results.
Compliant and effective: This is the ideal scenario in which there is balance between process compliance and process effectiveness. In this case, there is a strong probability the results will be consistently positive. It also provides an environment in which sustainable process improvements can be made by simply modifying documented procedures.
A rigorous analysis of audit performance versus process metrics adds a new dimension to data already being gathered and allows for repeatable improvement mechanisms that ultimately lead to enhanced performance.
International Organization for Standardization, ISO 9001—Quality management systems—requirements, 2000.
Daniel J. Le Saux is the corporate quality systems manager at Skyworks Solutions Inc. in Woburn, MA. He earned an associate degree in electrical engineering at the Lycée Kérichen in Brest, France. Le Saux is a senior member of ASQ and is a certified quality engineer and quality auditor.