Caveat Emptor

Standards and audits will have value only if organizations being audited--and their customers--demand it

by Dale K. Gordon

What does it take to have a competent third-party audit of your quality management system?

Surely, if you were doing it yourself, you would know exactly where to look to find the weakest links in the system. An internal audit should give you a good idea of what is right and what needs improvement. Why, then, is so much being made of having a third party perform an audit just so you can have a piece of paper or plaque to hang on the wall and show to your customers?

It's the practical way

Ever since the world agreed on standards for quality management systems (the ISO 9000 series), we have come to believe if we show evidence our operations meet these standards, we can provide some level of confidence to our customers.

They can be confident because they know there are systems, processes and controls in place to assure product conformance and continuous improvement in operations. An in-depth audit of a quality system is the accepted method of demonstrating this conformance.

To have audits performed by all our customers in this global economy is neither a practical nor a cost effective use of resources--especially as we look at the depth and breadth of the supply chain, from raw material or concept to product and service.

We have therefore devised a process by which we hire supposedly impartial third parties (often called registrars or certification bodies) to audit quality management systems. If we are compliant with the necessary or appropriate standard, they provide us with certification (called registration outside the United States).

For customers to have confidence in the registrars hired by our organizations, we require they be accredited by an official accreditation body, which may or may not be tied to a government controlled process of approval.

Finally, in order for these accreditation bodies to agree the process of accreditation is equivalent from one location (country) to the next, the accreditation bodies have gotten together to sign agreements and audit each other's processes.

It sounds reliable

To further ensure confidence in the system, the ISO 10011 series of auditing standards attempts to describe auditing requirements as follows:

  • Auditors need to know something about the standard they are measuring organizations against.
  • They should know something about the types of processes being audited, the organization being audited and its customers.
  • They should have the knowledge, the temperament and experience in auditing to be credible.

It sounds pretty complicated, doesn't it? At the same time, the various levels of control and the auditing requirements make it sound pretty reliable, and that's what customers are paying for.

The problems

This certification process has been in place for more than 10 years. How is it doing? Is it as reliable as it sounds? Is it as reliable as it ought to be?

The actual results of this auditing system are mixed. Many fine registration companies are now doing business, but some are not delivering what they advertise--unless the objective is a meaningless piece of paper. Certification/registration has become a big deal involving lots of money. As more competitors have entered the field, more variability has entered the process.

It's amazing to me the number of times I have heard companies going to great lengths to prepare for audits by developing elaborate plans to assure auditors are kept busy and the real work of the organization is not exposed.

Similarly, I have heard countless tales of auditors who had little knowledge of the organizations they were auditing, poor auditing skills or even incomplete or incorrect knowledge of the standard.

In an extreme case, I saw an auditor proceed to tell a client how it should be complying with the standard. Even worse are auditors who tell their clients how to perform a given process or what should be changed in the system to satisfy an audit finding.

Auditors aren't supposed to do any of these things. The purpose of an audit is simply to establish the standard is being met and assist the client in identifying weaknesses in its systems. The outcome of this process should be some assurance to the client and its customers the system is working and accomplishing its objectives.

Who's in charge?

The client of the audit should be in charge. The client should demand capable, honest and meaningful examinations in exchange for the money being paid. The organization's internal audits should lead to continuous improvements; the third-party process should only be a validation that the company's existing quality system works as designed.

But if elaborate measures are taken to deceive or placate a third-party auditor or if the auditor or audit process is flawed, that purpose will be lost.

A big concern is that neither the audit client nor those who rely on the certificates have direct input into the oversight and review of the third-party process. While the accreditation bodies are doing what they can within their purview, as with the audit itself, they are just sampling the process.

What some sectors have done

Sectors such as aerospace, automotive and telecommunications have supplemented ISO 9001 requirements, recognizing the value is not only in the standard, but also in its consistency of application. To control variables, many of the sector specific standards they developed mandate auditor knowledge, training and the process by which the audit is performed.

Some sectors have gone as far as to put strict controls on auditor training; others regulate qualifications and experience. In each case it is the sectors that know their processes, systems and customers. They make the determination of how an audit can determine compliance.

It's up to the customers of the auditors

The clients of the audit are in the same position as these sectors and should exercise similar rights with the registration companies. It's up to the customers to demand their money's worth, in both the process and the results. They should demand appropriate behavior and a high level of knowledge and competency from the auditor so the end result will be a thorough, accurate and complete audit.

It's not out of line to request the proof of qualifications, knowledge and experience of an auditor prior to letting him or her in the door--in fact, this information should be required. If an auditor is not satisfactory, he or she should be refused and a suitable person found. If the auditor's behavior becomes unacceptable during an audit, the registration company should be asked to discontinue the audit until a replacement can be sent.

If the audit itself is not thorough enough to test the system, a complaint should be lodged and a request made to have a more complete examination.

Sound crazy? Why? Of what purpose and value is the certification process if it has no integrity?

DALE K. GORDON is director of quality and business improvement for Rolls-Royce PLC's Defense North American Business Unit in Indianapolis. He is chair of the American Aerospace Quality Group and was one of the writers of the AS9100 standard. Gordon earned a bachelor's degree in industrial engineering from the General Motors Institute (now Kettering University) and a master's degree in business administration from Butler University in Indianapolis. He is a Senior Member of ASQ.

If you would like to comment on this article, please post your remarks on the Quality Progress Discussion Board, or e-mail them to editor@asq.org.

Average Rating


Out of 0 Ratings
Rate this article

Add Comments

View comments
Comments FAQ

Featured advertisers