Auditing ISO 9001:2000
Challenges arise because some of the requirements aren't specific or traceable
by J.P. Russell
Because many of its clauses are nonprescriptive--they don't contain specific or traceable requirements--some quality professionals are expressing concern about the auditability of ISO 9001:2000, the new quality management standard.
"Auditability" refers to the capability of applying audit techniques for positive verification of requirements. If conformance to requirements is not verifiable and traceable, the credibility of the audit function (audit program, registrar or regulatory oversight) could be questioned.
First, however, it is important to point out that most of the ISO 9001:2000 clauses contain very specific and traceable requirements. For example, the internal audit clause ISO 9001:2000: 8.2.2 requirements include the following:
1. A documented procedure.
2. Auditing at planned intervals.
3. Audit planning based on status and importance.
4. Reporting of results.
5. Maintaining of records.
6. Taking action on detected nonconformities.
7. Verifying of actions.
8. Reporting of verification results.
Auditors can use various techniques to verify specific audit program requirements have been addressed in the quality management system. They can evaluate documents and verify records; interview document users to verify the process described by the document has been established, implemented and maintained; and trace the process forward or backward to verify activities are being performed. Verification of specified requirements is fast, efficient, reliable and traceable.
The technique for verifying specified requirements can be stated as, "Show me the document, record, procedure, plan, schedule, material or activity."
Less prescriptive or nonprescriptive clauses
Most can agree audit evidence should be verifiable and traceable for both conformance and nonconformance to requirements. But the less prescriptive clauses may have no requirement for a procedure, schedule or record. Without prescriptive requirements, the traceability between the standard and the users of the quality management system is less obvious and may be suspect.
One clear example of a nonprescriptive clause in ISO 9001:2000 is element 7.5.5--preservation of product. The organization is required to "preserve the conformity of the product during internal processing to final destination." There is no requirement for a procedure, schedule, inspection, plan, method or record.
This requirement is so general it is possible for an organization being audited to believe it can simply declare conformance to the requirement and challenge the auditor to prove otherwise. Because that can be like looking for a needle in a haystack, it is important for the auditor to take the opposite approach by challenging the auditee to show why there is no needle in the haystack.
To audit the less prescriptive clauses, the auditor must verify the organization conforms to the intent of the requirements of the standard by determining whether an approach has been established, implemented, maintained and improved.
This technique lends itself to several of the clauses of ISO 9001:2000. For some clauses the auditor must seek to determine the existence of a process, how it was planned and implemented, its outcomes and whether management determines ongoing effectiveness.
What to ask
An auditor might ask the following questions for the less prescriptive clauses:
- Is there a plan or method for conforming to the requirements?
What is it? Has it been established? Audit evidence may include
an outline, flowchart, markings in a work area, procedure, work instructions,
specifications or criteria. Clause 7.1 contains requirements to be considered
for planning the realization processes.
- Has it been implemented (deployed)? Audit evidence may include the existence of records, corroboration by multiple interviews or observations.
- Have planned results been achieved? Audit evidence may include trend diagrams, records, bar charts, matrices or comparisons. Data collected to meet clause 8.4 requirements may be helpful (for example, data summaries, analyses, metrics and performance indicators).
- Is there improvement? Has the system/process been changed? Audit evidence may be changes to documents, designs or the ways business is conducted.
The auditor should keep a record or log of the audit evidence to show conformance and nonconformance for traceability and to provide consistency from audit to audit. Examples of audit evidence and perhaps log entries may be included as part of the audit report.
To start, the organization being audited might describe how the requirements are addressed in the quality manual--as an overview, executive briefing or in procedures. Based on the description and the requirement in the standard, the auditor can interview personnel to collect audit evidence, using either open-ended or closed-ended interview question techniques.
A closed-ended question will result in specific yes, no or item by item answers. Normally open-ended questions give us more information, but it is up to the auditor to sort and determine the relevance of the information.
The same is true for nonprescriptive requirements. They are open-ended, and it is up to the auditor to match the information with the requirement and determine relevance. In some interview situations, an auditor may be surrounded by people ready to provide information needed to verify conformance to the requirement. As one person leaves to find a memo or checklist, another is ready to seek out the next piece of evidence (record, procedure or chart) the auditor needs to verify conformance to requirements.
In each situation the auditor must determine the appropriate data collection plan to ensure the information is free from bias. A worksheet is an ideal tool for listing the clause or requirement on the left and recording the evidence provided in a space to the right.
The technique for verification of a nonprescriptive requirement is, "Show me how you conform to this requirement with the existence of a plan (approach), its implementation, achievement of planned results and continual improvement."
The key is to follow the plan by:
- Examining the plan addressing the requirement.
- Examining the implementation of the plan.
- Examining the achievement (outcomes) of the plan.
- Examining the improvement of the plan.
Determining conformance or nonconformance
Once the organization being audited has had an opportunity to provide evidence and the auditor has made his or her observations, it is time to determine conformance or nonconformance.
Good audit practice also requires the auditor to indicate the importance of the nonconformities detected. Some nonconformities represent high risk to the organization, while others represent low risk. One of the simplest methods to gauge importance is to classify nonconformities as major or minor. Each organization should establish its own classification system.
The auditor must make a judgment based on the data presented and audit program guidelines to determine if there is conformance, a minor nonconformance or major nonconformance. The credibility of the audit function will be questioned unless there is consistency in judgment between the auditor and organization being audited.
The measurement system should be fair, unbiased, consistent and standardized. One method is to assess first the planning and implementation and then the results (outcomes) of the process. For example, an auditor's guideline for assessing the planning and implementation may state:
- Major nonconformance: No process is evident from the information presented, or there is partial implementation but significant gaps still exist.
- Minor nonconformance: There are sound methods but some minor gaps in deployment.
- Conformance: Sound methods are fully implemented.
For assessing the results and outcomes, the guidelines may state:
Major nonconformance: There are no data, limited data or data that can't be assessed against criteria.
Minor nonconformance: There are some trend data, and they can be evaluated against objectives and criteria. But the data are not comprehensive or being maintained.
Conformance: Comprehensive and current trend data can be evaluated against ISO 9001 criteria to determine conformance.
Preparing for the audit
To audit to ISO 9001:2000 and other nonprescriptive standards, auditors will need to use new audit techniques to verify conformance and provide traceability. In preparing for the audit, auditors may request the completion of a survey that spells out how requirements are addressed.
Auditors may also evaluate quality management plans and objectives. At the opening meeting, the methods and techniques that will be used during the audit should be shared with the organization being audited.
During the performance of the audit, open-ended techniques
should be used to verify the intent of the requirement has been addressed.
Observations should be recorded on checklists, in log books or by completing
worksheets to ensure traceability of conformance as well as nonconformance.
The degree or importance of what was found must be determined and reported
to the client and area or organization audited.
Of course, auditors must be competent to be able to determine conformance or detect nonconformances.
To work toward the goal of auditor competency, the Registrar Accreditation Board and the ASQ Quality Audit Division are forming a working group to study the criteria for auditor competence for internal and external audit programs.
If you are interested in participating in the working group or being part of a larger contributing group, contact Terry Regel, ASQ Quality Audit Division chair, at firstname.lastname@example.org or Cindy Miller, Registrar Accreditation Board certification and course accreditation director, at email@example.com.
The information in this article is an adaptation of material taken from the ISO 9001:2K Delta Web based training class owned by J.P. Russell and Associates.
J.P. RUSSELL is president of JP Russell &
JP-Russell.com), a quality management training organization and sponsor of Web based training programs. Russell is author or editor of several Quality Press books: The ISO Lesson Guide 2000, After the Quality Audit: Closing the Loop on the Audit Process (both editions), Puzzling Auditing Puzzles, Puzzling Quality Puzzles, The Quality Audit Handbook, Quality Management Benchmark Assessment and The Quality Master Plan.