A Case Study Demonstrates
The Value of ISO 9000 Derivatives
Case study shows different needs exist when safety is at stake
by Dale K. Gordon
Those of us who are in the standards writing business need to keep reminding ourselves why we put those supposedly noble words on paper to begin with.
I recently had the opportunity to review the operations of a supplier in the aerospace business that I will call "Bigalow, Irwin & Gutman Parts Inc.," or BIG Parts for short.
The occasion for my visit to BIG Parts resulted from an escalation of customer complaints about some quality problems that were discovered during a review of its quality processes. A customer had performed a routine product acceptance audit during the parts delivery process and uncovered some disturbing items.
It seems BIG Parts has an excellent reputation in the design, development and fabrication of some pretty sophisticated hardware used in the flight systems of military aircraft. Its products perform very well and are recognized as being on the cutting edge of technology.
BIG Parts recently finished development of some very technically difficult products that will go into the next generation of aircraft for the U.S. military and will also be used by the air forces of our allies.
The United States Department of Defense (DoD) decided some time back to adopt commercial practices whenever it was feasible to reduce procurement costs and prevent overspecification of weapons systems and other DoD purchases. This decision was widely viewed as a correct course of action and heartily endorsed by the defense industrial base.
This means, of course, that auditors and other quality professionals must take the generic words of the ISO 9000 standards and their derivatives and make sure that we apply them with the degree of rigor and knowledge fitting the products and components to be produced.
Well, our friends at BIG Parts seemed to believe that the ISO 9001 requirements were a "no brainer" and went about their business of designing and manufacturing sophisticated hardware upon which the lives of men and women in the armed forces would depend. They had all the necessary requirements for the 20 elements of ISO 9001 well-documented. They performed internal audits, and BIG Parts' senior management even got together occasionally to perform a management review of the operations.
Here is a blow-by-blow account of what was uncovered during a quality system review of BIG Parts.
All the right words were in the procedures, and the company does have a quality policy practically tattooed on each employee. The quality manager hung charts on the wall with care.
But the quality personnel worked for the functional organizations, and no one was clearly responsible for solving quality problems involving either the systems or the products. A certified operator program was in effect, but there was no way for the operators to initiate corrective action or have the organizational freedom to prevent nonconformities.
This resulted in processes that were not capable and in repeated nonconformances. No actions were taken to correct the problems; instead, there were continual material review board actions.
Procedures were written around ISO 9001 requirements, but personnel did not know what the procedures said and were never trained on how the procedures should be applied.
Manufacturing engineers, who were never trained on such things as gage accuracy, gage repeatability and reproducibility, or inspection methods, were used to perform the quality planning. This resulted in operators using gages that were not suited for the required accuracy or, in many cases, using unproven (although calibrated) gages for the features being inspected.
Contract review was handled by the program management part of the organization. When the program transitioned from the development phase to production, there was no review of the production contract by the quality or production organizations.
Consequently, BIG Parts missed all the customer flowdown supplier control issues and production inspection requirements such as sampling plans and gage verifications prior to production. In several cases state-of-the-art inspection methods were being used, but the risks associated with their use were not evaluated with respect to product acceptance.
The customer supplied computer models regarding what the design parameters should be. These models were translated into numerical control (NC) and coordinate measuring machine (CMM) programs.
When it came time to verify whether the parts and details met the engineering requirements, there was no definitive agreement between the customer and BIG Parts on how the items should be measured. (In other words, there were no complete drawing definitions or complete dimensions to use to verify if the parts met the design intent.)
Document and data control
While there was adequate document and data control, the procedures were all maintained online, and only certain personnel had access to the computers.
Most personnel were not trained or even knowledgeable about how to access the procedures or locate the ones applicable to their jobs. This resulted in most tasks being performed by using the "tribal legend" method--or worse, by using any method that made sense at the time.
While the selection and approval of suppliers were adequate, customer specifications and essential technical information were not required to be flowed down to subtier suppliers, and the subtier suppliers were not controlled in any fashion.
Also there was no system to indicate what happened when supplier performance began to fall and at what frequency the reviews should be performed before there was a need to obtain corrective action or replace the supplier. This resulted in suppliers that missed technical requirements and receipt of a large amount of noncompliant product.
Customer supplied product
The customer had directed the use of certain raw material suppliers by BIG Parts because these suppliers were known to have proper controls in the critical manufacturing processes.
This led BIG Parts to believe that no oversight of these suppliers was necessary. While test reports from these suppliers showed that the material was within specification, BIG Parts performed no independent verification or evaluation of the material. This resulted in failed product and arguments with the customer over responsibility.
Product identification and traceability
While there were procedures and processes in place to trace critical parts and assure proper identification, individual items were not traceable to specific processes or inspections. This precluded assurance that the product met all of its requirements. Likewise, the product could not be traced to processes that were later determined to be noncompliant.
BIG Parts had an effective process documentation system, including shop travelers, appropriate operator sign-off and inspection verifications. However, when changes were made to the process or workarounds were required, there was no control of these alternative methods or verification that the process changes achieved the same level of quality that the original process possessed. Also NC machines and CMM programs were not controlled or proven to meet design intent.
Inspection and testing
Inspection plans were in place, and proper inspection points were available. However, sampling inspection was only utilized upon initiation of part manufacture. There was no evidence that the process was capable or that critical characteristics were adequately controlled before statistical sampling began.
Control of inspection, measuring and test equipment
While a robust calibration and recall system was in place, there was no verification of the measurement software used in some of the advanced measurement equipment.
Inspection and test status
BIG Parts' employees were all assigned stamps, and there was effective stamp control. However, there was no evidence that the stamps were used to preclude product that had not been stamped appropriately.
Control of nonconforming product
While nonconforming product discovered internally was segregated and reviewed, there was no control over suspect product returned from the customer. There was also no mechanism for reporting to the customer those nonconformances that might have already left the quality system.
BIG Parts prided itself on the thoroughness of the corrective action process it employed. One small problem was that this same process was not carried through the supply chain to assure supplier corrective and preventive action.
Handling, storage and delivery
The procedures for handling the parts and shipping them were adequate, but there were no controls to assure that the parts were properly cleaned, that foreign objects were removed to prevent blocked oil passages, or that the documentation that accompanied the shipment was not lost or destroyed.
Control of quality records
No problems were found in this area.
Internal quality audits
The BIG Parts internal audit process was to list all of its procedures and have managers audit the areas not under their control.
While corrective actions were sometimes written, the audit process contributed little to the overall improvement of the quality system. Such items as customer complaints, scrap rates and nonconformance trends were not used to determine where audits should be directed. Checksheets against requirements, formal audits and protocols were not used.
There were comprehensive records of training requirements for all employees in the facility, but nowhere did BIG Parts require or document the training of employees on the quality management system (QMS) procedures or changes to existing procedures.
BIG Parts created service manuals and instructions for support of its products, but there was no mechanism for collecting and analyzing product usage data to put back into the design requirements or to correct product problems in service.
Sampling plans employed were based on Mil-Std-105D, and BIG Parts used a sampling number in which known defectives could exist in a manufacturing lot. The sampling plan was valid, but it did not use a zero acceptance number, thereby allowing for the possibility that defectives could exist in the lot.
ISO 9001 is not always enough
BIG Parts has an ISO 9001 certification from an accredited certification body, but it can be argued that the deficiencies noted above are not directly covered in the standard. But the deficiencies are implied. How should a company determine exactly what quality requirements are needed?
I think we all recognize that the design, application and implementation of the QMS should fit the product and processes used by the individual company. But the fact that the ISO 9001 criteria are considered the minimum requirements for a QMS means that in instances where the product or service is critical, more definition is required.
AS9100, the ISO 9000 derivative for aerospace, begins to articulate those requirements in an industry where people's lives are at stake.1 Even the interpretation of standards such as AS9100, QS-9000, TL 9000 or others needs to be carefully considered in light of the product, process and services being provided.
The more we try to make requirements generic, the more we need to make sure that the interpretations of the requirements fit the application.
1. AS9100 Quality Systems--Aerospace--Model for Quality Assurance in Design, Development, Production, Installation and Servicing (Warren-dale, PA: Society of Automotive Engineers Inc., 1999).
DALE K. GORDON is director of quality and business improvement for Rolls-Royce PLC's Defense North America Business Unit in Indianapolis. He is chair of the American Aerospace Quality Group and was one of the writers of the AS9100 standard. He earned a bachelor's degree in industrial engineering from General Motors Institute (now Kettering University) in Flint, MI, and a master's degree in business administration from Butler University in Indianapolis.