MEASURE FOR MEASURE
Effective hardware management can boost your bottom line
by Stephen Doty and Del Caldwell
Are your customers dissatisfied with your products due to marginal performance or interoperability problems? Are you experiencing schedule delays, callbacks or degradation of services you provide? Are you noticing an increase in warranty returns and test failures that result in decreased margins and increased rework, scrap or replacements?
If you answered yes to any of those questions, your measurement information may be of such poor quality that it prevents you from making sound decisions related to your product or service.
The reliability of the decisions made using measurement information that results from testing is heavily influenced by the relationship of the product’s test tolerances and performance distribution to the measurement process uncertainty. The smaller the uncertainty is, the better the decisions.
Traditionally, the quality of measurement information is influenced by the management of three things:
- Conditions: This includes controlling the situational influence factors, investing in the training of personnel and maintaining a robust internal quality review.
- Processes: This includes rigorous test design, documenting and validating procedures and software, and using process control measures.
- Measuring and test equipment (M&TE): This includes ensuring the equipment meets the environment and process conditions for the proposed use, and calibrating the M&TE periodically to maintain the desired levels of measurement performance.
After you purchase the M&TE and verify it is suitable for the intended applications, its level of performance degrades with time and use. Accordingly, periodic calibration is required to maintain the needed performance levels.
Failing to calibrate your M&TE often enough can lead to degraded performance and a negative impact on deliverables. Calibrating too often may raise the performance of the equipment but at higher support cost and reduced availability.
The need to calibrate your M&TE is clearly identified in ISO 9001:2008 sub-clause 7.6, which states: "Where necessary to ensure valid results, measuring equipment shall … be calibrated or verified, or both, at specified intervals, or prior to use, against measurement standards traceable to international or national measurement standards."
Metrology and calibration laboratories are plentiful, but questions remain. How do you specify the calibration requirements for your M&TE? How do you manage the process of getting it calibrated so the service you receive meets your needs and that the equipment is calibrated only as often as needed? And how do you ensure all of this is cost effective?
This basic answer is to define an M&TE calibration system or program that manages and satisfies your calibration needs via a traceable system of competent calibrations (see Figure 1). The documentation of this calibration system would typically be included in the organization’s quality manual and would also apply to suppliers, depending on the influence they have on the company’s products.
Still, more questions remain. Do you need to start from scratch to define your calibration system? What elements should you include in the system requirements? How will you know your system will produce measurement results compatible with other organizations?
The answers to these questions are found in a national consensus standard from the American National Standards Institute (ANSI) and the National Conference of Standards Laboratories International (NSCLI), Z540.3-2006—Requirements for the Calibration of Measuring and Test Equipment.
Examining the standard
In 2003, a group of experts from industry and government met to develop a standard for establishing the technical requirements for M&TE calibration. The resulting standard bridges the gap between the ISO 9001 requirement to have your M&TE calibrated and having the calibration performed in a purposefully designed process.
Conceptually, the standard provides for managing M&TE performance through a system of functional components. Collectively, these ensure the equipment’s accuracy and reliability are in accordance with identified performance requirements. The standard brings a calibration systems approach into any organization looking to improve its products, services and bottom line through the effective management of M&TE performance.
In many organizations, M&TE is used in R&D, testing, evaluation, production and support of products and services provided to a wide range of internal and external customers. The information gained from the use of the equipment contributes to an organization’s knowledge of its product or service and to the associated decisions about quality and suitability for the intended applications.
The validity of measurement results and decisions made based on those results are significantly affected by the accuracy of M&TE. As a result, the equipment is of particular importance to the organization’s success.
The standard prescribes requirements for a calibration system that ensures the continued accuracy of M&TE used to support the organization’s endeavors. The scope of the implementation may be limited to the implementing organization when the calibration system is within the organization.
Or, when the system extends beyond the organization, it may include a flow down through contracted or outsourced functions. If that’s the case, the organization may implement the standard based solely on business strategy or on meeting customer contract conditions or requirements.
Part of a system
The types and applications of M&TE, the calibration of which may be managed using a calibration system, are broad. They may include equipment used for final testing of a complete product or components of the product, or equipment used directly in providing a service, such as in the healthcare industry.
M&TE that plays a more indirect role—such as supporting R&D, monitoring utility parameters or evaluating component selection—is also addressed by the standard and is included in a calibration system in which the equipment may affect product or service quality. From a practical standpoint, it’s not often the equipment is expected to be excluded from calibration system management unless there is a clear rationale for the decision.
Converting M&TE performance requirements to specific requirements for use in the calibration system may require further interaction between the end user of the equipment and the organization’s quality, calibration and engineering operations. M&TE calibration requirements are parameters used by the calibration system to ensure the equipment meets the defined performance requirements.
The principal parameters are:
- Calibration required or not.
- Measurement quantity test requirements (such as ranges, values and tolerances).
- Allowable measurement uncertainty or false-accept risk of the calibration.
- Allowable measurement reliability or uncertainty growth at the end of the calibration period or interval. Note that the allowable uncertainty growth is the magnitude of the difference between the uncertainty of the calibration results and the M&TE’s measurement uncertainty requirements.
These parameters control the degree to which M&TE managed by the calibration system meets the performance applications and the organization’s expectations. Figure 2 shows an overall view of the functional components and process flow within an organization’s calibration system.
Compared with previous standards, Z540.3 provides mechanisms to improve the quality of M&TE performance, its management and the related calibration services. Some of these improvements include:
- Adding M&TE application performance criteria to ensure the compatibility of calibration intervals and calibration services.
- Improving requirements for calibration procedures, including use objectives, tolerance test criteria, measurement decision risk criteria, test uncertainty ratio determination and validation criteria.
- Requiring traceability to the International System of Units and improving compatibility with the official measuring system from the U.S. National Institute of Standards and Technology.
- Providing improved criteria for calibration intervals and the use of measurement assurance processes.
- Providing requirements for optional use of accredited calibration services.
- Reducing the number of overall requirements.
- Adding consolidated requirements for assessment, quality control monitoring and system improvement.
Of these, three topics raised the most discussion during the early development of the standard: measurement decision risk, test uncertainty ratio and use of calibration laboratories accredited to ANS/ISO/IEC 17025. These topics and others are addressed in the NCSLI’s Handbook for ANSI/NCSL Z540.3. In addition, accreditation bodies now offer assessments of calibration laboratories to Z540.3 requirements.
And finally …
During the development of the new standard, the NCSLI-sponsored ANSI Accredited Standards Committee Z540 invested a significant amount of time and effort ensuring the requirements for M&TE calibration fulfilled needs unmet by other standards in a way that would improve compatibility with international standards. The committee was also conscious of the need to express the requirements in a straightforward and explicit manner to facilitate implementation and interpretation.
As with any change, there are costs and benefits the standard brings. But, in the end, the benefits to all stakeholders and the bottom line outweigh the modest costs.
Stephen Doty is program director of the U.S. Navy’s Joint Naval Audit Certification Program in Corona, CA. He received an MBA in defense acquisition from the Naval Postgraduate School in Monterey, CA. Doty chaired the commission that developed the handbook for ANSI/NCSL Z540.3.
Del Caldwell is owner of CCG, a consulting group in Claremont, CA. He earned an associate’s degree in science from San Diego City College. Caldwell is past president of the National Conference of Standards Laboratories International.