Calibration Buyers, Beware!
What to look for when considering a commercial calibration laboratory
by Philip Stein
Measurements are pretty useless without calibration.
Of course, you can buy or build a new instrument and trust that
the answers it gives you are okay, but sooner or later you should
wonder how long
that trust is warranted. All measuring systems are subject to change after time, and sometimes outside influences (being overloaded, mishandled or dropped) can have an effect as well.
Calibration and the choice of how often to calibrate are dependent on economic risk. Calibration costs something, often quite a bit. And not calibrating will eventually cost a great deal when the measurement is wrong and expensive rework or a catastrophe is the result. The calibration interval is therefore an economic tradeoff in which you balance the frequency of calibration against the cost of calibration and against the potential consequences and costs of not calibrating.
Regulators and international standards (such as ISO 9000) play a part as well. Most of the standards that require measurements also require periodic calibration. This is done to increase the trust that customers place in the measurements provided by an organization. In addition, all standards and regulations that require calibration require traceability--the calibration must be able to be related to national standards (except in some unusual circumstances).
Now comes my warning to buyers. Calibration can be achieved in many ways. You can do it yourself--even managing the traceability in many cases. However, this requires expertise and can be quite expensive. If you're part of a large industrial or government organization, you may decide that it's cheaper to do it all inhouse, and you may be right.
For the rest of us, though, the method of choice is usually to hire a commercial calibration laboratory to perform the work. Even if you do the simpler hand tools and electronic meters yourself, you may very well need to use a commercial service to achieve traceability between your work and the top of the pyramid. If you're already near the top of the pyramid, you will probably send your reference standards directly to the National Institute of Standards and Technology or to the standards body of another country.
Commercial laboratories are businesses and, as such, compete on several fronts. One is price and another is quality of service, neither of which differs significantly from other retail operations.
Calibration labs may also choose to register their quality systems to ISO 9002 or even ISO 9001. As a quality professional, you know how to factor registration into your buying decision, but beware: ISO 900X registration does not testify to the correctness or traceability of the answers a laboratory provides. As with all other ISO 900X processes (at least the 1994 version), registration only addresses the quality system and not the quality or fitness for use of the final product.
You may infer that a registered company is more likely to do a good job because it has a conformant quality system, and you will probably be right, but it's no guarantee.
Calibration and testing laboratories do have a way of distinguishing themselves--laboratory accreditation. Being accredited to perform specific work offers strong assurances that the answer--the work you're paying for--will be correct and traceable.
Laboratory accreditation is with reference to ISO/IEC Guide 25, recently replaced by ISO/IEC Standard 17025 but still in use for the next year or two. In addition, the U.S. standard ANSI/NCSL Z540-1 adds unique U.S. requirements to Guide 25.
While there are no absolute guarantees and nothing in this world is perfect, accreditation is the best evidence available that a calibration supplier is trustworthy, traceable and likely to get the right answer. (Testing labs can be accredited too, but this is a measurements column so we're discussing the issues using calibration as an example).
Accreditation is accomplished through a third party process very similar to ISO 9000. Accreditation bodies send out assessors who collect data, evaluate the applicants and, eventually, issue certificates. Accreditation bodies acknowledge each other through mutual recognition agreements and accreditation cooperatives. Three such cooperatives exist: a European cooperative, an Asian-Pacific cooperative and one that is being organized to cover North America.
Here's another reason to beware. The accredited laboratory you use may or may not be accredited by a body that is a signatory to one or more of these cooperatives. If it is not, its work might be fine, but it probably won't be recognized as accredited. This is especially important when something you have had calibrated needs to be traceable. Traceability delivered by organizations that aren't members of these clubs may not be good enough to pass muster by a thorough assessor.
By the way, these clubs aren't exclusive because they discriminate. Any organization can join, but it will be thoroughly assessed to ISO/IEC Guide 58, the standard of behavior for accreditation bodies.
Confused yet? There's more ...
The next item to be aware of is the scope of accreditation. Each accredited laboratory is issued a scope indicating the measurement parameters and ranges for which it is accredited. In addition, every accredited organization is permitted to use the logo of the accrediting body on their advertising, product literature, and calibration reports or certificates. Therefore, even if a laboratory advertises that it is accredited and places a logo on its reports, it may not be able to do your job within its scope--and it's up to you to find that out.
Some labs will accept both accredited and nonaccredited work (and maybe charge different prices). If a quotation, solicitation or certificate has a logo but some of the work is not within the scope, it must disclose that clearly. But continue to beware: The rules are clear but you need to pay attention as your supplier may not emphasize these distinctions.
Are there circumstances under which you must use accredited labs? Well, yes and no. While there are no fixed rules, your customer or regulatory agency may require that you use an accredited lab. Similarly, an accredited lab may be necessary for some projects and measurement parameters, but not for others.
Accreditation and the automotive industry
The most important regulation, however, comes from the automotive industry. Paragraph 4.11.2.b.1 of the third edition of QS-9000, states:
Commercial independent calibration facilities shall be accredited to ISO/IEC Guide 25 or national equivalent, or shall have evidence, e.g. assessment by an OEM [original equipment manufacturer] customer or an OEM customer-approved second party that they meet the intent of ISO/IEC Guide 25 or national equivalent.
Similar rules are stated for commercial testing laboratories.
At first glance, this seems pretty rigorous, but actually, it's as porous as Swiss cheese. The laboratory you use could be accredited, but not by a body that has signed one of the collective agreements. In fact, you could use an accredited laboratory that does not have an approved scope that included the particular services you need, but you'd still be in conformance with the standard. Your lab could be accredited to perform the required services, but it will charge you more for an accredited certificate. You could have contracted the cheaper work and saved some costs, while still following the QS-9000 standard.
To make matters even worse, the auto industry has been under quite a bit of pressure to ease up on the rules. QS-9000 registrars would like to get a piece of the business but don't have the technical knowledge or experience to pass muster with Guide 58. Some of these companies are offering "registration to the intent of Guide 25" or "registration of the laboratory quality system to Guide 25." Sort of an ISO/IEC Guide 25-Lite!
My understanding is that, at least for now, some automakers are viewing these alternatives as acceptable responses to 4.11.2.b.1. Some registrars are actually trying to gear up to deliver ISO 17025 accreditation, but it will take a while for them to sign on to a cooperative agreement. What a mess!
What's an organization to do?
The first thing you need to do is decide whether you or your customers want and need fully accredited calibration services. If your customers are demanding it, it's a no-brainer. If not, it's still a very good idea because of the high degree of confidence you can place in the results. You will then need to balance extra costs against what that high confidence is worth to you.
Personally, I think that confidence is priceless, but that's my bias. It's only fair to disclose at this point that I am a Guide 25 lead assessor under contract to the American Association for Laboratory Accreditation (A2LA), one of two U.S. signatories of the laboratory accreditation agreements.
You can pressure your regular calibration supplier to become accredited if it is not already. If you have the expertise, you can assess the supplier to the requirements of 17025 yourself rather than require that it become accredited. But if the supplier goes through the trauma with you, it might as well do so for a third party accreditation body and get its ticket punched for real.
See the sidebar "Checklist for Buying Calibration Services" on p. 113, for a summary of everything you need to look for when interested in calibration services. While the checklist is written as though you need fully conformant services, you can certainly waive some of the requirements due to costs or simply because you don't think you need accredited work.
PHILIP STEIN is a metrology and quality consultant in private practice in Pennington, NJ. He holds a master's degree in measurement science from The George Washington University, in Washington DC, and is an ASQ Fellow. For more information, go to www.measurement.com.