Compensate for measurement error in test results
by Philip Stein
A colleague whose company makes products for the aerospace industry wrote to me recently about a problem. Most items carried on airplanes or launched into space have a maximum weight specified, and often this is a critical parameter. In production, parts are weighed and a specification is applied. In some cases, when a part gains weight through the process (as components are added or treatments are applied), the part may be weighed several times before it's completed.
At each weighing step, a digital scale is used, and a typical instrument will resolve perhaps 1/10,000 of a pound, or four digits past the decimal point. If the drawing specifies a maximum weight of 3.5 pounds (in exactly those words), what constitutes an appropriate scale reading for rejection? Various workers, inspectors and quality engineers in this firm would accept readings of 3.4999, 3.5000, 3.5499 and even 3.5999 pounds.
The difficulty here comes from a disconnect between the specifications, with one digit past the decimal, and the instrument, with four digits past the decimal. In the absence of a harmonization of the measurement and the specification, there is no way to know which of the above readings is acceptable. Because scales and balances have different resolutions, and because drawing practices may vary according to industry or customer requirements, it's probably better to document a companywide policy on rounding and interpretation of significant figures. This may not completely prevent errors, but it will make a consistent local definition of right and wrong.
It's up to company policy
An analogous question came across my desk years ago. An engineer and an auditor had practically come to blows over a question of the end of a calibration interval. The auditor claimed a micrometer was out of calibration on the day marked on its sticker. The engineer insisted the interval didn't expire until midnight of the marked day. The micrometer didn't care; it remained in tolerance (or perhaps out of tolerance) throughout the whole argument. Both parties turned to me to settle the dispute, for which there was, of course, no correct answer.
It's up to a company policy to state exactly when a tool may no longer be used because its calibration has expired. Some organizations allow use of an expired tool through the end of the expiration month. All these choices are likely to yield the same result, so it's only a matter of policy.
Suppose, for example, you decide a reasonable policy is that the maximum weight of a part should be 3.5000 pounds. Rounding confusion will be eliminated, but measurement error won't. Suppose the part actually weighs 3.5021 pounds, but the scale reads 3.4996. This will result in a type two error (customer's risk), and the part will ship when it shouldn't. Similarly, a part that actually weighs 3.4988 pounds, but reads 3.5002, will be incorrectly rejected, a type one error (producer's risk).
A fairly new, relevant international standard, ISO 14253-1, refers to dimensional measurements, but really applies to anything. The issue is that anytime you are close to the edge of a specification, measurement error could take you over the edge in either direction. In this situation, the standard says the part cannot be called good or bad, but has to be rated as uncertain. This makes good technical sense but poor commercial sense. After all, a rating of uncertain won't make your customer or your production department happy. No one will know what to do with the part, and no one will want to accept or reject it.
The same issue comes up when a laboratory issues a calibration certificate. The customer just wants to know if the instrument is within manufacturer's specifications. Sometimes this is stated as in tolerance or out of tolerance. In either case, a firm decision is required.
ISO/IEC 17025 states, in clause 220.127.116.11, that when statements of compliance to an identified metrological specification (in spec or out of spec) are made, measurement uncertainty must be taken into account. The requirement here is clear: Use some rule like that of 14253-1 so measurement error is not ignored. This will satisfy the requirements of the standard but make the customers angry. After all, "uncertain" is not seen as a firm decision.
Leave it up to the customer
My favorite approach here is the most straightforward and bypasses the requirement entirely. If you report a value from the measurement and report the uncertainty, both without making a statement of compliance, you leave the decision up to the customer as to how to interpret an answer close to the edge of the specification. Since this is a business decision rather than a technical decision, it is better left in the hands of the customer.
However, in the original example, the maker of the measurement and the customer for the measurement are the same organization. It's still a business decision, though--it's just one in which the risk of shipping marginal or defective product is balanced against the costs of rejecting usable material. As a quality engineer, you should be able to quantify those risks so the appropriate decision is made.
Once it is decided how to interpret uncertain measurements, the results are often stated as guardbands. A guardband is an artificial change in a specification in order to compensate for measurement error.
In the case of the weight of an aircraft component, a medical
device or even the contents of a box of cereal, there are practical, safety
or regulatory reasons to set an absolute limit. In the example, the part as shipped should actually weigh 3.5 pounds or less, period. If the uncertainty of the weighing measurement is 0.01 pounds, set the guardbanded specification at 3.49 so the part will never actually exceed 3.5. The guardband makes the specification appear a bit smaller than its actual value.
In the case in which the customer (internal or external) can tolerate a part that might be a tiny bit over the specification value, set the guardband to make the specification appear a bit larger than its actual value. If the part is listed as 3.5 pounds and the uncertainty is 0.01 pounds, a guardbanded specification of 3.51 pounds will accept all material whose weight is in spec at least within our ability to measure it.
If this situation is intolerable either to you or to your customer, you can make a more precise measurement when possible and make the guardband smaller.
PHILIP STEIN is a metrology and quality consultant in private practice in Pennington, NJ. He holds a master's degree in measurement science from the George Washington University in Washington, DC, and is an ASQ Fellow.