MEASURE FOR MEASURE
Four Steps To Ensure Measurement Data Quality
by Robert M. Graham
Millions of measurements are made every day in this country, ranging from the very simple to the extremely complex. Unfortunately, many of the people making the measurements do not obtain valid results because they make unrealistic assumptions about their measurement capability.
When most people make a measurement, they assume it will be good enough. This can be incorrect or even dangerous, especially when good measurements are critical for success.
To be sure your measurements are providing data that actually are good enough for the application, you should ask and answer four relatively simple questions:
- What do you measure that’s important?
- How good do your measurements need to be?
- How do you know they are good enough?
- In the future, when you need to, will you know what you measured and how?
Honest answers will ensure you are getting the results you need from your measurements.
In some cases, data that are less accurate than you believe may not have a significant impact on your decisions or operation. But in other cases, when the validity and accuracy of the data are crucial to the success of your operation, you can use the four questions to ensure the measurements and resulting data can be relied on to make the right decisions in nearly any situation.
Although the questions can apply to any type of operation, the following examples will concentrate on metrology related measurement equipment.
The first two questions will be discussed in this issue and the other two in the September issue.
Question 1: What Do You Measure That’s Important?
The first step in understanding your measurement data is to determine the many types of measurements you make in support of a specific project. The next step is to sort them in order of importance or level of impact on the project.
In this time of increasing competition and decreasing budgets, it is important to focus on the areas in which the return on investment will be greatest.
Remember: Importance is relative. A measurement that is critical to one project area at a specific time may have limited or no importance in a different project or time. It is dangerous to make a blanket statement such as, “This measurement is always important,” and apply it universally to all projects and applications.
It is both necessary and proper to initially focus on limited areas when determining the importance of measurements. If you try to focus on too much at once, it is easy to miss important measurement components.
However, you should not forget to get input from other stakeholders. Communication is critical at this step, not only with your customers but also with your suppliers and co-workers. Make certain your suppliers know your exact needs and your customers know what you can provide.
This is especially true with customers and suppliers you trust and know are competent—that is when assumptions are made that may exceed capabilities. “I thought you knew what I was asking for” or “I thought you knew our capabilities” are common statements heard after a problem is discovered. Only clear, concise communication among everyone involved can prevent this.
Once you have completed your list of important measurements, document the reasoning that went into the decisions. The documentation, which does not need to be extensive, can play a critical role in the future. You never know when you or someone else will need to reconstruct the thought processes that went into these initial decisions. Documentation of results and decisions is extremely important and should be stressed repeatedly.
There are many aspects to consider when determining the importance of a measurement. If you are using a specific piece of equipment to make a measurement, does the unit’s performance meet the minimum requirements of the measurement? Are there any legal ramifications if the measurements are incorrect?
Don’t just think about statutory or regulatory requirements; also consider the consequences if the measurement data are not accurate. Are safety issues involved that could turn an ordinarily inconsequential measurement into an important one?
What about the reputation of you or your organization? If you publish or present data that turn out to be flawed, it can hurt your standing among your peers and have a negative impact on your organization’s reputation.
Customer satisfaction is another factor to consider. And remember, customers are not just those who purchase your product or service; there can also be intermediate customers within your organization. Keep their expectations in mind when making and sorting your list of measurements.
One major measurement result can be composed of many preliminary measurements. For example, say you need to determine the mass of something using a precision electronic balance. Being conscientious, you verify the operation of the balance before use, using a certified mass standard at or near the range you plan to measure. Because you are using a calibrated mass standard, all is well, right?
Not necessarily. Many factors can affect the quality of the measurement. What is the uncertainty of your mass standard? Usually, you can get it from the calibration certificate. The mass standard’s value may have changed since the last calibration, either through wear and tear (the mass may be lighter than expected) or accumulated dust and dirt (the mass may be heavier than expected).
What is the resolution of the balance? How repeatable is the balance? Are there any environmental conditions, such as temperature, humidity, air pressure or buoyancy corrections, that should be considered?
This example illustrates the many factors that usually have an influence on your one important measurement and the need to evaluate the impact each has on your measurement. In some cases, the influence may be negligible and can be ignored safely, but you should still document that they were considered and did not affect the quality of the measurement. In others, the influence may be significant and turn out to be the most important factor considered. Again, document your results.
Question 2: How Good Do Your Measurements Need To Be?
This is an important question that can be difficult to answer in a quantitative way. Everyone would like their measurements to be as good as possible. Practical issues, such as the cost of equipment, time available, staff and safety, always limit the measurement. Increasing the accuracy (or reducing the uncertainty) of a measurement almost always increases the cost, both in time and money.
When considering the level of accuracy required, don’t just concentrate on the primary aspect of the measurement. Think about any possible secondary (or even tertiary) considerations that could impact the accuracy constraints.
For example, a signal generator used for making frequency measurements may not need amplitude accuracy—the amplitude just needs to be large enough to trigger the recording device. But if the generator also serves as a backup signal source for another measurement that does require accurate output amplitude, both applications must be considered.
In uncertain situations, it may be safer (but more expensive) to take the pessimistic viewpoint and be as conservative as possible in making your measurement decisions. It can be time consuming to go back and redo all your measurement evaluations if some important aspect was overlooked.
Another consideration concerns the accuracy and precision (the two terms are not synonymous) of your data. The accuracy of a measurement is defined as the closeness of the agreement between the result of a measurement and the true value of what is being measured. Precision, on the other hand, is the closeness of agreement between independent measurement results obtained under stipulated conditions.
Figure 1 shows the difference between the two
concepts. Mathemati-cally, accuracy includes precision of the
measurement, as well as all other factors affecting the
measurement. Thus, when deciding how good your measurements need
to be, you should determine the overall accuracy and not be
misled into using precision as a measure of accuracy. You can
usually achieve a higher level of accuracy by using better
measurement equipment or methods.
On the other hand, precision requires a very stable measurement system, environment and methods so multiple measurements of the same quantity have very little variation. Let’s look at some examples of the difference between accuracy and precision and how to determine when either or both are required.
Say you need to perform a measurement in which the temperature stability of a process over a given time is a critical parameter. In this case, the precision (stability) of the temperature measurement is important, not the absolute temperature itself.
Or suppose you need to weigh several boxes, one at a time, to ensure you do not overload a truck. For this measurement, accuracy is more important than precision. You don’t need to know the exact weight of the boxes—an approximate weight will suffice. But since you probably will not try to load the truck to within a few grams of its maximum load weight, the precision of the measurement is not as important.
As another example, if you need to supply a 10V ±1?V DC voltage level to calibrate a high-accuracy digital voltmeter, both precision and accuracy are important.
Finally, let’s say an item needs to be cooled for at least an hour before being removed from its mold. If you are a few minutes early, it probably will not affect the quality. If you are late, it also probably won’t matter—you can cool the item for as long as you like.
In this case, neither accuracy nor precision is required. By understanding what factors are important, you can make better decisions when determining your exact measurement needs.
The next step is to determine exactly how good the measurements need to be with regard to both accuracy and precision. Many times, this can be determined easily through specifications, drawing call-outs or customer requirements. In other cases, the customer may not know the level of accuracy required, or there may not be definite guidelines that must be followed. In these events, it may be easier to bracket the requirement by asking as many “dumb” questions as possible.
Ask your customer, “Would there be a problem if the measurements were off by 0.1%? How about 1%? 10%? 50%?” As long as the answer is, “That shouldn’t matter,” keep increasing the value until they say, “I would probably start to worry.” Then you can begin to get a better understanding of the required accuracy. Document the conclusions and make sure the customer agrees with them. But remember: Costs almost always increase as accuracy and precision requirements get smaller.
The second part of this column will discuss the third and fourth questions, as well as a tool you can use to help document necessary information.
This column is based on the author’s presentations at the annual technical meeting of the Measurement Quality Division and Inspection Division (September 2005, Corona, CA) and the ASQ 60th World Conference on Quality and Improvement (May 2006, Milwaukee).
ROBERT M. GRAHAM is a principal metrologist at the Primary Standards Laboratory of Sandia National Laboratories in Albuquerque, NM. He has a master’s degree in technical management from the University of Phoenix in Albuquerque. Graham is a member of ASQ’s Measurement Quality Division and is a certified calibration technician.