Rules, Tools and Good Judgment

by Norman Belecki

In reviewing the past four years of “Measure for Measure” columns, I was reminded of Phil Stein’s talent for educating those of us who were fortunate enough to have contact with him.

He had a knack for giving nonspecialists an effective view of detailed technical topics and explaining the complex of ideas that underlie most of what metrologists and quality specialists do. He was especially impressive when lecturing, and it was a pleasure to watch him gauge an audience and aim his discussion at a level appropriate to his listeners.

His columns reflect this talent as well. Each gives a complete overview of a topic and encourages the thoughtful reader to look into it further. He appeals to common ground and common sense and thus makes his subjects accessible to nearly any reader. Most columns have meaning beyond the obvious, and it is through these implications he draws us to the cohesiveness of what we do and the environment in which we do it.

A Delicate Balance

One such implied topic that resonated with my experience was the balance between the use of tools and rules and sound judgment in quality systems. We all rely on an ensemble of tools, such as analytical statistics and decision rules, to ensure the quality of our work. But more often than not, when a problem does occur, good judgment is required to resolve it.

Occasionally the use of tools and rules becomes ritualized, and the system thwarts the effective use of judgment. I’ve seen this occur on both technical and management levels, and in such cases it not only precluded resolving the immediate problem but was likely to lead to future problems.

Many metrology laboratories, especially higher level ones, use measurement assurance program (MAP)1 techniques to ensure the reliability of the data they produce. These techniques merge quality statistical practice with metrology concepts and provide a systematic means of detecting problems and investigating them. Over the years, a number of MAP short courses has been offered by training companies, academia and the National Institute of Standards and Technology.

But MAPs, too, can become ritualized and thereby less effective. There have been several instances in which clearly out-of-control data were added to control charts but no action was taken—as though the act of keeping such charts in itself would preclude problems. In a well-run MAP, the control variables are carefully chosen to identify the cause of any out-of-control situation, not just see it exists. For a complex process, setting up a MAP is not trivial. Many parameters can be measured usefully; the trick is to find the most effective combination.

A successful MAP includes frequent perusal of the data, especially in graphical depictions. Humans have evolved to be effective detectors of patterns in information and are far more effective than present day computer programs.2 Patterns that are unaccounted for are generally good indicators of uncompensated influence factors or other overlooked sources of error. Such data are analyzed to determine how to correct the process or compensate more exactly for variations of influence factors.

Open Communication

In the end, tools are just tools, not ends in themselves. Once past the simple decision that a process has gone out of control, it is up to the judgment of those responsible for the process to determine the best course of action.

The rules to make the out-of-control decision are also tools, not answers. Most real systems are sufficiently complex that generating an exhaustive set of rules for detecting and fixing problems is untenable. Sooner or later—usually sooner—a situation will require the judgment of a competent person.

Making effective judgments (having competence) requires both training and education. The latter must include practical experience with the specific process or with similar processes. This is necessary so its behavior can be understood quantitatively—especially when the process malfunctions. The more complex or accurate it is, the more understanding is required to get it back on track.

In manufacturing processes, quality and metrology are symbiotic. Good metrology requires the use of common quality tools and concepts. Quality cannot be practiced without reliable data produced through process measurements, and metrology is required to ensure the reliability of both instrumentation and data.

In fact, most modern high technology manufacturing processes rely on metrology being practiced as a part of their operation, not just in the metrology lab. This is particularly true in precision machining and electronics production and will become increasingly true as discoveries from fields such as cryogenics and nanotechnology are employed in products. It augurs the need for more formal quality and metrology education in engineering schools, even at the graduate level. Fortunately, some of this is already occurring in the mechanical engineering field.

A similar sort of problem occurs on a management level. During dinner on the night of my first ASQ meeting, the discussion at the table got around to the relationship between the quality and metrology functions in the companies of those present. For most, this was an odd topic as the Measurement Quality Division/Forum had just been formed and there were no other metrologists present.

Several people said their metrology systems had been set up to eliminate the possibility of decision making being influenced by poor data, and that was that—problem solved. They had little interaction with the lab because its activities were perceived to involve a different, specialized discipline.

Most were from relatively new and small businesses whose metrology staffs consisted of one or two people, many with backgrounds inadequate to applying common quality principles to their work. It is clear both the quality and metrology functions in such companies were handicapped by management rigidity—going by rules rather than thinking. Both would have profited from interaction.

The Consummate Facilitator

Phil Stein was a consummate facilitator. One of his constant endeavors was to bring the metrology and quality communities together by educating metrologists in the use of quality tools and by effectively advocating metrology and the Measurement Quality Division/Forum within ASQ. The existence of the calibration technician certification program, The Metrology Hand- book3 and the recent Measurement Quality Division-Inspection Division Joint Conference are sure signs he was succeeding.

Melding the metrology and quality communities, however, is a long-term process. Embedding MAP and measurement process control concepts in routine quality improvement thinking might be the best way to do this. It would certainly lead to more complete understanding of manufacturing processes, better information about the units under test and their test systems and, most likely, lower product cost.

While we may not be as effective as Phil was, we all need to step forward and help strengthen the coupling between the quality and metrology communities.


  1. Philip Stein, “Quality Tools for Metrology,” Quality Progress, July 2000, pp. 79-81.
  2. Murray Gell-Mann, The Quark and the Jaguar, W.H. Freeman, 1994, p. 89.
  3. Jay Bucher, editor, The Metrology Handbook, ASQ Quality Press, 2004.

NORMAN BELECKI is a metrologist who retired after a 30-year career in the National Institute of Standards and Technology’s Electricity Division. He holds a master’s degree in measurement science from George Washing-ton University in Washington, DC, and is a charter member of ASQ’s Measurement Quality Division/Forum, an Institute of Electrical and Electronics Engineers Fellow and a member of the American Physical Society.

Average Rating


Out of 0 Ratings
Rate this article

Add Comments

View comments
Comments FAQ

Featured advertisers