In the toolbox
Q: I recently noticed surveys were added to the list of tools for data collection and analysis on ASQ’s website (http://asq.org/learn-about-quality/data-collection-analysis-tools/overview/overview.html). Aren’t these subject to the audience and tools or technology used by the survey? I didn’t see process benchmarking on the list, either. Any thoughts on either of these decisions?
San Jose, CA
A: Clearly, the design of a survey and the data collected would vary considerably depending on the nature of your product and customer base.
For example, if you have relatively few customers, selling each one a custom-made product and requiring a field acceptance visit, the survey would probably be integrated into a detailed acceptance plan.
At the other extreme, a product sold to thousands of customers would involve a considerably different survey plan. Obviously, when designing any survey, you need to ask: What information do you want? How will you organize it? How can you gather the necessary data?
Regarding benchmarking, there is some controversy here. Benchmarking can be useful if you want to see how far you must improve to at least be competitive. But you must be careful not to generate a "me too" environment that fails to encourage you to be the best provider, exceeding the current leaders.
Quality improvement consultant
Fort Collins, CO
For More Information
Pecoraro, Joel, "Survey Fatigue," Quality Progress, October 2012, p. 87.
Q: We are in the process of developing a new internal compliance auditing program. The purpose is to determine individual compliance with a management plan that helps our institution remain compliant with federal regulations.
We are having debates about how big the scope should be for our first audit. In other words, how many plans should we select for the audit? We already have selected a population to audit (calendar year 2009) and have developed criteria according to risk level, with a total of four risk categories. Each category has progressively less risk involved.
My director and I are advocating for 100% auditing of the two highest risk groups, which would total 17 plans. Each plan has between two and five points (actions) to audit. Our associate vice president thinks 17 is too many and is advocating for 25% of each risk category.
My greatest concern is ensuring due diligence should we be audited by the federal government. The federal regulations governing our work do not provide any standards for auditing. I’ve been trying to get this auditing program off the ground for three months, so any professional guidance you could provide would be extremely helpful.
A: Because your internal audit program is new, I would recommend deferring to a 100% audit. If you only have 17 plans to audit, I would make the investment and do them all. Given that you are only auditing between two and five points per plan, you will be looking at between 34 and 85 elements.
Again, make the investment and audit everything. The advantage of doing so is simple: You increase the amount of data to which you can compare when federal regulators come calling. Because you are a new group, this correlation will be valuable.
Director, continuous improvement
Lincoln Financial Group
Q: The ISO 17025:2005 standard includes section 220.127.116.11, "Reference materials." I’m wondering what is included in this section and what is mistakenly included when, in fact, it shouldn’t be. Could electrical standard resistance be considered reference material?
Jeddah, Saudi Arabia
A: First, let’s look at section 18.104.22.168 of ISO/IEC 17025, which says, "Reference materials shall, where possible, be traceable to SI (international system) units of measurement, or to certified reference materials. Internal reference materials shall be checked as far as is technically and economically practicable."
In short, electrical standard resistance cannot be considered reference material because it is a quantity. It should be considered a measurement standard or a reference standard, which is covered in section 22.214.171.124 of ISO/IEC 17025.
There is nothing mistakenly included or excluded in this section of ISO/IEC 17025, but I’ve provided the definitions of measurement standard, reference material and certified reference material according to JCGM 200:2012 (ISO Guide 99:2007) to clarify the terms.
Please pay particular attention to the following examples and notes explaining the terms in the document:
- 5.1 (6.1)—measurement standard. Realization of the definition of a given quantity, with stated quantity value and associated measurement uncertainty, used as a reference. Example: 100Ω measurement standard resistor with an associated standard measurement uncertainty of 1 μΩ.
- 5.13 (6.13)—reference material. Material, sufficiently homogeneous and stable with reference to specified properties, which has been established to be fit for its intended use in measurement or in examination of nominal properties.
- 5.14 (6.14)—certified reference material. Reference material accompanied by documentation issued by an authoritative body and providing one or more specified property values with associated uncertainties and traceabilities using valid procedures. Example: Human serum with assigned quantity value for the concentration of cholesterol and associated measurement uncertainty stated in an accompanying certificate, used as a calibrator or measurement trueness control material.
The JCGM 200:2012 document can be downloaded free here for future reference regarding any terms related to metrology.
President E=mc3 Solutions
For More Information
Shah, Dilip, "Time to Adjust," Expert Answers, Quality Progress, July 2011, pp. 8-9.