This month’s question

I am trying to build a product quality scorecard that shows the quality of our product from a product development and production standpoint. What metrics can be used when assessing the quality of a system that has been deployed to production? The obvious ones include defect slippage and Six Sigma measurements, but what other measurements do you use or suggest?

Our response

Many metrics can be used, so I will suggest several and recommend a few.

The goal is to capture value-added data indicative of the quality of the product development process, the manufacturing process and the transfer process from development to manufacturing, called design transfer.

As a general principle, metrics should be simple and easy to understand, practical and tied to business objectives. Metrics also should be time-based to see trends, gain knowledge and take corrective action, if necessary.


There are many measurables that reflect the quality of the product development and production processes. I include product quality metrics and business metrics in the list because different audiences appreciate different metrics that consider the emphases of their positions in the organization. Data presented to executives at management reviews, for example, are quite different from data presented to operators responsible for manufacturing process quality.

It also is difficult to separate development metrics from production metrics because product development should include design for manufacturability early on. Potential manufacturing-related problems should be anticipated through requirements and risk analysis in design control, and implemented throughout product and process design.

Four areas of potential measurables

  1. Financial and business. When evaluating the quality of new products from development, management will be interested in financial and business metrics, including:
    • Actual revenue from a new product versus plan.
    • Percentage of total revenue from all new products developed in a timeframe, such as the last three years.
    • New product return on investment, net present value or other financials versus plan.
    • Penetration rates or market share versus plan.
    • Cost of purchased materials that were selected during development.
    • Supplier nonconformances and corrective action requests. Supplier selection and initial management activities should occur during development.
    • Inventory versus plan, backorders, ship holds, on-time shipping performance and anything that affects the availability of good quality, acceptable product getting into the customers’ hands on time. These metrics also are related to manufacturing quality.
  2. Design and manufacturing. Considering quality directly, product development and production performance can be monitored simultaneously by counting the following events and keeping track of whether the causes can be attributed to design or manufacturing:
    • Customer complaints and recalls assigned to the new product.
    • Internal nonconformances.
    • The number and significance of corrective and preventive action projects or other improvement projects that must be initiated.
  3. Design causes imply an opportunity for product development improvement, and manufacturing causes identify improvement needs in production.

  4. Production processes. A series of potential metrics related to production processes can be included, such as:
    • Process time—The actual time it takes to make the product. This is an element of cost to manufacture and ultimately part of the overall cost of goods sold.
    • Cycle time—Also called lead time. Cycle time measures the length of time it takes to manufacture the product in terms of calendar days.
    • Yield—Yield by process step or process, ultimately rolling up to an overall yield. Alternately, yields can be converted to defects per million or defects per 10,000 depending on the production rates.
    • First-time percentage complete and accurate (% C&A)—An alternative to yield, % C&A represents how much product moves through each process and all processes together without any rework or nonconformances. The difference between first-time % C&A and overall yield is that yield may consider some positive gain from rework.
    • Process capability metrics—Process capability metrics, such as Cpk, measure key process parameters.
    • Uptime—Production system availability or manufacturing uptime versus plan.
  5. Design transfer. Audits are a good tool for determining the quality of the design transfer. Consider monitoring:
    • Internal process audit nonconformances, including planned process steps being executed incorrectly or incompletely.
    • Internal product audit nonconformances.

What to include

Considering all these metrics, a list of the critical few metrics for new products implemented in manufacturing could include:

  1. Overall yield.
  2. Customer complaints attributed to design, manufacturing and supplier causes.
  3. Process time.
  4. Cycle time.

These four metrics reflect internal and external failures, and process effectiveness and efficiency. They also tie into the cost-related business metrics, which must be tracked and made visible for top management.

Depending on where the data identify issues, investigation into more detailed metrics may be necessary. If overall yield is problematic, for example, the yield by process or process step should be determined to ensure problems are resolved at the correct level. If process time or cycle time is too long, problematic process steps creating bottlenecks should be studied in detail for improvement.

Ultimately, there are many options, so internal discussions and decision-making tools should be used to determine the right metrics for your organization.

This response was written by Scott A. Laman, Teleflex Inc., Reading, PA.

--John Elwer, 12-12-2017

Average Rating


Out of 1 Ratings
Rate this article

Add Comments

View comments
Comments FAQ

Featured advertisers