ONE GOOD IDEA
Tips on Effective-Quality
Defining quality throughput using constraint management
by Boaz Ronen and Alex Coman
We were asked to measure the effectiveness of a large commercial bank’s audit department. The department’s output consists of recommendations for action—scored from one to five according to the level of potential risk. The common measurement is the number of recommendations that the department makes during the year. A better measurement limits the count to severe recommendations. But these measurements have two weaknesses:
- They encourage auditors to make more recommendations—in other words, reducing the department’s signal-to-noise.
- The measurements ignore the auditors’ actual impact—in other words, the effect they have on the organization.
It is essential to measure effective-quality (EQ), which equals throughput divided by output. Throughput is the effective output of the system.
Back to the bank’s audit department, the EQ is measured as the number of recommendations that were implemented by the organization divided by the number of recommendations actually made. The EQ—represented in decimal points—can only be refined by counting the most important recommendations (scored four or five). To evaluate such a department, we use additional measurements, especially the absolute throughput of the department. To assess continuous improvement, however, we use the EQ measurement.
Making a sale
Salespeople are measured by the amount of sales made. To focus on improvement efforts, you must identify where waste hurts an organization. The EQ measurement highlights areas where a high number of salesperson meetings with potential clients do not result in a sale. This leads to cause-effect analysis on garbage time.
A study by the pharmaceutical company Merck shows that if 100 sales reps call on a physician, 17 depart without seeing the physician because competitors’ reps are waiting for the physician and 27 drop off samples at the receptionist’s desk because the physician is unavailable. Less than 24 conduct meaningful two-way discussions with the physician and are remembered because the average call lasts 4.6 minutes.1 An EQ of 0.24 is unacceptable, and Merck urges "new ways of engaging physicians on their terms … with greater productivity."2
We want to assess the throughput—the effective output—and measure its ratio to the system’s output. We noticed that many organizations do not even measure the output of a process, but assume that the system’s input (budget and person-months,3 for example) accurately reflects its output.
For example, many IT departments in financial organizations are measured by person-months invested rather than by their effective throughput (for example, the number of features delivered, value added to the organization and bugs fixed).
Measuring the effectiveness of the New York Police Department (NYPD) is another example. Before Bill Bratton’s tenure as the city’s police commissioner, NYPD seemed waste prone: widespread patrols of the subway system, massive paperwork and attempted arrests during daytime hours. When measured in terms of input, this seems reasonable.4
The waste is only salient when effective actions are measured. Bratton virtually eliminated wasteful tasks by replacing them with preventive quality-of-life crimes and performing arrests during sleeping hours. The input didn’t change while the EQ rose significantly.5
particularly effective at reducing waste when bidding for new business.
Orion, bidding for heavy-civil marine construction projects, submitted bids to projects worth $670 million, and it won projects worth a mere $76 million6—an EQ of 0.068. Years later, Orion focused its efforts and bid for projects worth only $429 million. Less bidding resulted in higher-quality bids. Orion’s wins rose to $117 million,7 and the EQ rose to 0.27.
A common mistake senior executives make is to measure output rather than throughput (effective output). EQ drives the organization toward process improvement. A better process reduces waste—which in turn improves the quality and value delivered.
References and Note
- John Mack, "Merck Rejiggers Its Marketing Mix," Pharma Marketing News, Vol. 6, No. 7, 2007, VirSci Corp., www.news.pharma-mkting.com/pmn67-article01.pdf.
- Businessdictionary.com defines "man month" as a unit of work representing the productive effort of one person during a four-week period. It is also called "labor month." See www.businessdictionary.com/definition/man-month.html.
- W. Chan Kim and Renee Mauborgne, "Tipping Point Leadership," Harvard Business Review, April, 2003.
- Orion Marine Group, "Orion Marine Group Inc. Reports Third Quarter 2011 Results," press release, Nov. 3, 2011, http://tinyurl.com/orion-pr-2011.
- Orion Marine Group, "Orion Marine Group Inc. Reports Second Quarter 2014 Results; Reports Record Backlog," press release, July 31, 2014, http://tinyurl.com/orion-pr-2014.
Boaz Ronen is a professor (emeritus) of technology management and value creation at Coller School of Management, Tel Aviv University in Israel. He has his doctorate in business administration from Tel Aviv University and co-authored Focused Operations Management: Achieving More With Existing Resources (Wiley, 2008).
Alex Coman is a professor of project management and value creation at the Academic College of Tel Aviv-Yaffo in Israel. Coman has his doctorate in IT from Claremont Graduate University in California.