LEADERSHIP & MANAGEMENT
Predicated On Proof
The importance of evidence-based management and its role in quality
Henry J. Lindborg
W. Edwards Deming famously asserted, “Hard work and best efforts, put forth without guidance of profound knowledge, may well be at the root of our ruination.” Finding that guidance has proved to be no small feat. Curious about how far we have come in applying contemporary knowledge to management issues, I interviewed Denise M. Rousseau, the H.J. Heinz II University professor of organizational behavior and public policy, and director of the project on evidence-based organizational practices at Carnegie Mellon University’s Heinz College of Information Systems and Public Policy, and the Tepper School of Business.
Lindborg: You have long advocated evidence-based management (EBM). What is it? What do you mean by “evidence”?
Rousseau: In 2005, I gave my Academy of Management Presidential Address on EBM and asked whether there is such a thing as EBM. The answer then was, “No, not yet.”
EBM means the conscientious use of multiple sources of evidence when making management decisions. It is a direct outgrowth of evidence-based practice (EBP) in medicine, public policy, criminology and other professional fields. When we say multiple sources, we mean much the same thing across all these fields: The use of scientific evidence where available, contextual information (organizational data in management and patient information in medicine), stakeholder concerns (the people affected by the decision and those who would implement it), along with professional judgment or expertise.
Evidence is information of multiple kinds, but it is only meaningful in relation to a claim, hypothesis or an assumption. If we say that financial incentives are needed to improve worker performance, for example, that claim would be judged in relation to several different kinds of evidence.
One kind of evidence is the body of available scientific evidence on incentives and performance, which is quite mixed and depends on the cognitive complexity of the work being performed—positive for simple jobs and negative for complex ones.
Another would be the organization’s own data on the performance effects of different pay systems, along with stakeholder reactions and considerations related to different kinds of incentives and the judgment of compensation experts and experienced managers.
How has EBM evolved?
EBM certainly has evolved since I first started advocating it as an approach to teaching business students and practicing management. My own initial focus on EBM addressed how to close the gap between management practice and organizational research.
In the past 15 years or so, I have come to realize two things: First, we don’t have relevant research available for many of the claims and assumptions managers make in their day-to-day decisions, so multiple sources of evidence beyond science are critical to daily management decisions.
Second, EBM is part and parcel of the broader initiative toward EBP across many fields and is not particularly distinct. Thus, all EBP fields must rely on multiple sources of evidence. It has been said that only about 25% of standard practice in medicine is based on scientific research. I cannot judge the accuracy of the claim, but I will bet that in management decisions, the percentage is closer to 10-15%.
A few years back, I gave a keynote address at a conference of the American Accounting Association and asked the audience what accounting practice might not be evidence-based. You know what audience members said? Budgeting. I think all fields are in the same boat. We have some research and some areas of good research relevant to practice, and we have many, many holes in our systematic knowledge. Multiple sources of evidence are essential to all forms of good EBP.
Is it related to quality management and improvement?
Yes. EBM is part and parcel of the larger industry focus on quality, which has caused organizations to pay greater attention to the decision-making process and doing work, and assess outcomes to institute process improvements.
How does EBM improve a manager’s or organization’s performance?
I think the key attribute that EBM brings to a manager’s performance is critical thinking and a basis for developing skills at reflecting, assessing and learning. Think of how seldom managers actually measure the outcomes of their decisions. Little wonder that older managers are not necessarily thought of as better managers. If you aren't learning every year, you are repeating the same mistakes over and over. I like to say there is a big difference between 20 years of experience and one year of experience repeated 20 times.
For the organization, EBM provides a basis for more systematic decision making—for establishing decision processes that help people learn what works, what doesn’t and how to make improvements over time. EBM begins with asking critical questions, obtaining multiple sources of evidence, applying them and assessing outcomes. If you assess the outcomes of your decisions, you have the basis for improving performance over time.
How about benchmarking?
Benchmarking provides perspective. One of the valuable offshoots of the total quality management movement is scanning for alternative ways of doing things, and benchmarking can provide insight into one’s own organization’s practices and metrics. However—a big however—“best practice” is an incredibly worrisome concept because it is applied in situations where people often have little quality evidence that those practices actually work. In other words, are the causes of the outcomes credited to those practices?
We like the term “promising practices” in EBP because it signals that there is some evidence from actual outcome assessments and controlled studies that suggests certain practices can cause desired results. So the short answer is best practices often have only weak evidence to support them, while promising practices have quality evidence behind them.
What are some barriers to EBM and consequences of not implementing it?
The barriers to EBM are managerial incompetence, poor organizational information systems and stakeholder complacency. EBM is a self-improving process, and to have the capacity for learning and improvement, individuals must have some basic abilities to learn and reflect, the motivation to do so and the opportunity to practice critical thinking, information gathering and assessment. One key feature we see over and over is the role of leadership support (or lack of it) in promoting managerial ability, motivation and opportunity to practice EBM.
How do you develop EBM skills? What are some resources?
For individuals, I strongly recommend taking advantage of the free online courses available at Carnegie Mellon,1 which take you through the building blocks of EBP. There also is a new book that can walk you through the process.2 And importantly, starting small with a specific decision to improve or a problem to solve. Remember: The perfect is the enemy of the good. Start with something doable that you care about.
What advice do you have for quality professionals on EBP?
All self-improving people benefit from conversation and reflection with others. It is really difficult to debias ourselves, but having thoughtful conversations with others about the decisions you are trying to make can help you identify the kinds of questions that would be useful to submit to an EBM approach. EBM is a social project as much as it is self-development.
- “Open Learning Initiative,” Carnegie Mellon University, www.oli.cmu.edu.
- Eric Barends and Denise M. Rousseau, Evidence-Based Management: How to Use Evidence to Make Better Organizational Decisions, Kogan Page, 2018.
Henry J. Lindborg is executive director and CEO of the National Institute for Quality Improvement in Fond du Lac, WI. He holds a doctorate from the University of Wisconsin-Madison and teaches in a leadership and quality graduate program. Lindborg is past chair of ASQ’s Education Division and of the Education and Training Board. He is a past chair and current member of the Institute of Electrical and Electronics Engineers Career Workforce Policy committee.