Poor Poll Position
Dealing with the growing number of unreliable, inaccurate political surveys
One political website called it "one of the greatest upsets in modern political history."
Bernie Sanders, the Democratic presidential nominee underdog, won Michigan’s primary election over front-runner Hillary Clinton in early March, 50% to 48%. No poll taken during the previous four weeks had Clinton leading by less than five percentage points.1 The poll average, in fact, predicted Clinton would sack Sanders by 21 points.
There have been other polling pratfalls in recent elections, and it’s not just happening in American politics.
In 2012, Mitt Romney, the Republicans’ presidential nominee, had his own campaign’s polls, among others, predicting he would just barely beat President Barack Obama.
That, of course, didn’t happen, and Obama cruised to a second term by five points.
Pollsters misread the 2014 referendum on Scottish independence, which was resoundingly rejected, and couldn’t predict decisive victories last year for Britain’s Conservatives and Israeli Prime Minister Benjamin Netanyahu’s Likud Party.2
"It’s a worldwide problem," said Roger Tourangeau, a survey methodologist and vice president of Westat, a research firm in Rockville, MD. "It’s a different world now."3
Others agree. "The science of public surveying is in something of a crisis right now," said Geoffrey Skelley, a political analyst at the University of Virginia’s Center for Politics.4
Two big challenges seem to be driving the increasing unreliability of election and other polling: the increasing number of cellphones and fewer people willing to participate in surveys.5
Ten years ago, about 6% of Americans relied primarily on cellphones; by 2014 that percentage had increased to 60%.6 Cellphones present unique challenges: People often transfer their cell-phone numbers when they move from area code to area code, which can make it difficult for pollsters to target specific regions.
To connect with those on cellphones, pollsters must manually dial these phone numbers, according to federal regulations. This adds to the cost of conducting surveys because landline phone numbers can be dialed via computer programs.
If pollsters lean too heavily on surveys of those they reach on landline phones, those responses can skew the results. In general, older generations keep their landline phones, and many in that generation are conservative. This can sometimes misrepresent the actual members of the population.
In addition, people are using voicemail and caller ID to avoid answering phone calls from unknown numbers, which affects response rates. For more than a century, people answered their landline phones faithfully, but now they’re much less likely.7
"In the late 1970s, we considered an 80% response rate acceptable," said Cliff Zukin, former president of the American Association for Public Opinion Research. By 2014, the response rate had fallen to 8%8
"Telemarketing poisoned the well," said Charles Franklin, director of Marquette University Law School’s poll,9 which has successfully predicted every major election result since the Milwaukee school began its polling in 2012.
Many people simply don’t want to spend the time answering the pollsters’ series of questions, which can sometimes take up to 30 minutes to complete. This avoidance and lack of cooperation from poll-takers perhaps reflects in part the increased disdain for politics and politicians, and the decline in confidence in America’s political institutions.10
Online polls are becoming more prevalent, but because they’re voluntary, people who choose to answer them may be unusually ideological—again, perhaps skewing results.
Together, these challenges have made conducting high-quality research much more expensive, so there is less of it. Less scientifically based and less well-tested techniques have filled this information void. In addition, how to identify "likely voters"—always an election polling problem—has become more difficult.11
"The problem is simple but daunting," said polling aggregator Nate Silver, of FiveThirtyEight.com, a website focused on opinion poll analysis, politics, economics and sports blogging. "The foundation of opinion research has historically been the ability to draw a random sample of the population. That’s become much harder to do."12
What are pollsters doing to bolster sample sizes and gather better data to make more accurate predictions?
Some pollsters are working on new data-savvy methods to increase accuracy. One organization, for example, has developed the Net Presidential Score, which builds from the Net Promoter System some organizations use to measure customer and brand sentiment.13
Other pollsters are weighting polls according to historical accuracy and by factoring in demographics and other data to create election models.14
Another organization, Reconnect Research, developed technology that accepts inbound calls and turns them into research and polling opportunities. If someone misdials a call to his or her bank or credit card company, for example, Reconnect Research will direct that call to a researcher.15
The organization says thousands of people per day—millions per month—misdial by one or more digits. Reconnect Research showed 14.5% of people stayed on the phone long enough to complete an entire three-minute survey.16
Earlier this year, the Pew Research Center announced it will increase the percentage of respondents interviewed on cellphones from 65% to 75% in most of its 2016 telephone surveys.
The center said this effort is meant "to ensure our survey samples properly represent the now nearly half (47%) of U.S. adults whose only phone is a cellphone."17
Responders should always consider the source before trusting a poll. Polling organizations that are members of American Association for Public Opinion Research’s Transparency Initiative, for instance, provide a standard level of technical disclosure about their methods for anyone who wants to know more specifics.18
Election surveys have become a part of the political process, but despite these improvement efforts, some will continue to fall short.
The best piece of advice to voters, several experts offer, is to avoid using polls solely to pick among candidates.
—compiled by Mark Edmund, associate editor
- Harry Enten, "What the Stunning Bernie Sanders Win in Michigan Means," FiveThirtyEight.com, March 9, 2016, http://tinyurl.com/bernie-win-538.
- The Week, "The Problem With Polls," The Week.com, April 10, 2016, http://tinyurl.com/the-week-polling-probs.
- Joseph P. Williams, "The Problem With Polls," U.S. News and World Report, Sept. 28, 2015, http://tinyurl.com/us-news-poll-probs.
- Cliff Zukin, "What’s the Matter With Polling?" New York Times, June 20, 2015, http://tinyurl.com/nytimes-poll-probs.
- The Week, "The Problem With Polls," see reference 2.
- Euel Elliott, "Bernie Sanders is Proof That Voters Shouldn’t Trust Polls," Fortune, March 30, 2016, http://tinyurl.com/fortune-bernie-polls.
- Zukin, "What’s the Matter With Polling?" see reference 5.
- Jeff Haden, "Here’s How Net Promoter Might Just Predict the 2016 Presidential Election," Inc., April 1, 2016, http://tinyurl.com/inc-net-promoter.
- The Week, "The Problem With Polls," see reference 2.
- Eric Chemi, "Call a Wrong Number? You Might Be Asked to Take a Survey," CNBC, Nov. 9, 2015, http://tinyurl.com/cnbc-wrong-number.
- Kylie McGeeney, "Pew Research Center Will Call 75% Cellphones for Surveys in 2016," Pew Research Center, Jan. 2, 2016, http://tinyurl.com/pew-center-cell.
- Rob Santos, "Why the Polls Get It Wrong," Los Angeles Times, March 27, 2016, http://tinyurl.com/latimes-polling-probs.
Celebrating IAQ’s Golden Anniversary
This year, the International Academy for Quality (IAQ) celebrates its 50th anniversary, an appropriate time to reflect on how international collaboration around quality has evolved in the decades following World War II.
Three organizations first took the lead in the modern quality movement:
- ASQ was founded in 1946 to preserve the lessons learned after quality methods were integrated into the mass production systems to support the global efforts of the Allies.
- The Union of Japanese Scientists and Engineers (JUSE) was established in 1946 to integrate quality into the reconstruction of Japan.
- The European Organization for Quality (EOQ) was established in 1956 to disseminate quality lessons that were an adjunct to the Marshall Plan for rebuilding Europe.
Two decades into postwar reconstruction efforts, a growing awareness of the need for more formal international coordination and collaboration among these organizations emerged. The initial idea of forming an international association originated from the president of the European Organization for Quality (EOQ), Jan van Ettinger.
His idea was assimilated by ASQ Past-Presidents Armand V. Feigenbaum and E. Jack Lancaster. In his ASQ presidential acceptance speech, Lancaster sparked general interest within the society’s leadership in developing such an organization as a global quality alliance, and he coordinated a meeting of the leaders of these three organizations to determine the feasibility of founding such a quality group.
Subsequently, in 1966, a six-person board was established to form what was first called the International Quality Association (now the International Academy for Quality, or IAQ), whose invited membership would equally represent all of the three major quality organizations. The board’s purpose was to design and develop a new global organization to facilitate an international exchange of information about quality to promote it throughout all nations.
JUSE, EOQ and ASQ each nominated two individuals to form this study team, and the target was to complete the formal organization by 1971. JUSE nominated Kaoru Ishikawa and Masao Kogure (who was later replaced by Tetsuichi Asaka). Two board members of EOQ were named to the new organization’s board: Frank Nixon (from the United Kingdom, also supported by the British Productivity Council) and George Borel (France). From ASQ, the participants were Feigenbaum and Lancaster.
The original purpose of this group was threefold:
- Coordinate attention to technical problems in quality.
- Ensure the broad dissemination of results of such work to the greatest benefit of those concerned.
- Promote recognition of the role and importance of quality in other disciplines as a concept and as a decisive factor in stimulating success in all disciplines.
Another force behind IAQ’s establishment was Walter E. Masing, who stayed in the background because he was in high demand as the principal originator of the Deutsche Gesellshaft für Qualität (DGQ, or German Quality Association) and the EOQ. Because of their important contributions and early roles in the founding of IAQ, Feigenbaum, Ishikawa and Masing are honored by IAQ with a Founders Medal, an award that recognizes a member’s exceptional service to the academy.
The first Americans initially elected as IAQ academicians were Leon Bass, Charles A. Bicking, C. Eugene Fisher, Julius Y. McClure and Thomas C. McDermott, in addition to Feigenbaum and Lancaster. All had served previously as ASQ presidents. Lancaster was elected the first IAQ president and was succeeded by Feigenbaum and Ishikawa for the following two three-year terms.
Since its founding, about 250 individuals have been elected as academicians, and their membership has been equally distributed across the Americas, Europe and Asia, and represents more than 60 nations.
In addition, the symbiotic relationship between IAQ and ASQ is observed in the fact that academicians have included 28% of the ASQ past-presidents, 46% of its Distinguished Service Medalists and 42% of its honorary members.
Perhaps even more noteworthy is that of the recipients of the ASQ Lancaster Medal—which recognizes international service and has a strong correlation to the criteria for election as an academician—85% have been IAQ members.
Today, ASQ hosts the IAQ secretariat and provides its administrative support through a contractual arrangement.
IAQ members were instrumental in founding all of the leading global professional quality organizations: JUSE, EOQ, Asia Pacific Quality Organization, Asian Network for Quality and Middle East Quality Association. They played an important role in founding more than 24 national quality organizations, and the academy continues outreach to promote quality in developing nations.
In 1983, a team of academicians—with guidance and translation support of its first Chinese member, Yuanzhang Liu—made a quality tour of Chinese cities and introduced government and business leaders of mainland China to the concept of total quality management.
Gregory H. Watson is a past president of IAQ and ASQ. He has been elected an honorary member of IAQ and a fellow of ASQ, and has received the IAQ’s Founders Medal, the Deming Medal from JUSE, the Borel Medal from EOQ and ASQ’s Distinguished Service Medal.
A Celebration of 70 Years
Throughout 2016, ASQ will be marking its 70th anniversary. As part of the celebration, ASQ is inviting members to submit their earliest or most memorable experience as a member of ASQ’s global quality community.
For instance: What interaction or event stands out most to you in remembering your years as a member of ASQ? When did you join, or better yet, why? Did a friend or colleague suggest it? What are some of your fondest memories of the organization and connections throughout the years?
Visit asq.org/70-birthday-wishes to submit your story in 300 words or less. We’ll compile an online collection of anecdotes you’ll see throughout 2016 and beyond. Photos are also welcome so we can post one alongside each special memory.
In addition, feel free to post your favorite ASQ memory or a wish to ASQ on social media. Be sure to use #ASQ70 when you’re sharing your thoughts and memories.
Virgin America Ranks No. 1 in Airline Quality Ratings
Virgin America is tops in airline quality among the biggest U.S. airlines, according to the 26th annual national Airline Quality Rating report, released last month.
Although the No. 1 airline’s overall score declined slightly from 2014, Virgin America still managed to retain its top ranking for 2015. JetBlue finished second, followed by No. 3-ranked Delta Air Lines.
Despite overall industry improvement in three of the four categories tracked, customer complaints rose by 38% from 2014 to 2015.
"This is the worst complaints have been for 15 years, as a rate. The last time it got this high was in 2001," said report co-author Dean Headley, a researcher at Wichita State University’s business school, which produces the report.
Passengers may be annoyed by ticket change fees of up to $200 and a lack of improvement in airline amenities, despite the fact that four of the nation’s top airlines saved more than $11 billion in fuel costs in the first nine months of 2015, Headley said.
Rankings of the 13 major U.S. airlines were based on data reported to the U.S. Department of Transportation for 2015, focusing on four criteria: on-time arrivals, denied boardings, mishandled bags and customer complaints. The final overall ranking is calculated as a weighted average of all four scores.
To see the full report, visit www.airlinequalityrating.com.
NEW CASE STUDIES ASQ’s Knowledge Center released two new case studies. The first is about a how a hospital’s nursing department used the define, measure, analyze, improve and control method, value stream mapping and other quality tools to improve efficiency of its staff’s shift-change assignments. For more information, visit http://tinyurl.com/case-study-nurse-dept. The second explains how combining TRIZ and lean Six Sigma through a structured approach can help reduce the effort and duration of lean Six Sigma projects by nearly 10 times. Visit http://tinyurl.com/case-study-triz to download that case study.
Call for Nominations
QP wants to spotlight the best and brightest young quality professionals making a difference today.
If there’s someone who is a young, passionate and an up-and-coming quality leader—pushing boundaries and making a difference in quality—let us know. Nominate your boss, an employee, a colleague—or yourself.
Submit your nomination at http://asq.org/qualityprogress/quality-professional/form.html before June 15.
In the nomination form, be sure to present the most significant accomplishments and attributes of your nominee, along with other observations. Nominees must be 40 years of age or younger and display professionalism, potential and a passion for quality.
Editors are planning a feature for the November edition of QP. Five years ago, QP profiled young quality professionals in the feature article "40 New Voices of Quality".
ASQ WORLD CONFERENCE
ASQ to Honor 19 Thought Leaders
ASQ will present its Distinguished Service Medals and other society medals and awards at its World Conference on Quality and Improvement May 15 in Milwaukee.
The 19 medal and award recipients are:
Distinguished Service Medal: Paul E. Borawski, Vernal Management Consultants, Milwaukee; Harry S. Hertz, National Institute of Standards and Technology, Gaithersburg, MD; John J. Knappenberger, ANSI-ASQ National Accreditation Board, Milwaukee; Ronald D. Snee, Snee Associates LLC, Newark, DE; and Ken Stephens, retired, Sun City Center, FL.
Crosby Medal: T.M. Kubiak, Performance Improvement Solutions, Fort Mohave, AZ.
Deming Medal: Rocco J. Perla, Health Leads, Boston.
Edwards Medal: Krishan Kumar, Maruti Suzuki India Ltd., Gurgaon, India.
Feigenbaum Medal: Tina Kanti Agustiady, Agustiady Lean Six Sigma, Tampa, FL.
Freund-Marquardt Medal: Joe Cascio, George Washington University, Washington, D.C.
Hutchens Medal: Vijay Chheda, Bidada Sarvodaya Trust, Los Angeles.
Juran Medal: Akio Toyoda, Toyota Motor Corp., Japan.
Lancaster Medal: Tang Xiaofen, Shanghai Association for Quality, Shanghai, People’s Republic of China.
Shainin Medal: Jd Marhevko, Accuride Corp., Evansville, IN.
Shewhart Medal: Connie Margaret Borror, Arizona State University West, Glendale.
Brumbaugh Award: Necip Doganaksoy, General Electric, Schenectady, NY; and Hans van Meer, Applied Materials, Gloucester, MA.
Gryna Award: Dana M. Johnson, school of business and economics, Michigan Technological University, Houghton; and Roberta S. Russell, Virginia Tech, Blacksburg.
For more details on the honorees, visit http://tinyurl.com/asq-award-winners-2016.
GLOBAL STATE OF QUALITY RESEARCH
Big Data Insights Offered In Upcoming ASQ Report
Big data, when used as part of quality and continuous improvement efforts, can improve organizational performance through forecasting, predictive modeling and developing unexpected insights, according to a new report released as part of the ASQ Global State of Quality 2 Research.
Despite the benefits of using big data however, only 20% of respondents said their organization is leveraging big date for a competitive advantage. Additionally, only 37% "somewhat agree" or "highly agree" that big data is being used to identify and improve the understanding of customer needs.
For the report, available at globalstateofquality.org, ASQ and its research partner, American Productivity and Quality Center (APQC), interviewed Elmer Corbin, director and project executive, client success at IBM Watson and Watson Health, and Silvia Veronese, director big data solutions, Hewlett Packard Enterprises Co.
Corbin and Veronese address how their organizations use big data, offer advice for organizations considering using big data and link big data to predictive analytics to anticipate quality issues.
The report was released as part of the ASQ Global State of Quality 2 Research, which examines the state of quality and continuous improvement worldwide, providing organizations with insights into gaps and opportunities. The latest research expands on the inaugural 2013 research, which provided the first-ever view of quality and continuous improvement on a global scale.
The ASQ Global State of Quality 2 Research: Discoveries 2016 report, which will include quantitative and qualitative data based on the survey results, will be revealed at the ASQ World Conference on Quality and Improvement, May 14-16 in Milwaukee. The report will be available online at globalstateofquality.org.
24 Members Named to 2016 Class of ASQ Fellows
The ASQ Board of Directors named 24 fellows, who join ASQ’s nearly 650 previous active fellows. The 2016 class includes:
- John B. Bowles, University of South Carolina, Columbia.
- Kristin L. Case, CaseConsults, Tulsa, OK.
- John J. Casey, Honeywell, Tempe, AZ.
- Rachel M. Delisle, Karl Storz Endovision Inc., Charlton, MA.
- Roberto Guzman, CR Bard, Morrisville, NC.
- Glenn I. Hamamura, Systems Excellence LLC, Honolulu.
- Christianna P. Hayes, Impact Performance Solutions, Morgan, UT.
- Eduardo Heidelberg, QRM Consultants LLC, Parlin, NJ.
- Michael Alan Hirt, ADAC Automotive, Grand Rapids, MI.
- Rajesh Jugulum, Cigna, Bloomfield, CT.
- Lucita D. Kahn, retired, U.S. Army, Research, Development and Engineering Command, Armament Research, Development and Engineering Center, Picatinny, NJ.
- Anantha Kollengode, Mayo Clinic, Rochester, MN.
- T.M. Kubiak, Performance Improvement Solutions, Fort Mohave, AZ.
- Kim Hung Lotto Lai, Hong Kong Science and Technology Parks Corp., Hong Kong.
- Austin S. Lin, Google, Mountain View, CA.
- Kathleen M. Lyall, Whirlpool Corp., Benton Harbor, MI.
- Chhanubhai Govind Mistry, retired, Ontario, CA.
- Tracy L. Owens, LexisNexis, Dayton, OH.
- Steven J. Schuelka, SJS Consulting, Crown Point, IN.
- Chad A. Smith, Continuous Improvement Solutions LLC, Bella Vista, AZ.
- Robert William Stoddard II, Software Engineering Institute, Pittsburgh.
- Zeljko M. Torbica, West Virginia University Institute of Technology, Montgomery.
- Wayne R. Wesley, Human Employment and Resource Training Agency, Kingston, Jamaica.
- David Zubrow, Software Engineering Institute, Pittsburgh.
Who’s Who in Q
NAME: Michael Bump.
RESIDENCE: Cocoa, FL.
EDUCATION: Diploma from Titusville High School in Florida. ASQ-certified quality auditor and inspector, certified aerospace technician from Space TEC/Brevard Community College in Cocoa, FL.
CURRENT JOB: Quality inspector/group leader at Harris Corp.
INTRODUCTION TO QUALITY: Quality inspector while stationed onboard the U.S.S. Emory S. Land during his service in the U.S. Navy.
PREVIOUS JOB: During his time in the U.S. Navy, he worked on Submarine Safety (SUBSAFE) and reactor plant systems. While at Boeing at the Kennedy Space Center, he worked on manned (Space Shuttle) and unmanned (Delta 2 and Delta 4 rockets) space programs. During his time with SpaceHab Inc., he worked on the Space Shuttle program processing the scientific and logistics modules.
ASQ ACTIVITIES: Assistant proctor for ASQ exams.
OTHER ACTIVITIES/ACHIEVEMENTS: Becoming a group leader for the quality inspectors in proprietary systems at Harris Corp.
RECENT HONORS: The ASQ Inspection Division’s 2016 Chuck Carter International Inspector of the Year Award.
PERSONAL: Wife, Michele; stepson, Angelo; and daughter, Victoria.
QUALITY QUOTE: An understanding of the importance of quality is just one disaster away.