ASQ Quality Report - March 2008

The Quarterly Quality Report

March 2008

The ASQ Quarterly Quality Report provides a detailed look at a variety of quality-related topics and issues. The report is developed by the American Society for Quality in keeping with its role as the steward of the quality profession—to promote the use of quality as a global priority, an organizational imperative, and a personal ethic, and to promote quality concepts, technology, and tools to make the world a better place.

Download the Report (PDF, 67 KB)

Voting Systems—A Quality Update

With the 2008 general election in high gear in the United States, many are wondering if the voting machine errors that made national news and caused voter ire in the past several elections will make a reappearance in the months ahead.

In the aftermath of the 2000 presidential election in the United States and the election system quality problems it exposed, many changes have been made to improve the security, reliability, and efficient administration of voting. But issues still remain.

Among the most persistent of these issues are the continuing lack of confidence in the accuracy and security of electronic voting systems and lack of attention to the human element in the voting process.

What is the answer?  A few well-placed quality procedures may help to restore voter confidence. This can be accomplished by insisting on voter-verified paper records of all voting systems and by greater use of a combination of auditing of election results and pre-election systems auditing. Enhanced training of elections personnel would ensure higher levels of satisfaction among voters.

“It’s very important for citizens to have as much trust in their voting systems as they have in their medical care systems or air travel,” says quality expert Liz Keim, a past president of ASQ. “So why not leverage some of the same basic tools that bring reliability to critical functions like medical care and air transport to solve quality problems that continue to plague our elections,” she states.

Basic quality approaches that Keim refers to include things like establishing standardized procedures, implementing simple checklists based on those standardized procedures, adequate training of personnel focused on critical points in the system where problems are likely to occur, and adequate testing of proposed solutions.

Voting Glitches—Still More of the Same

The problems continuing to plague the current round of elections are mostly the same as those that surfaced in the 2004 national elections and the 2006 midterm elections. Although these problems may not be as severe as the 2000 disasters and not as disruptive as some observers have anticipated, nonetheless they are widespread, occurring in multiple states and voting precincts, despite improvements that have been put in place since 2000. And as before they involve both hardware and software, procedures and personnel.

In the current round of 2008 primaries, machine malfunctions constitute one of the largest classes of problems encountered. Reports of equipment problems have come in from numerous states.

  • In Cuyahoga County, Ohio, the central computer that tallies votes from memory cards taken from the county’s touch-screen voting machines locked up twice and stopped counting votes.
  • In Los Angeles County, it took poll workers several hours after the polls opened to get the machines running.
  • In Union County and four other counties in New Jersey, vote totals recorded on paper-tape backup did not match vote totals on the machines’ internal cartridges that print tallies.
  • Poll opening was delayed by a half hour at Potomac High School in Oxon Hill, Maryland, because poll workers did not have the computer code needed to start voting machines.
  • In Anne Arundel County, Maryland, three touch-screen machines at Cape St. Claire Elementary School shut down, one of which remained inoperative all day.
  • Presidential candidates did not appear on the ballots on some touch-screen machines in Miami Dade and Broward counties in Florida.

The glitches have not been confined to touch-screen machines. Malfunctions of optical scan systems have been reported in many locations, including the District of Columbia; Milwaukee; Bedford, Pa.; the New Hampshire towns of Hanover and Exeter and cities of Nashua and Manchester; Sarasota County, Fla.; Sacramento County, Calif.; Lake County, Ill; and several locations in Connecticut.

Bad planning in the administration of elections has been another recurring problem. Numerous voting precincts across the country failed to provide enough paper ballots to handle the large numbers of voters turning out for the 2008 primaries. And in some precincts there haven’t been enough voting machines to handle the volume of voters.

Increased complexity in voting technology is an added source of problems and voter dissatisfaction. And some problems originally attributed to machine malfunction have been found, on closer inspection, to be the result of programmer, operator, or voter error.

In 2004, the National Protection Coalition, an election watchdog group, reported receiving more than 600 calls from around the country on election day in November concerning problems with electronic voting machines.

During the midterm elections in November 2006, reports of malfunctioning electronic voting machines surfaced in at least 10 states. Long delays at Denver polling places were attributed to problems with new voting machines coupled with an unusually long statewide ballot. In Texas, a computer glitch was blamed for incorrectly showing a long-shot minor party candidate in the lead for the 28th district seat in the U.S. House of Representatives.

Frequent reports of inadequate training and human error, including poll worker error, also marred the 2006 elections.

About half of the voting precincts in Indianapolis and Marion County, Ind., had problems starting their voting machines. The problems were attributable to inadequate training of the poll workers. Incorrect programming of cards used to activate machines delayed opening of polls in 75 precincts in Delaware County, Ind., causing a judge to keep the polls open past normal closing time. In a September primary contest, election officials in Montgomery County, Md., neglected to give polling places the access cards required to make electronic voting machines work.

In an effort to solve issues surrounding voting machines, and to allay voter fears, Congress passed the Help America Vote Act (HAVA) in 2002 to assist in administration of federal elections. HAVA established the Election Assistance Commission (EAC) and provided funds to the states to upgrade their voting equipment. The EAC has been working with the National Institute of Standards and Technology to establish standards and guidelines for voting systems used by states and local units of government in the administration of federal elections.

The good news—HAVA’s goal of replacing error-prone punchcard voting systems—the machines that produced the infamous hanging-chad debacle in Florida during the 2000 presidential election—has been virtually accomplished. Usage of this type of equipment declined from 37.3% of voting jurisdictions in 1996 to 4% in 2004 and to 0.4% in 2006. Adoption of direct recording electronic voting equipment, on the other hand, rose from 7.7% of jurisdictions in 1996 to 9.3% in 2004 and 53.6% in 2006.

Voting—An Engineered System

There are those who are convinced that more can be done to improve the quality and safety of electronic voting processes and systems.

“We can bring quality tools and software engineering principles to bear to give voters much more confidence” in voting, says Taz Daughtrey, an instructor in the computer science department at James Madison University who is also a past chair of the ASQ Software Division and the founding editor of ASQ’s journal, Software Quality Professional.

To prove that point, Daughtrey is challenging students in his undergraduate software engineering course to design a better, more reliable electronic voting system.

Teams of students are working from the ground up, analyzing all aspects of the elections system, including legal requirements, and dissecting the engineering aspects to see if they can design better voting systems. They are designing computer/voter interfaces, validation and testing routines, and security provisions that ensure that votes are captured and recorded accurately and confidentially. To test their designs, they use data from student elections on the campus.  

Daughtrey believes the students’ work can shed light on ways to take advantage of the best new technology to improve voting and enhance voter confidence. He believes electronic systems hold great promise for elections. But, he acknowledges, “The complexity of electronic voting is a two-edged sword. It has great potential for error-proofing, but that same complexity can introduce reliability concerns.” 

Daughtrey also is a proponent of using basic quality management principles to make good, data-driven decisions regarding the purchase of voting systems and the certification or decertification of these systems.

Lingering Skepticism

Yet, despite improvements in many areas in recent years, voters remain highly skeptical.

“The single biggest question in voters’ minds is, ‘Can we trust the numbers that come out of our voting systems?’” says Keim.

The current focus of much of that skepticism is directed toward the direct recording electronic (DRE) voting machines. In the decade between 1996 and 2006, election administrators eagerly acquired these machines, attracted by their potential benefits: rapid processing of large volumes of votes, multilingual capabilities, accessibility to voters with disabilities, the immediate feedback that they can provide to help voters cast the votes that they intend, and above all the promise of eliminating ambiguity over voter intent that plagued punch-card systems in 2000.

Federal funding under HAVA set off a rush by the states to acquire these electronic voting systems. A 2004 report by ASQ expressed concerns, suggesting the states were hasty in making these acquisitions: 

“…states and localities are making purchases of new voting machinery without the benefit of promised guidelines from the EAC. A large part of the hundreds of millions of dollars of HAVA funds already disbursed by the General Services Administration to the states has been designated for the purchase of equipment to replace punchcard and lever voting machines, even though the EAC, due to lack of funding, has yet to issue equipment guidelines or conduct programs for testing election equipment and technology.”

Persistent criticisms from voter advocacy groups and computer scientists have prompted intense scrutiny of electronic voting mechanisms. Two of the main concerns are software susceptible to attack and malicious manipulation and the lack of a voter-verifiable paper trail that can be used in the event of a recount and as a means to verify to voters that their votes were recorded as intended.

Quality Improvements Get the Vote

Driven by the public’s mounting lack of confidence in security and reliability, some states have decertified their electronic voting equipment, and there is a movement under way to go back to paper voting systems.

Claiming that California’s electronic voting machines were vulnerable to errors and sabotage, Secretary of State Debra Bowen in August 2007 decertified most of the state’s electronic voting machines, which were used in 39 counties. The state ordered counties that had been using machines without paper trails to switch to paper ballots.

In December 2007, Colorado Secretary of State Mike Coffman decertified the electronic voting machines used in Denver, Arapahoe, Jefferson, Pueblo, Mesa, and Elbert counties, once again citing security or accuracy concerns.

Florida ended its brief, six-year, $30 million experiment with touch-screen voting when it elected to spend close to $30 million more to replace it with optical-scan technology. The final straw for the touch-screen machines (which had replaced Florida’s infamous punch-card machines) came in the 2006 midterm election, when voting officials could not explain why 18,000 fewer votes were cast for the 13th Congressional district race than in other races in Sarasota County. These 18,000 “no-votes” seriously clouded the outcome, which was ultimately settled only after action by the U.S. House of Representatives.

Ohio, which is considering a ban on DRE machines, gave voters in counties that use electronic voting machines the option to use paper ballots in the March 2008 primaries. Large numbers of voters chose that option, causing ballot shortages and polling-place delays.

New Mexico abandoned DREs in 2006 with legislation that installed a uniform statewide system of paper ballots and optical scanning equipment. Maryland recently announced its intent to do the same.

New Jersey issued a statewide directive in 2007 to retrofit touch-screen machines with printers that would give voters a paper record of their vote, but voting activists are pushing the state to scrap the machines altogether and replace them with paper ballots and optical scanners.

These problems are not limited to direct-recording electronic voting machines. The studies that led to decertification of DRE machines in California and Colorado and a statewide review in Ohio found security vulnerabilities in optical scan machines as well.

Human Factors Get Inadequate Attention

On voting’s frontlines—in more than 100,000 polling places nationwide—the mostly volunteer force of poll workers is stretched very thin. Understaffed polling places have been the cause of voting delays at some polls and can be a main source of voter dissatisfaction. Administrators are finding it increasingly difficult to recruit and retain the estimated 1.3 million volunteer poll workers needed for a nationwide election. To make matters worse, these workers often receive inadequate training. And as voting technology becomes more complex, many of these volunteers, whose average age is over 70, lack the skillsets needed to understand voting technology, troubleshoot problems, and assist voters.

Technical expertise is also stretched thin among the elected officials and paid staff that administer elections across the country. They rely on the manufacturers of voting machines for this expertise, which worries some voting advocates.

“The people running our polling places are frequently the overlooked link in the system,” states Keim. “You don’t send out a waiter to serve diners without adequate training, and you don’t send a hospital lab tech to draw blood if they’ve never done it before, so why do we seem so unconcerned about having the right people with the right training to run something as important as our elections?”

Keim advocates strict adherence to polling-place regimens that include standardized and documented procedures and the implementation of checklists similar to the checklists that pilots use to prepare for flight.

In 2004 ASQ lamented that states and counties were rushing into what it thought were premature buying decisions, spending money on electronic systems before standards were established for such things as paper backup (and for all the other things that could be used as benchmarks for what constitutes quality in an election process) and paying inadequate attention to the human factors that make elections run smoothly. That situation has not changed much.

Audits Complement Paper Voting Trails

Although there has not been a documented case of a malicious attack on electronic voting systems, computer security experts say the possibility exists, and these systems remain vulnerable to hacking.

The media and various voting advocacy groups have placed a great deal of emphasis on the need for a voter-verifiable paper trail to preclude election fraud and to provide voters with confidence that their votes have been recorded and counted correctly. Nearly half the states still do not require a paper record of voting. Legislation was introduced in the U.S. House of Representatives in January 2008 to reimburse jurisdictions for the costs of obtaining paper-ballot voting verification systems in time for the 2008 federal elections in November.

But paper trails from voting machines in themselves are not the last word in making voting systems more secure. Even with a paper printout, an electronic voting system remains vulnerable to software glitches and programming errors and to the deliberate introduction of software designed to disrupt or change the reported results. Machines that print the paper can fail and are vulnerable to tampering, and the paper trail may be useless if poll workers do not handle it properly.

Audits have been suggested as a way to overcome this shortcoming—not only with DRE voting systems but also with any other form of vote recording and counting. There are two ways that auditing, a common quality assurance practice, could aid the election process.

The first is the use of routine post-election audits of election results to detect anomalies that indicate either fraud or systemic errors in the voting equipment. In this type of audit, paper records from randomly selected machines and randomly selected precincts are compared to the voting system’s electronic records. There are 30 states that mandate a voter-verifiable paper voting record for electronic voting systems, but of these only 17 also mandate audits. Audits complement the paper record and make it truly useful.

“A properly designed sampling scheme will ensure that the post-election audit does two things to increase the public’s confidence: It validates the balloting process and uncovers anomalies in the voting,” according to Mary Strutzenberg, chair of the ASQ Tellers Committee that has oversight for the organization’s elections.

The other auditing application consists of quality audits of election systems (processes, procedures, and equipment) against established standards. This type of auditing, which would take place in the months prior to an election, is similar to the quality management systems audits carried out under the ISO 9000 international standards and the environmental management system standards of ISO 14000. The effect would be a better understanding of the voting processes and actionable knowledge for continuous system improvement.

“The value of doing quality management systems auditing lies in validating and documenting where the election system works well—where it’s functioning as you say it’s supposed to function—and where improvement is needed,” Strutzenberg adds. She points out that from there, the election administrators can then employ repeating cycles of continuous improvement to address non-value-adding steps in the process identified by the audits. 

The EAC’s voluntary voting system guidelines form a basis on which to conduct quality system audits of election management systems. A new draft version of these guidelines is currently being circulated for public comment prior to adoption. Marrying this body of knowledge about voting systems best practices with the already established discipline of quality systems auditing (á la ISO 9000) can form a solid foundation for establishing credibility and public confidence in our nation’s election systems. Publishing the results of these audits would lend greater transparency to the voting process.

Auditing is just one of many ways that quality methods can help improve the voting process. Quality management tools and statistical thinking can be used to improve systems and processes in order to reduce or eliminate opportunities for voting and counting errors. Traceability guidelines drawn from the ISO 9001 quality management standard are useful for election officials in dealing with issues such as making sure the right people are voting and voting only once and in establishing confidence that votes are counted correctly. Data-based decision making, a hallmark of quality management, can go a long way toward eliminating suspicions and ensure that decisions regarding voting procedures and systems are made on a rational basis, not based on fear or influenced by partisan political considerations.

Taking as much mystery as possible out of the voting process—through audits, better trained personnel, and systems that are lean and efficient and voter instructions that are clear and unambiguous—would provide greater transparency and help to restore the public’s confidence while increasing satisfaction with elections. These things can be done while still maintaining voter confidentiality.

The close scrutiny that election systems are receiving today from public officials, computer scientists, quality engineers, and voting advocates should be viewed as a good thing. We may one day look back on today’s turmoil and uncertainty in voting processes as a grand and mostly successful experiment in democracy and customer satisfaction.

Copyright © 2008 American Society for Quality

About the American Society for Quality
The American Society for Quality is the world’s leading authority on quality. With more than 93,000 individual and organizational members, the professional association advances learning, quality improvement, and knowledge exchange to improve business results and to create better workplaces and communities worldwide. As champion of the quality movement, ASQ offers technologies, concepts, tools, and training to quality professionals, quality practitioners, and everyday consumers, encouraging all to Make Good Great®. ASQ has been the sole administrator of the prestigious Malcolm Baldrige National Quality Award since 1991. Headquartered in Milwaukee, WI, the 60-year-old organization is a founding partner of the American Customer Satisfaction Index (ACSI), a prominent quarterly economic indicator.


Featured advertisers

ASQ is a global community of people passionate about quality, who use the tools, their ideas and expertise to make our world work better. ASQ: The Global Voice of Quality.