Six Sigma for Internet Application Development - ASQ

Six Sigma for Internet Application Development


Download the Article (PDF, 86 KB)

H. James Harrington, Systemcorp, and Tom McNellis, Temple University School of Business

Six Sigma is a quality concept made popular by Motorola’s quality improvement initiative during the 1988-1989 timeframe and, since that time, has been applied successfully in many service-based organizations. When Web development and Six Sigma are used together, the result is Internet excellence.

Testing has always been an important and time-consuming process in any Internet development initiative. Since the customer is next in line, there is no room for error. Rapid application development (RAD) has allowed the Six Sigma Black Belt to deliver Internet functionality faster than ever before. However, one bottleneck in the information technology development life cycle remains: "Getting the deliverable right the first time around." Error detection, message description, frame isolation, and error rectification are extremely difficult, if not impossible, to catch once an Internet initiative is in production. When customers use that application, they do not care how the problem got there, they just know that the Web site "doesn’t work." The last thing a company wants in this seamless and competitive environment is to tarnish its image and efforts with inferior Web quality.

Key words: customer requirements, dashboard, development model, organizational change, process improvement



The Internet is transforming the world of business. With that change, software professionals are taking their business organizations to new transactional levels. Change is necessary for progress, but the Internet is demanding development at such an accelerated pace that the definition of Internet quality remains somewhat undefined. Speed to market, customer convenience, and competitive advantage make the Internet a valuable tool. The pace of change, however, is placing the development effort into a constant state of flux. In fact, research indicates that many Internet initiatives fail to add value to the customer, end in rework, and exceed budgets by significant amounts (Harrington and Mc Nellis 2001).

What can a company do to change the Internet failure scenario? One alternative is to improve continuously upon deliverables through Six Sigma process improvement and the “best practice” sharing that is associated with Six Sigma excellence. There is a tremendous amount of cost with rework, not to mention the time, resources, and bad customer experience associated with “unquality.” Six Sigma is a unit of measure that improves process capability by 20,000 times that of a three sigma effort and is defined statistically as 3.4 defects per million opportunities. Some successful Web sites realize a million hits in one day. Since the customer is a partner in this seamless environment, there is not much room for error. Customers are demanding a higher assurance of quality in both product and service. Planning Six Sigma quality into an information technology project adds the benefit of a shared vision toward excellence and smooth transactional service.

This article will address how an organization can combine Six Sigma and the Internet to reduce costs, improve the delivery success rate, and increase service quality. The model that was developed to map the Six Sigma quality process against the software development life cycle will be introduced. A case study in which the model was applied will be discussed; use of the model enabled the information technology (IT) organization to infuse Six Sigma into the Internet development effort and gave the rest of the company an understanding of the necessary steps for that attempt to be successful. Total commitment is necessary to carry this process out. This commitment, however, allows organizations as a whole to “get it right the first time” while satisfying client and business requirements.


Technological change, over a relatively short period of time, has brought Internet connectivity to a central role in current business models. Internet value may be evaluated as the sum of business process integration plus the satisfaction level attained from customer service. In many cases, the output of that equation becomes greater than the sum of it parts. Organizational processes that are integrated into one seamless set of transactions, from the supplier side through the customer experience, eliminate duplication and dramatically drive down factors of costs. That cost reduction is lean-sizing organizations to the point where the culture of a business is changing based on the way transactions are Web enabled. But, that is only the beginning. As time advances, technological creativity and the boundaryless search capability of the Internet will provide suppliers with unlimited enhancers to increase the level of customer satisfaction. Global access will provide the ability to search every corner of the world to find the right commodity, at the right price, at the right quality, and with product changes that align with the real-time “voice of the customer.”

Competition will be fierce, and the methods used by the IT organization will be crucial for survival. This article provides a business model and case study that supports the belief that development teams, under the best circumstances, need a structured and disciplined environment in which to cooperate toward Internet success. Six Sigma quality can push the bar of operational excellence to a level of excellence never before imagined.


Internet-driven change is reshaping the paradigm of business transaction management. In an effort to adapt to that change, the IT organization is modifying its structure and approach to better align with client support. When delivering applications to the Internet (where the customer is approaching a seamless partnership with the business), client and customer expectations must be closely allied for maximum benefit.

This case study will demonstrate the techniques used by one company to combine Six Sigma quality with the Internet development effort. As the case will demonstrate, not only did the client benefit, but the IT organization improved business processes by making a performance commitment to Six Sigma quality.

The company began in 1993 as a supplier of medical equipment and realized substantial growth in a very short period of time. In the company’s vertical organizational model, a focus on the “inside-out” approach was driving most of the development initiatives. After reviewing a benchmarking analysis, the CEO realized that some of the competition began building a business-to-business Web infrastructure, hoping to solidify long-term commitments with premiere customers and the marketplace. The CEO therefore asked the chief information officer (CIO) to get involved. The company soon adopted a “launch and learn” Internet development strategy.

That method allowed the company to save a significant amount of time in the race to be “one of the first” in that market segment with an Internet site. Today, utilizing a business-to-business infrastructure, the new emphasis is on customer service and operational integration. The company had recently adopted the Six Sigma quality methodology. The chief operating officer (COO), after reviewing the monthly customer support desk complaint log, determined that too much time was spent taking development deliverables out of production. The COO asked the quality audit team to assess the issues and report on the outcome. Four problem areas immediately surfaced because of a “business as usual” attitude that did not recognize the contribution that Six Sigma could make when applied to the Internet. The four problem areas were defined as follows.

Problem Area One: Production Issue

The development process was broken. Although deliverables correlated closely with scope definition, the error rate was unacceptably high. This situation was somewhat confusing, as the defects should have been detected during the system level testing initiative. The quality team began the investigation by plotting a stability factor for the combined Web-site reliability against the last four quarters of time. The sigma measurement substantiated what everyone already knew to be true. The Web-site rework factor was excessive. Some deliverables were taken out of production, resulting in down time. This fractured Web effort cost money, tied up resources, and gave customers the wrong message.

The first objective was identified. The team had to find a way to increase release reliability and accuracy.

Problem Area Two: Customer Dissatisfaction

Customer dissatisfaction was at an all time high. The team plotted customer complaints against time, for the last four quarters. The results indicated that customer expectations were not being met, there were some customer retention issues during this period of time, and Web downtime was not proactively communicated to the customer base.

The second objective was identified. The team had to improve customer satisfaction.

Problem Area Three: Cycle Time

Cycle time (CT) was considered to be an important metric and a long-term parameter of quality. CT was calculated as the time from the initial client request to the day of Web-page delivery. Since CT did not include fixes that were made after delivery, that estimate was made separately under the category of maintenance. The steps that make up a CT metric usually consist of both core activities and support activities. This critical path calculation was based on core activities only and was quite erratic.

A third objective was identified. The team had to find a way to reduce total cycle time.

Problem Area Four: Excessive Turnover Rate

The high resource turnover rate might have adversely affected the transference of process knowledge. Not only were there many unnecessary handoffs, it took an average of six months for new resources to get a handle on development standards. That problem did not bode well for the long-term continuity of quality. To complicate matters further, a decision was made to outsource the development effort. To balance that effort without losing additional quality in the transition was one of the greatest challenges of the IT organization.

A fourth objective was identified. Reduce rework in a virtual environment.


An IT standard is defined as a reference of measurement that is used to compare work effectiveness against what is considered to be the preferred method of operation. A model is an idealized design to be imitated by workers who use standards to reach a desired outcome. In an effort to understand the IT development process on a high level and drive operational excellence, the quality team designed a standardized model for Internet development. The model mapped the software development life cycle for the Internet against the Six Sigma quality cycle: define, measure, analyze, improve, and control (DMAIC), and then aligned those cycles against the project management life cycle: initiate, planning, execution, control, and closeout. The model not only assisted the development team to focus on reducing development variation, but also gave the rest of the company an understanding of the IT involvement necessary for success (see Figure 1).

Previously, the IT organization did not get involved with the Six Sigma cycle until the project team started the improve phase. That placed the IT development effort into a reacting mode. By that time, decisions were already made regarding project scope, timelines, objectives, and budget.

This model provided IT with the opportunity to get involved with the define phase of the project and carry full alignment with business strategy. The improved process allowed a cross-functional focus on user requirements from the start of the Six Sigma life cycle and effectively enabled IT to meet or exceed customer expectations with every deliverable.


A business process can be described as a series of steps that bring a key service-based transaction to a state of closure. If a company’s Web pages are not correlated with business strategy, the resulting image becomes jumbled with mixed messages, the development work becomes burdensome, and the organizational culture tends to stray away from true customer service. Alignment begins with the mapping of core business processes and subprocesses to the voice of the customer. A business process matrix (BPM) assists with that correlation by arranging the most critical processes with the user experience. The BPM brings together prioritized projects with a company’s mission statement, strategy, and with the totality of customer events. The BPM shown below builds upon the “house of quality” concept by breaking the user experience down into core process steps. Web-enabled projects, shown on the left, are driven toward those transactions that enhance the user experience (see Figure 2).

There is also an element of interruption in every development initiative (bug fixes, quick turnaround items, and so on) that has to be monitored. Interruption is not planned and takes up valuable resource time. The resources who receive those requests become “interrupt driven,” with more critical projects suffering from time constraints and budget overruns. To avoid that pitfall, those quick hits should be directed to dedicated resources that keep their fingers on the pulse of development quality (termed as issue resolution planning). Issue resolution planning enables an Internet development manager to focus resources on critical workload. This provides immediate feedback to the executive staff of the available resources, team assignments, skill-set inadequacies, and resource constraints. The result is more control, greater success, and better alignment with customer requirements.



Define is the first step in the DMAIC cycle, and as shown in the Internet development model (see Figure 1) maps to project selection and initiation. Since the project was already approved and launched by the COO, executive buy-in was strong. The CIO was selected as project sponsor, and a Six Sigma Quality Black Belt with an IT background was assigned to the project. The project team consisted of those individuals who exhibited an understanding of the scope and were passionate about seeing the project to a successful completion. Once the team was identified, a roles and responsibilities matrix was started. An initial agreement was reached on project parameters. Surveys were conducted. Information was correlated against customer requirements and internal processes that touched the customer (see Figure 3).

High-level requirements were aligned with business strategy, and a high-level scope document was placed into an intranet file. Stakeholders signed the scope definition document. A dashboard was created to focus the team on project objectives. The dashboard was placed on the Intranet for round-the-clock communication to the executive staff.

It was determined that the scope baseline would be frozen at some point, and future changes would have to be handled through a rigorous change control process. That change process would include all project factors, from requirements to objectives, down into the code base. Project objectives were specific, measurable, understandable, as well as acceptable to all team members and stakeholders (see Figure 4).

Tools used during the define stage

  • Brainstorming
  • Critical-to-quality (CTQ) analysis
  • Customer surveys
  • Process map
  • Dashboard
  • Scope definition document
  • Roles and responsibilities matrix

Steps followed during the define stage

  • Obtain an organizational overview
  • Identify key stakeholders and executive sponsors
  • Identify steering committee and key managers
  • Identify project team members
  • Meet executive sponsor to review project approach
  • Conduct interviews
  • Document “as is” process at high level
  • Identify low-hanging fruit for quick improvement
  • Report out on key finding from the interviews and produce problem/opportunity statements
  • Establish a high-level timeline for the improvement
  • Outline the approach
  • Assemble champion and project team
  • Review high-level project approach and schedule
  • Discuss first impressions of strengths and opportunities
  • Identify major sources of variation
  • Operationalize process objectives


Measure is the second step in the DMAIC cycle, and as shown in the Internet development model, maps to project process review and planning (see Figure 1). During the measurement stage, process efficiencies were analyzed and documented, improving the team’s understanding of quality, variation, and cycle time. The analysis began with an Internet SIPOC (premier customers, suppliers, input, process, output, customer base). The Internet SIPOC diagrammed the voice of the customer against the enablers that would satisfy the customer via the Internet. Starting with the premier customer base, and then targeting suppliers, business input, e-business process, business outputs, and the customer base, the entire SIPOC focused effort on Internet transactions. Listed vertically in this diagram were the prioritized requirements that would add value to the Internet experience. In each case quality and error reduction were important characteristics for transactional excellence. The number 1.1 would represent the most important quality requirement for premier customers, e-business marketing (see Figure 5).

The prioritized requirements were in alignment with the critical to quality criteria review. The SIPOC was planned on the intranet and reviewed by all organizations for clarity and alignment. In an environment of continuous improvement, everyone is responsible for understanding process flow. Small changes in one process area can result in significant improvements for an entire organization. The SIPOC was also compared to core business values to make sure that the alignment was not only with the objectives of the company but also the culture.

The Internet is causing organizations to constantly shift the paradigm of quality assessment, creating challenges never before experienced. In this dynamic environment, balance is attained through process control.

The team defined a process as a series of events and transactions that give value to the business and align with organizational vision of customer satisfaction. The testing process was chosen as an area of investigation, as the issue was directly attributable to deliverables taken out of production.

The dependent and independent variables were identified and followed for data trends: y = time for defects to be resolved, and x = number of defects. A decision was made to measure the total number of defects with each delivery (see Figure 6).

Tools used during the measure stage

  • Defect definition diagram
  • Operational definition diagram
  • Testing flowchart

Steps followed during the measure stage

  • Develop Internet SIPOC
  • Develop a data collection plan regarding process defects
  • Establish a voice-of-the-customer diagram
  • Establish a CTQ criteria review
  • Collect voice-of-the-customer data
  • Determine what competition is doing right
  • Benchmark available data
  • Identify parameters (accuracy, cycle time, client info)
  • Determine measures needed for dashboard
  • Determine and graph current and past accuracy
  • Look for trends in data timelines


Analyze is the third step in the DMAIC cycle, and as shown in the Internet development model maps to prototyping and execution (see Figure 1). During the measurement stage, process efficiencies were analyzed and documented, improving the team’s understanding of quality, variation, and cycle time.

A cause-and-effect diagram was used to review the different areas of possible impact to the development effort. Each team member was assigned an area for investigation and reported on his or her findings at a follow-up meeting. The area of critical mass was determined to be unit testing. This area appeared most poorly controlled by existing quality assurance and audits (see Figure 7).

The team used a Pareto chart to visually display the collected data. This analysis enabled the team to focus on the “vital few” areas of extreme variation with 86 percent of the errors rooted in the Web page development effort. Those errors should have been caught during the unit test initiative. Interviews with development personnel indicated that developers were performing their own unit testing and, in an effort to meet critical development timelines, were reducing the robustness of the unit test.

An analysis of alternative solutions was made using an options chart (see Figure 8) and the team determined that developing a standardized unit testing strategy would be the best opportunity for improvement.

A cross-functional team met to determine the “must have” testing criteria for a unit test. During this process, every developer appeared to have a different idea of what constituted a unit test. Standardizing the unit test would infuse quality into the development process. Placing the information in an intranet file would allow best-practice sharing among developers across the entire company.

Tools used during the analyze stage

  • Pareto diagram
  • Opportunity value diagram
  • Process map
  • Cause-and-effect diagram
  • Options chart

Steps followed during the analyze stage

  • Develop detailed process diagrams of the affected process
  • Identify value-added steps of the process
  • Identify nonvalue-added steps of the process
  • Analyze for root-cause effects of defects
  • Review variation
  • Discover the causes of variation
  • Discover the opportunities


Improve is the fourth step in the DMAIC cycle, and as shown in the Internet development model maps to Internet enablement and project monitoring (see Figure 1). During the improve stage, the team designed and implemented the standardized unit test. Testing has always been an important yet time-consuming process. Now that the customer is next in line via the Internet, there is no room for error. Error detection, message description, frame isolation, and rectification are extremely difficult, if not impossible, to catch at the system level and must be tested for at the unit level. At the unit level code can be most
easily comprehended and viewed.

The aim of a standardized testing process is to eliminate defects while sharing best practices among developers. Placing the testing templates and actual tests on the intranet provides a shared vision of quality and is a critical step toward successful delivery. As with the initial adoption of any new process, using a new template might create resistance. As more experience is gained, however, the amount of time required for thorough testing is reduced. There are no short cuts around building quality into the Web site. The additional time taken up front will provide more benefit to the client or customer at the time of delivery.

As part of an integration plan an Internet development team might set-up the scope document in an intranet file, and receives sign-off from all of the IT components: development, operations, infrastructure, facilities, training, production, and the help-desk areas.

The Integration Factor of Improve

The benefits of a standardized testing process can be: 1) better familiarity to developers with the products they work on; 2) solid guidelines for new team members to refer to for quality standards; 3) an enhanced feeling of ownership in the product built; 4) overall improvement in testing skills; 5) more time available to spend on developing code and less time used on misguided testing; and 6) improved customer satisfaction.

Integration is an important step with Internet development as the final product should be seamless across all functional areas. This stage allows for the evaluation of risk implications with inclusion of everyone in the communications process. The entire business knows how the Internet vision is growing, resulting in one vision, a fully integrated schedule, and a success factor that is shared by all.

Tools used during the integration stage

  • Integration impact matrix
  • Standardized unit-level test
  • Intranet unit-test template file
  • Issues log
  • Quality assurance plan (unit-test templates)
  • Quality control plan (review completed tests)

Steps followed during the integration stage

  • Choose an integration review team
  • Review internal sources of integration
  • Review external sources of integration
  • Develop an integration impact matrix
  • Review integration impacts
  • Identify the causes of impact
  • Identify the opportunities
  • Identify the integration points
  • Identify areas of risk
  • Identify risk mitigation plan


Control is the fifth step in the DMAIC cycle, and as shown in the Internet development model maps to contract review and closeout (see Figure 1). In this case study the team wanted to be sure that the improvements, once implemented, held value and did not revert to the error-riddled baseline. The team designed a dashboard that allowed an effective review for customers, clients, and development staff. The dashboard was also placed on the intranet for round-the-clock review and assessment (see Figure 9).

As part of the control initiative, the project team should seek feedback from the entire business team, whether that input is no impact or large impact. If the feedback is negative, correction should be taken immediately. In this case study the project team delivered many benefits.

Benefits derived from the project

  • Reliability and accuracy increased by a rate of 88 percent.
  • Customer complaints were reduced by 75 percent.
  • Cycle time was reduced by 28 percent.
  • Rework was reduced in the virtual environment by 82 percent.

Tools used during the control stage

  • Dashboard
  • Planned vs. actual revenue by month
  • New extranet hits
  • Repeat extranet hits
  • Intranet
  • Dashboard

Steps followed during the control stage

  • Finalize process measurements
  • Review the magnitude of improvements to be realized
  • Determine “lessons learned”
  • Determine if lessons learned can be shared via the intranet
  • Commit to applying lessons learned to other initiatives
  • Compare final advantage to initial return on investment
  • Apply initiative into competitive advantage


“Remember, the (Internet) applet doesn’t fall far from the tree.” If the tree is strong, the fruit produced will be of acceptable quality. If the tree is weak, the fruit from that tree will also be inferior. Within this context, there has been an effort to infuse Six Sigma quality into Internet development initiatives so that the user experience is of the highest quality possible.

An Internet team, like a tree, has many branches that bear fruit, yet that same team grows from one robust presence, or trunk. If the basic process or foundation that supports the branches is diseased, the fruit will be of poor quality. The e-business arena has experienced significant change during the past year. Wireless technology, Web-based project management tools, outsourcing, Web development, supply-chain management, and a focus on customer relationship management led the chain reaction that morphed the business environment into something more dynamic than what we have been traditionally accustomed. The opinion of most business executives on the role of e-business in the organization has changed substantially over the past year. A year ago, a company without an e-strategy was strategically challenged. If that same business does not have an e-strategy today, it is not considered to be a challenge at all by their competition. Since the customer is next in line, how does an organization make sure that the fruit from the development tree is as ripe and well formed as fruit from the Six Sigma tree of knowledge? The answer lies in the integration of Six Sigma process quality with the Internet development effort.

Ideally, Internet objectives should be in alignment with a company’s mission statement. While the mission statement represents the high-level revenue expectations of the company, the tactics that take a company to the level of revenue where it wants to be are inherent in the basic objectives of each Internet initiative. The e-business development process is the catalyst that links the two together. If an Internet initiative is not positioned for that alignment, the business may want to question the relevance of spending resources, time, and money on an effort that will not bear fruit of acceptable quality.

For the Internet to stand the test of time and sponsor support, the Web must supply the customer with value and business advantage. Those basic elements of business survival cannot be present without the right approach to Internet quality. Changes in core Internet development processes were reviewed in this article. These changes resulted in a greater advantage for stakeholders, clients, and customers.

A Six Sigma Internet development model was created by the team, and used as a template for Internet development training. All future members of Internet teams were trained in the entire process, and IT was from that time forward included in the define phase of every initiative rather than the beginning of the improve phase.


Harrington, H. J., and T. Mc Nellis. 2001. The E-business project manager. West Chester, Pa.: BestSellersPublishers.


ASQ’s Quality Management Division. 1999. The certified quality manager handbook. Milwaukee, Wisconsin: American Society for Quality.

Duncan, W. R., and PMI Standards Committee. 1987. A guide to the project management body of knowledge. Newtown, Pa.: PMI.

Harrington, H. J., Daryl R. Conner, and Nicolas L. Horney. 2000. Project change management. City: McGraw Hill.


H. James Harrington currently serves as chief operating officer for Systemcorp and as chairman of the board of the Harrington Group, TQnet, Inc., 3D Technologies Ltd., Emergence Technology Ltd., and Performance Improvement Network.

Thomas Mc Nellis is an adjunct professor at the Temple University School of Business. Mc Nellis currently serves as director of operations for the Association for E-business Research, a research-based consulting firm, and is one of the founding members of Angels on the Internet, a program to introduce Internet education into inner-city grade schools. Mc Nellis earned four degrees, and holds four certifications: PMP, ASQ Certified Quality Manager, Six Sigma Master Black Belt, and EQM, and teaches e-business for the American Society for Quality and the Project Management Institute. He can be reached at 1215 N. Ashbrooke Dr., West Chester, PA 19380 or by e-mail at .

If you liked this article, subscribe now.

Featured advertisers

ASQ is a global community of people passionate about quality, who use the tools, their ideas and expertise to make our world work better. ASQ: The Global Voice of Quality.