Volume 6 · Issue 2 · February 2001
QuEST Forum Meets in Atlanta January 22-26
Voting on Revised TL 9000 Handbooks Ends February 15
On February 1, 2001, the 117 full members of the Quality
Excellence for Suppliers of Telecommunications (QuEST) Forum
began balloting on approval of versions 3.0 of the two TL
9000 handbooks. Expected approval of the TL 9000 Quality
Management System Requirements Handbook when voting concludes
on February 15 will represent the first alignment of a sector-specific
supplement with ISO 9001:2000, slightly more than 1 year after
the first registrations to TL 9000 and 3 years after the founding
of the QuEST Forum.
Revisions to both the quality management system (QMS) requirements
handbook and TL 9000 Quality Management System Measurements
Handbook were completed and reviewed by work groups (WGs)
at a QuEST Forum meeting in Atlanta January 22-26, 2001, at
which both were approved for balloting by the Forums
full members (total membership is 159, although only 14 service
providers and 103 suppliers are presently full members entitled
At the Atlanta meeting, the Forum also recognized TL 9000
registrations by 14 suppliers to the telecommunications industry,
bringing to 56 the number of registrations reported since
the Forum recognized the first 16 TL 9000 Pilot Program registrations
in January 2000.
The activity within the telecommunications sector as represented
by the QuEST Forum and the development and future of TL 9000
is notable not only because of the incredible QMS developments
within this sector, but because of the healthy debate that
took place over the direction of sector-specific standardizationand
the resounding support for that directionduring an Executive
Round Table held during the Forum meeting.
At a time when the release of ISO 9001:2000 may raise questions
about the need for sector-specific requirements aligned with
ISO 9001, the youngest industry group to engage in QMS standards
development remains committed to maintaining both ISO 9001
linkage and sector-specific "adders" to the generic
ISO standard. Those "adders" include not just the
80 requirements added to ISO 9001:2000 to satisfy QMS needs
in the telecom supply chain, but enhancement of its measurements
program and development of its Business Excellence Acceleration
Whether your organization is a supplier to the telecommunications
industry or you are interested in developments within the
QuEST Forum program that might be replicated in other fields,
you may find information of interest in the following reports.
They detail the efforts of the work groups (WGs) that completed
the latest editions of the TL 9000 handbooks, the debate during
the Executive Round Table and developments involving the BEAM
WG and Supply Chain Subcommittee.
QMS Requirements and Measurements Reach Version 3.0
In revising the QMS Requirements Handbook, the QuEST Forums
Requirements WG met in July, September, October and November
2000 to review 392 suggestions that Forum members submitted
for consideration. It also involved updating the draft of
Version 3.0 after each meeting and circulating the revised
drafts to the WGs members for review and comment.
The Requirements WG used subteams to conduct much of the
work in a way that would maximize the WGs resources.
For instance, one subteam was dedicated to combining the glossary
in the version 2.5 Requirements Handbook with the one in the
version 2.5 Metrics Handbook (now Measurements) so as to create
a single, consistent TL 9000 Glossary that would be updated
for the version 3.0 handbooks and contained in both.
Three other subteams focused on various aspects of the Requirements
Handbook revisions. As has been previously reported, the purpose
of revising the QMS Requirements Handbook was to align its
contents with ISO 9001:2000, since version 2.5 was aligned
with ISO 9001:1994. This involved three objectives:
- Comparing version 2.5 with the Final Draft International
Standard (FDIS) of ISO 9001:2000 and then ISO 9001:2000
to identify any changes to the ISO requirements that would
eliminate the need for or require any new TL 9000 "adders"
to the ISO 9001 requirements. While version 2.5 contained
83 adders to ISO 9001:1994, version 3.0 contains 80 adders
to ISO 9001:2000, with 3 previous requirements now covered
by ISO 9001 and no new requirements needed.
- Restructuring the adders to fit the new structure
of ISO 9001. The reorganization of the requirements from
a 20-clause structure in ISO 9001:1994 into a process-approach
structure in ISO 9001:2000, whereby requirements are grouped
into 5 sections (Sections 4-8), was the biggest challenge
for the subteams. The restructuring meant that the subteams
had to determine where the TL 9000 adders should be placed
in the ISO 9001:2000 framework, since some requirements
from ISO 9001:1994 were separated into different parts of
ISO 9001:2000 while others were combined with other parts.
- Ensuring the wording of the TL 9000 adders was properly
updated to reflect any agreed upon adjustments based on
the Forum member suggestions and to complement the changed
language usage in ISO 9001:2000. The intent was to verify
that the requirements added to ISO 9001 to meet telecom
sector needs did not change once the handbook was aligned
with ISO 9001:2000, the subteam also was verifying that
the intended requirements contained in the adders actually
were reflected in the wording. In other words, was the intent
of the drafters of TL 9000 conveyed by the requirements
as written in versions 2.5 and 3.0?
The entire WG took "one last look" at version 3.0
of the Requirements Handbook in Atlanta before sending it
to the Forum for membership ratification and publication in
March 2001, according to Brendan Pelan, Chair of the WG. "During
one of the work group sessions, an observer made the comment
that the members were the best working team he had ever witnessed,"
remarked Pelan, who expected the revised edition to receive
overwhelming approval. "It is a tribute to all members
of the work group that we have been able to produce another
quality document in a relatively short period."
The QuEST Forums Measurements WG met January 25, 2001,
in Atlanta, resulting in approval of the final draft of the
TL 9000 QMS Measurements Handbook, Version 3.0, for release
to the Forum membership for balloting. Almost 40 telecommunications
organizations are represented on the WG, with a total membership
approaching 70, which reflects on the importance of the WGs
According to Rick Werth of SBC Communications, Chair of the
Measurements WG, upgrades from version 2.5 of the handbook
- The handbooks title has been revised to replace
"quality system" and "metrics" with
"quality management system" and "measurements",
as reported previously.
- An Initial Return Rate Measurement was added to the Return
Rates Section to cover the first 6 months of a products
- The software section was reorganized to better align with
GR-929-Core RQMS (Core Reliability and Quality Measurements
for Telecommunications Systems).
- The Product Category Table was changed to cover the addition
of Section 4.3, Ancillary Operations and Maintenance Products,
Section 7.7, Procurement Services, Section 7.8, Logistical
Services, and the splitting of Section 3.2.1 into two partsManual
Cross Connect Systems and Digital Cross Connect Systems.
The Measurements WG also set new priorities for 2001 in Atlanta,
with several subteams being evaluated to see what role they
will playand what changes may be needed to the subteamsas
the WG begins drafting Version 4.0 of the measurements handbook.
Priorities for the next version of the Measurements Handbook
- A performance indicator (Outgoing Quality Measures)
- Handset quality measures
- The product category table.
Other areas to be evaluated by the Measurements WG in version
- Continued input from the Oversight and Training WGs
and the Supply Chain Subcommittee
- Industry best practices submitted by QuEST Forum members
or presented at the September 2001 Best Practices Symposium
- In-process quality measures.
The Measurements WG estimates that version
4.0 will be ready for publication in mid- to late-2002, while
the Product Category Table will be updated separately several
times in the interim. For a look at the role of measurements
in TL 9000 and the telecom industry, see "Measurements,
Telecommunications Quality and TL 9000".
Executive Round Table: Is TL 9000 the Right Direction?
The QuEST Forums leadership decided to hold its first
Executive Round Table in Atlanta to provide attendees with
meaningful information and a chance to ask specific questions
about TL 9000 and the Forum. Members of the QuEST Forum leadership
were selected to briefly explain their perspectives on one
of four subject areas, followed by an open-floor question-and-answer
session that gave these leaders a chance to address specific
questions and concerns from the audience.
The four subject areas were as follows:
- Service providers vision of the QuEST Forum
- TL 9000 and ISO 9001:2000
- How to prepare for a TL 9000 audit
- Why become a member of the QuEST Forum.
The first subject area included representatives of two of
the regional phone companiesSteve Welch of SBC and Don
Pickens of Bell South. They discussed the overall vision of
the QuEST Forum and how the QMS measurements data collection,
which is part of the TL 9000 program, will help their organizations
and the telecommunications sector as a whole improve their
operations and provide better service to their customers.
However, it was the third panelist that generated the greatest
discussion and debate during the Round Table requiring more
time to be devoted to the session than had been planned. Marty
Lustig of Sprint, which provides long-distance phone and nationwide
wireless communications services, began his presentation by
indicating that he spoke on behalf of Sprint, as well as AT&T
and WorldCom, when he said that their vision of the QuEST
Forum involved the need to redirect the Forums efforts
towards an effective improvement orientation.
Lustig recommended that best practices be identified and
that practical ways be sought to identify and disseminate
these practices. It was the opinion of these three major telecommunications
service providers (TSPs) that TL 9000, including the measurements
database, was not likely to lead to overall industry improvement
without the BEAM approach.
Lustig also indicated an opposition to requiring suppliers
to obtain registration to TL 9000, because he did not view
it as likely to be effective in the long-term. Indeed, he
urged voluntary usage of the Malcolm Baldrige National Quality
Award (MBNQA) assessment approach.
Pickens and Welch were joined by many Round Table attendees
who responded to Lustigs vision by warning that companies
should not polarize around advocating and/or requiring the
use of TL 9000 or MBNQAthat TL 9000s QMS
requirements and measurements and the Baldrige approach are
both beneficial and complementary.
It was clear that there was little support among most TSPs
and their suppliers for abandoning the use of the TL 9000
QMS requirements and measurements in the industry. The suppliers
in particular find these TL 9000 elements to be valuable because
of the consistent QMS requirements they provide and the benefits
they expect to see as the TL 9000 measurements data collection
is used to develop a database that will permit organizations
to compare their QMS performance against the industry.
Clearly, the vision of three major telecom companies for
the QuEST Forums future direction is worth considering,
and their overall vision supports the goal of identifying
and disseminating best practices throughout the sector.
However, the Forums membership by and large demonstrated
support for continued development and use of sector-specific
QMS requirements aligned with ISO 9001:2000 and of QMS measurements
identified and/or developed specifically for the sector. Organizations
in the telecom sector find both the TL 9000 requirements and
measurements useful in improving their processes and identifying
areas that need the most attention.
The intention of many organizations is to benchmark their
performance against the industry database to see where each
organization is ahead of the competition and where it is behind.
To clarify its position on this subject,
AT&T submitted a letter
to the QuEST Forum subsequent to the Round Table. The letter
acknowledged the Forums support for TL 9000 development
and use but advocates the use of ISO 9001:2000 and state and
national quality award programs rather than pursue a TL 9000-based
registration and measurements approach. THE OUTLOOK
has obtained a copy of the letter, which is reprinted on the
Much of the question-and-answer session is available in the
FAQ section of the QuEST Forum web site at www.questforum.org.
BEAM and the Supply Chain
Eugene Hutchison, Chair of the BEAM Team, announced that
Ron Asbury, Chairman and past president of the American Society
for Quality (ASQ) and an advocate for business excellence
models (BEMs), has agreed to serve as a subject matter expert
(SME) and facilitator for the BEAM Team.
A key objective of the Team in 2000 was to develop telecommunications
industry guidance for the application of BEMs to an enterprise,
which involved building relationships with the National Institute
of Standards and Technology (NIST) and EFQM.
In Atlanta, the BEAM Team continued work on this guidance
document, with a committee draft expected to be available
in February 2001. The document will provide guidance in the
following subject areas:
- Strategic partnering
- Managing the supply chain
- Efficient product delivery and support
- Life cycle management and planning
- Optimizing time-to-market
- Encouraging and managing innovation
- Managing continual improvement and organizational
and cultural change
- Technology assets management
- Network reliability and availability.
During 2000, BEAM has expanded its relationships with BEM
agencies, including the Director of the MBNQA program at NIST
and a representative of EFQM during the London QuEST Forum
meeting. The goal is to find ways for the Forums BEAM
Team, MBNQA and EFQM to support each others activities
to improve performance in the telecommunications industry
The Supply Chain Subcommittee is involved in the following
activities to assist direct and indirect suppliers to the
- Creation of a list of proposed additions and/or changes
to the Product Category Table to suit the needs of direct/indirect
- Identification of applicable supply chain requirements
- Development of a TSC Supplement, which is expected
to be submitted to the Forums Oversight and Measurements
WGs for review and approval. The TSC Supplement is still
being developed by a task team but would be a guideline/supplement
to the TL 9000 handbooks to help explain how an indirect
supplier can conform to the requirements of TL 9000.
Additional information about the Subcommittee will be provided
as its activities are completed and proposed supplements and
other documents are submitted to the Forum.
UTD and the QMS Data Indices
As previously reported, a pair of indices unique to the telecommunications
industry is being developed that will enhance the effectiveness
of TL 9000 conforming QMSs by evaluating the systems
impact on telecommunications products and services. These
indices are referred to as the TL 9000 Performance Index (TPI)
and Telecommunications Quality Report (TQI).
The University of Texas at Dallas (UTD) was tasked to lead
the research and development of the TPI and the TQI. The concept
of TQI matured during late 1999, when the following goals
were developed by the QuEST Forum leadership:
- Develop a TPI that will be reported quarterly to the
QuEST Forum and represents the effectiveness of the TL 9000
- Determine correlations between the TPI and measurements
taken to gauge related telecommunications customer satisfaction.
- Establish a TQI based on the TPI and the correlation
The following strategic plan was also developed:
- Derive a TPI from data residing in the TL 9000 Measurements
Repository System (MRS).
- Develop key telecommunications industry criteria that
will measure customer satisfaction and perception of service.
- Combine the output of the TPI and the customer satisfaction
key criteria to create the TQI.
- Use the TPI and TQI as tools for continuous improvement
by reporting them as industry performance indicators.
- Use the TPI and TQI results to target QuEST Forum
WG activities where appropriate.
The Forums MARCOM (marketing and communications) WG
has completed work on the QuEST Forum Brochure, which
provides information on the Forum, including information on
its board of directors and membership, the MRS and Registration
Repository System and meetings schedule. Copies of the brochure
can be obtained by calling Irene Gliniecki at ASQ (800-248-1946 (United States and Canada only),
A list of upcoming meetings is located
Further information on the QuEST Forum, its activities and
how your company might become a member can be obtained by
contacting Jeff Weitzer at ASQ (email@example.com).
THE OUTLOOK will provide a look at the revised handbooks
once they have been released and will provide updates on other
QuEST Forum activities in future issues.
Letter from AT&T
January 26, 2001
Dear Board Members,
In light of recent events, we at AT&T feel there is a
need to clarify our position regarding the QuEST Forum, the
TL 9000 quality management system, and the associated measurements.
We believe that a better approach for driving excellence within
our industry would be the establishment and sharing of best
practices, using National and State quality awards, and adopting
ISO 9001:2000 as the recognized cross-industry standard (one
that already has a well-established infrastructure). We believe
this is consistent with the BEAM model shared on 1/22/01 by
Steve Welch. We feel the more prescriptive an approach, the
less likely it will satisfy the needs of the entire supply
chain. This position was clearly communicated to the QuEST
Forum board in July 1999 and our suppliers were informed of
our position in the fall of 1999.
However, it would not be realistic to think that the QuEST
Forum would necessarily embrace our recommended approach and
cease activities already well underway by a significant number
of service providers and suppliers. As a leader in the telecommunications
industry, AT&T joined the QuEST Forum and has worked within
its structure to clarify and refine the existing requirement
and measurement documents. Additionally, as TL 9000 is deployed,
we have collaborated with other service providers and suppliers
to embrace meaningful measurements that will provide the data
necessary to accelerate improvement in our industry through
the QuEST Forum. Since 1998, AT&T has supported the work
of the QuEST Forum through annual membership, providing resources
to the Requirements and Measurement Work Groups and subteams,
and most recently, by sponsoring the QuEST Forum event in
Atlanta at the Platinum level.
Although the approach selected by the QuEST Forum was clearly
not the approach preferred by AT&T, we have worked and
continue to work within the structure of the QuEST Forum to
make our industry stronger.
AT&T Supplier Management
Stephen E. Ford
AT&T Quality Office
2001 Forum Meetings and Conferences
- Global Conference in Buenos Aires, Argentina April 24-25
- Forum Meeting in Berlin, Germany July 10-12
- Best Practices Conference in Dallas, TX September 11-12
- Forum Meeting in Singapore October 23-25
- Work Group Meetings:
- San Diego, CA March13-15
- Tampa, FL April 17-19
- Berlin, Germany July 10-12
- Dallas, TX September 13-14
- Singapore October 23-25
- Tempe, AZ November 13-15
Quality and TL 9000
By Art Morrical
In the absence of measurements, customers fear the worst
about the performance of a product. After all, without measurements,
a customer is buying blindly. Thus, taking measurements of
the product and making them available to your organizations
customers is a good way of marketing the quality of your product.
In theory, it is possible to create any number of measurements.
Of greater importance is that the information extracted from
the data be useful and accurate. When an organization undertakes
the collection of data, there is a tendency to overcollect
and underanalyze. The quantity of data collected and the number
of measurements does not need to be overwhelming. It is important
to understand that measurements, in and of themselves, are
not the goal of a useful QMSanalysis of the measurements
is. Analysis is the key to transforming raw data into useful
pieces of information. Keeping this in mind, it is essential
that a measurement program be analysis-driven instead of data-driven.
The QuEST Forum Measurements WG developed some of the TL
9000 measurements using the experience of the groups
members with GR-929-CORE RQMS (Core Reliability and
Quality Measurements for Telecommunications Systems), which
are telecommunications industry measurements that have been
in use for and have been evolving during the last decade.
Due to the RQMS linkage, when a measurement from RQMS is used
by a supplier registered to TL 9000, the supplier can use
the RQMS version of the measurement to conform with TL 9000s
requirements and can send those measurements to UTD for the
TL 9000 MRS without alteration.
The calculations themselves must be in compliance with the
latest issue of the RQMS to have been released. The intent
is to prevent redundant effort by suppliers when both RQMS
and TL 9000 measurements are contractually required. In that
situation, TL 9000 conformance audits would accept an RQMS-based
For example, at the time of TL 9000 implementation at Lucent
Technologies 5ESS Project, the Project was producing
the RQMS measurements needed for TL 9000 conformance and registration
and had been sending them to its subscribing customers. TL
9000 also requires that objectives are placed on the measurements
and that the results are sent to the suppliers management.
Since the RQMS measurements have customer-defined objectives
that the 5ESS Project was already embracing, the Project was
in conformance with the required measurements for TL 9000.
The 5ESS Project produces and sends quarterly reports of the
measurements to the customers that detail current trends and
provide analysis of the data.
An organization can gain many benefits by inputting data
to a data repository system, including:
- The use of drop-down menus provides the user with
- Data verification at the time the measurements are
collectedusing data checks and selecting from standard
field valuesavoids typing and abbreviation errors.
- Periodic database audits can clean up inaccuracies
in the data.
- Long-term measurement trends reveal a measurements
variability and assess a suppliers maturity and room
- The data identifies the areas where management can
initiate action to get the biggest payback.
In addition, the TL 9000 measurements will provide industry
means and ranges. This will allow a telecommunications supplier
to perform a self-evaluation and fine-tune its resource allocation
in response to its position relative to the industry direction.
Another key to an effective measurements program is keeping
the measurements visible. The measurement results need to
be visible to all project members, which the 5ESS Project
has been accomplishing by using web-based access to TL 9000
customer measurement data, called a "dashboard".
Dashboard measurements are tools to help employees monitor
the progress of the organizations business.
Just as a driver glances at the gauges on the dashboard to
monitor speed, temperature, oil pressure, etc., a project
member can look at his/her "dashboard" and, based
on the status of the business, take the appropriate actions
to maximize performance. The "gauges" are updated
monthly to give everyone the most timely and accurate figures
available. Management can set objectives based on the figures
and manage the measurements to close any performance gaps.
Using the TL 9000 measurements, quality improvements can be
focused around how customers define quality that can be demonstrated
Measurement results can be used in executive reviews, since
they provide a window into customer interactions, thus connecting
a customer sales team to a product teams performance.
Customer visits can then cover shared measurements results
and conclude with a joint setting of gap reduction goals.
The most important measurement area for improvement can also
be jointly determined.
Measurements are a key part of a healthy and effective management
system. Overall product quality improvement is the results
of increased visibility of measurements data that show the
level of compliance with product specifications and the integration
of these measurements into standard management practices.
Product quality improvement also results from increased focus
on areas needing attention.
When TL 9000 measurements are used, there can be enhanced
customer/supplier relationships and the introduction of improvement
programs based on well-defined measurements. More informedbetterdecisions
can be made as to where to apply resources. The TL 9000 measurements
also lead to standardized data collection that provides opportunities
for industry comparisons and benchmarking. The long-term use
of the TL 9000 measurements can result in improved industry
performance and improved telecom services.
Art Morrical is a Manager of the Lucent Technologies
5ESS Project and an initial participant in the QuEST Forum,
working with the Metrics, Metrics Implementation, Requirements
and Training Work Groups. He recently provided THE OUTLOOK
with an in-depth look at the TL 9000 registration effort involving
the 5ESS Project (see "TL 9000 Registration by Lucent
Focuses on Flagship Project", THE OUTLOOK, January
2001). Mr. Morrical also represents Lucent Technologies at
the RQMS Technical Forums. He has an MS in Computer Science
from North Central College in Naperville, IL, and a BS in
Electrical Engineering Technology from Bradley University
in Peoria, IL. Mr. Morrical has been an ASQ-Certified Quality
Engineer (CQE) and an ASQ Senior Member since 1989. He is
also the President of the Library Board of Trustees in Sugar
If you liked this article, subscribe