Q: During an audit closeout meeting, what is the right thing to do if the auditee refuses to accept a correct nonconformance report (NCR) by the lead auditor? Should the lead auditor submit his or her report with or without the disputed NCR regardless? What if the auditee—such as a director or CEO—interprets the case according to his or her opinion and sees there isn’t an NCR?
Ayad Mosab Al-Ani
Salalah Free Zone, Oman
A: If your nonconformance (NC) is factual, has business impact and appropriately references a standard or specification, you should not drop the finding.
It appears from the question that the auditee may not have been aware of the NC until the closing meeting. Don’t keep an NC a surprise until the closing meeting. If the audit is a multi-day event, you should discuss any NCs with the auditee as you find them, ideally during daily caucus meetings with the auditee. Discussing NCs before the closing meeting will provide the auditee an opportunity to clarify the situation or produce evidence.
Why would an auditee refuse to accept an NCR? The most common root causes are:
- The audit finding is outside the scope of the contract or audit. In the case of supplier audits, the finding goes beyond the agreed-on purchasing requirements.1 In the case of internal audits, the finding may be beyond the responsibility of the auditee function.
- Addressing the NC would not add value, such as recreating missing paperwork that has no value for the business.
- The auditee is not empowered to accept the findings. Auditees who are too junior in the organization’s hierarchy may worry about the consequences for accepting a finding on behalf of their function.
- The finding is wordy, poorly written or unclear about what to address or improve, so the auditee cannot understand what the real gap is.
- NC statements appear to be opinion-based rather than fact, failing to provide sufficient evidence and references.
- An opportunity for improvement is misclassified as an NC. Inconsistent use of audit terminology and definitions is often a rookie mistake.
- The same finding is unnecessarily repeated. An auditor who identifies a document control issue and counts separate NCs for the multiple areas in which it appears, for instance, will be perceived as being interested in generating a number of findings rather than providing valuable feedback.
- Issues unrelated to the audit are bundled with the findings. More often than you would expect, auditors incorporate known issues for the audit area into the current audit report, using the audit as leverage to get things done. There is nothing wrong with verifying through samples whether past issues are still potent and reporting them with evidence. Bundling past issues without verification, however, may cause auditee refusal.
- There is no sign of continued or future business. A particularly difficult supplier audit scenario can occur when an existing contract is near expiration and there has been no word on extension while the customer is auditing and issuing findings.
- Hostility between the auditor and auditee creates a strained, "gotcha" mentality.
- The auditor bullied the auditee into accepting a finding rather than convincing the auditee through careful and thorough reasoning.
Other reasons will likely be as varied as the personalities, character traits and capabilities of those involved in the audit. However, with awareness of the most common root causes, an auditor will be equipped to avoid most disagreements over NCs.
Obviously, an auditor’s soft skills, including tact, reasoning and persuasion, are important. Having management commitment from the auditing and auditee organizations, following auditing protocol, adequately planning and preparing for the audits, offering solid value-added feedback, promptly verifying corrective actions, linking audit action effectiveness to the organizational bottom line, and presenting the audit’s effectiveness to management will also help ensure a healthy audit management process.
Even if the auditee is a high-profile individual, such as a director or CEO, the auditor should stick to the task. During my audits, if I am convinced of the importance of a finding, I explain it to the auditee, outlining the benefits of addressing the gaps and the risks of not addressing the gaps, and provide references to support the NC. I also give the auditee the opportunity to explain any disagreement. If the reason for objection is not supported by facts and evidence, I will let the finding remain, and the auditee will have to respond to it. Management should be counted on to respect the integrity and value of the audit process, upholding findings that are genuinely offered.
Director, quality assurance
SunPower Corp., San Jose, CA
- Govind Ramu, "Expert Answers: Information Collection," Quality Progress, March 2013, pp. 8–9.
- Russell, J.P., "Exit Meeting," chapter in The ASQ Auditing Handbook, third edition, ASQ Quality Press, 2005.
- Russell, J.P.
and Terry Regel, "Chapter 1: Reporting Problems," After the Quality Audit: Closing the Loop on the Audit
Process, second edition, ASQ
Quality Press, 2000.
Q: I’m developing a customer-oriented system for updating a service operation that allows managers to gather information, make changes in service delivery and assess the effects of these changes. How do I implement that in banking? What is the best way to measure customer satisfaction based on this approach? Which quality system should I use? How do I develop a system framework for updating these services?
Dubai, United Arab Emirates
A: You have asked quite a few questions that will be difficult to answer in the short space allotted. Perhaps I can, at least, point you in the right direction.
The framing of your questions indicates the focus of your thinking is on the customer and you are intent on building a closed-loop system. This approach will provide a solid foundation for whatever method you develop.
Several important considerations should be addressed before you can begin to measure customer satisfaction. These include:
- Categorizing and segmenting your customer base to better identify and align customer requirements.
- Establishing a set of independent listening posts for each customer category or segment.
- Analyzing customer data obtained through the listening posts to determine customer requirements.
After the customer requirements are determined, they can be translated—using tools such as critical-to-quality analysis—into appropriate customer satisfaction metrics for each category or segment.
More than likely, your customer satisfaction metrics will involve performance around on-time delivery, quality and subjective measures of performance, such as friendliness, product knowledge and helpfulness.
Achieving satisfaction levels
Next, you must determine if your internal processes help you meet all of the requirements and how capable they are of achieving the required levels of customer satisfaction.
Where there are deficiencies in meeting requirements, processes must be either modified or new processes put in place. Where processes are not capable, improvements must be made.
Don’t be surprised to find gaps in requirements and processes not capable. In my experience, many organizations never originally designed and built their service delivery processes with the intent of supporting customer satisfaction. Now that service delivery is recognized as an important component of customer satisfaction and a key differentiator—particularly in high transaction environments—these processes are now receiving the attention they are due.
Without knowledge of your organization, it is difficult to recommend a specific implementation approach.
However, there are basic elements of most implementations that I suggest you consider, such as: involving those affected by changes, running pilots, selecting team members who will participate, ensuring the project has a sponsor and communicating to all stakeholders.
Your organization may have a preferred implementation approach that is already set up with software and a database. It would be wise to make a few inquiries at the outset.
Which quality system?
Again, without knowledge of your organization, it is difficult to recommend a specific quality system. Actually, a more basic question is, "Why do you feel the need to operate within a quality system?"
If your organization has already implemented a quality system, perhaps there is a natural fit for your service delivery system within that quality system. If so, I suggest you explore that avenue.
Otherwise, if no such system exists, you might want to complete your service delivery project before addressing a quality system project. That way, you’ll be able to complete your project in the event the organization does not have an appetite for taking on a quality system project. More specifically, if you must expand the scope of your service delivery project to include the building of a quality system, you may never complete the service delivery project. Keep the project scopes distinct and manageable.
T. M. Kubiak
Performance Improvement Systems
- Kubiak, T. M., The Certified Six Sigma Master Black Belt Handbook, ASQ Quality Press, 2012, pp. 148–156.
- Kubiak, T. M. and Donald W. Benbow, The Certified Six Sigma Black Belt Handbook, second edition, ASQ Quality Press, 2009, pp. 66–65.
- Shipley, David, Carl W. Keller, James Bossert, Steven S. Prevette, Duke Okes, Dale Crownover and T. M. Kubiak, “Multiple Choice—What’s the Best Quality System?” Quality Progress, July 2003, pp. 25–45.