MEASURE FOR MEASURE
Under the Microscope
Surviving your next calibration department audit
by Jay L. Bucher
Generally, there are three types of audits for a calibration or metrology department embedded within a biotech, pharmaceutical or medical-device company:
1. An internal or first-party audit is conducted by personnel from within an organization, department or quality function, and examines the system and records the results for internal eyes only.
Internal audits can also be used to show auditors the activities of periodic or continuous monitoring. To avoid potential conflict of interest, the audit is usually performed by a person independent of the department or process they are auditing.
2. An external or second-party audit can be conducted by a customer or for the purpose of evaluating a supplier. Most external audits are performed to see if an organization complies with a specific standard, guideline or regulation, and can be subjective or directive in nature.
For example, a directive audit would be for an organization that needs to comply with current good manufacturing practice (cGMP) requirements from the Food and Drug Administration (FDA). It would be informed of any findings by using the FDA’s Form 483, which is part of the public record. A subjective audit would involve an ISO standard or guideline, in which findings and write-ups are given in an audit report at the end of the inspection.
3. An external audit may also be conducted by an auditing agency, making it a third-party audit. In this case, results are forwarded to the organization’s management.
In some cases, accreditation on-site assessments typically require a demonstration of proficiency in the form of an over-the-shoulder evaluation or observation. Depending on the inspection criteria, it could include an examination of any or all of three areas: the test equipment, the technician doing the calibration or the process, which includes the calibration procedure, records and documentation.
Write it down
After an audit is conducted, the results need to be documented and kept on file for a predetermined amount of time. How long records are maintained should be stated in your organization’s records-retention policy. Follow-up audits to ensure observations, findings or write-ups have been corrected also need to be filed for future needs.
Some follow-up questions from an audit might include:
- Were any corrective or preventive actions identified?
- Were opportunities identified to improve the process in a more efficient manner?
- Was the proper authority sent the final audit results?
- Were the people using the quality system aware of the findings and updated on any system changes?
- If procedures were changed, were the technicians, supervisors and manager trained in the new procedures? Were their training records updated accordingly?
- Were your calibration procedures validated? If so, can you provide the documentation showing it was accomplished?
- Was there an area in the audit for checking if training records are maintained properly?
- Is there documentation that supports all of the above?
It is important to assign custodial responsibility for audit discrepancies. This person is assigned to follow through with the corrective action plan and to set the timetable for correcting discrepancies. That person is identified as part of the quality system, the responsible party to which the findings are sent and the one who determines how long the results should be maintained.
This is a serious process. In some instances, people’s lives, welfare, food, transportation and more are at stake. The world of metrology and calibration encompasses more areas than most people realize. To not take the necessary steps to ensure you’re doing your job to the best of your abilities and not make improvements in a proactive manner is a waste of time and money.
One of the ways you can continually improve systems and procedures is to review, examine and inspect them on a regular basis. To do this properly, you should have a second set of eyes look, review and inspect your calibration program, policies and records.
Bring in an outside party, or someone from another department or from quality assurance—if such a department exists at your organization—who is unfamiliar with what you do or how you do it. Even if you need to pay an outside agency to conduct an audit every couple of years, it is worth the time and effort to ensure your system is operating as it should.
A little guidance
What should be done during an external audit for compliance to a regulation or a standard? Is there a special set of rules or guidelines? How should everyone act? Is there a protocol or standard operating procedure (SOP) that must be followed?
Believe it or not, all these questions have been asked for decades prior to an auditor showing up on the doorstep. The following guidelines can be applied in most situations and modified to work in others.
During most audits, the auditor should look at several samples of your quality calibration program. Most audits need to be accomplished during a specific time frame, usually over the course of two or three days, and usually with one or two auditors. Because time is limited, they will want to review, observe and ask about the most important parts of your quality system.
Some of the usual items they will want to observe are:
- Evidence the quality calibration program meets the standard or regulation stated in your quality manual or subscribed to by your organization. Evidence will be in the form of your documentation and records.
- Evidence that all of your calibration technicians—and anyone else working under your quality system—understand the quality system and how it affects them, their work and their calibrations.
- Evidence the quality calibration program follows your written procedures, plans, instructions and directions. If the quality calibration program says you must follow written calibration procedures, the auditor should be able to see every calibration technician using calibration SOPs as they perform their daily work. The requirement for calibration records should be self-evident and easy to show the auditor, as should out-of-tolerance procedures, forms and customer notification documents.
- Evidence the quality calibration program is effectively providing quality products and services to your customers. High test-equipment pass rates and a low number of overdue calibrations—or none at all—are examples of quality that can be easily translated into providing the type of service and support every customer wants.
The old adage of working during peacetime the same way you would during war is almost too appropriate for the everyday calibration environment. Your quality calibration program should be set up to work the same as if every day was audit day for your organization. A quality calibration program cannot function efficiently if it only follows its guidelines just before and during an audit.
Follow your procedures, document your calibrations and perform every function as if an auditor was looking over your shoulder—because they are. Your calibration records will show an auditor how your system really works.
These are some of the deficiencies commonly found during calibration audits:
- There was incorrect or nonexistent traceability (the paper trail). If the documentation was not available to show calibration history back to a national or international standard through the working or reference standards used, from a legal perspective, calibration has not been performed.
- The calibration record’s dates did not match your database or the calibration label. In other words, they were done after the fact to prepare for an audit.
- Signatures were missing on the calibration record, calibration label or documentation, which show that test equipment was found out of tolerance and notification was provided to the user.
- Inability to perform reverse traceability when a working or reference standard was found to be out of tolerance.
- Overdue calibration stickers were on test equipment. But remember that a valid calibration record and software historical data could prove that calibration was performed and that, in the haste to turn around a lot of production, a calibration technician forgot to apply a new calibration label.
- Calibration procedures were not followed as a matter of course. For example, the auditor asked the supervisor why a particular calibration technician did not have a calibration procedure available and was told the person had done hundreds of those calibrations and did not need to have a calibration procedure.
- Where required for critical measurements, no records were kept on temperature or humidity.
- Sticky notes in the calibration procedure differed from the written procedure. There should be a change control system in place to update all parts of the quality calibration program, and sticky notes are not part of that process.
- Empty form fields were found in calibration records. This raises the question of whether data was not recorded or forgotten, or if the field is just missing an "N/A."
- Customers were found using test equipment with overdue calibration labels or that isn’t even in the quality calibration program. If records were kept of items that couldn’t be found when they were due for calibration or it can be shown the customer was ordering and receiving test equipment and circumventing the quality calibration program, most auditors will help the calibration function by stressing to upper management this problem needs to be addressed by management, not the calibration program.
This is only a small sampling of observations, write-ups and nonconformities that show how easy it is to overlook the small things while focusing on the big picture.
Keep in mind
But quality is everyone’s business, not just supervision or management. Quality calibration programs start with calibration technicians and works up from there. They are the foundation for doing the job correctly each and every day. So it’s important to remember a few things when they’re being conducted.
Auditors take notes throughout the audit. This is normal and should not cause any anxiety. If they need to make an observation or recommendation, they need data to support what happened, when it happened and the circumstances behind the event. Nobody can remember all the details, so they take notes as a matter of course.
Don’t be afraid to answer an auditor’s questions. You don’t need to memorize anything. That’s why everything is written down. You only need to know where to find it.
Also, the best answer is usually the easiest. Simply be polite and honest. Generally speaking, auditors are there to identify problems before they affect product quality. They are not out to get you or ruin your career. They are usually the same as most of the people they audit, only more experienced with a broader base of knowledge and training.
Of course, we’ve all seen those that have never calibrated a piece of test equipment in their lives and never intend to. That’s OK, too. If they know how to audit a quality system and its requirements, they usually do a good job.
Never lie to an auditor. But it’s also good policy to never tell them more than they are asking. Only answer the questions posed. Don’t volunteer additional information that may lead an auditor to pursue that information further.
This is how most auditors find out what part or person is throwing a wrench in the quality system. They let the talkative calibration technician tell them what they cannot readily find during the audit. Remember, only reply to their questions without trying to impress them with your great knowledge or bravado. It’s an audit, not a job interview.
Do your part
In most cases in which an auditor is reviewing an entire company and the calibration function is only a small part of the whole, there is usually enough time to hit a few areas and topics.
One of the first questions an auditor should ask is: "May I see your overdue list?" If you have overdue items, have they been segregated so they can’t be used by the customer? If they’re a standard part of production, have they been removed from service to prevent inadvertent use? Have they been tagged with "do not use" labels to show they are out of calibration?
There are times when there are too many items to remove from service and place in a special location. In that case, the test equipment should have calibration labels removed and "do not use" labels attached. This should happen as a matter of course, rather than just during audits.
Remember, audits are chances to see where improvements can be made and to find areas that can be improved upon. They’re a source for making process improvements. If you approach an audit with the right mindset, it can be a great opportunity for all parties involved.
The article is based on subjects found in The Quality Calibration Handbook by Jay L. Bucher, ASQ Quality Press, 2006.
Jay L. Bucher is president of Bucherview Metrology Services LLC in DeForest, WI. He is editor and coauthor of The Metrology Handbook and author of several books, including Paperless Records, Ethics: The Final Frontier and Paperless Calibration Compliance for National and International Standards & Regulations. Bucher is a senior member of ASQ, the chair of the measurement quality division and a certified calibration technician.