by Steve Gompertz, QRx Partners
While presenting workshops recently, the topic of training effectiveness kept coming up. In both instances, quizzes were mentioned in passing while discussing other aspects of training employees. I surprised both audiences by stating that quizzes aren’t really value-added when assessing training effectiveness, or employee competency for that matter.
As usual, neither the FDA regulations nor ISO 13485 are specific about how to establish training effectiveness. FDA only requires that the Manufacturer ensure that employees are trained to “adequately” perform their responsibilities. ISO 13485 requires that organizations evaluate the effectiveness of the actions taken to establish and maintain competence.
A hint is provided by 13485 in that clause 6.2(b) refers to providing training or taking other actions relative to competence. If training isn’t the only option, then quizzes aren’t the intended method. So, what’s wrong with relying on quizzes? And, while we’re at it, let’s look at the other common methods for confirming competence, such as observation, inspection, and “certification”.
In a nutshell, these methods only provide a limited snapshot of employee performance or competence.A quiz delivered immediately after training only confirms that the employee was paying attention.If delivered a week or two later, it only confirms retention of the training material.Observation of the employee performing the work is faulty because they know they’re being observed with a potential for the observer to bias the results positively or negatively.Inspection of the work performed, which is often used over an initial period of time to establish “certification”, again can be biased by knowledge of being inspected.
Overall, these methods fall short because they only establish competence during a point in or limited period of time. What’s missing is continued assurance of applying the training.
The whole point of ensuring that competent personnel perform the activities of Quality Management System (QMS) processes, is to ensure that the processes are effective. So, are we supposed to measure the people or the processes? As with all things QMS, it’s about the processes. Having the right process monitoring in place will tell you whether the process is effective. If it is, and since the people performing the process activities are critical to the process’ effectiveness, then you can assume that the people are competently performing their responsibilities, i.e. – if they weren’t, then the process metrics would show it.
You can also think of it in terms of verification versus validation.Effectiveness is determined by validation, while completion and correctness are determined by verification.Similar to how Design Verification only confirms that you did what you planned to do and that it works, artifacts like attendance records, quizzes, and observation checklists therefore only serve to confirm that training was completed and remembered. Process monitoring and measuring are where validation will be achieved and maintained, showing that the training continues to be applied and is effective in realizing an effective process.
If you’re struggling with how to effectively demonstrate training effectiveness and employee competency, or how to establish appropriate process monitoring, QRx Partners can help. Contact us at Contact@QRxPartners.com or 833-779-7278.
Comments