Enter to open, tab to navigate, enter to select
Training as a compliance tool: measuring effectiveness
Practical Law UK Articles w-021-6214 (Approx. 7 pages)
Training as a compliance tool: measuring effectiveness
by Alice Southall, Practical Law In-house
| Published on 07 Aug 2019 • ExpandEngland, Northern Ireland, Scotland...United Kingdom, Wales |
This article sets out expert recommendations on how to make compliance training effective, along with some recent statistics on how compliance training is delivered and how different training formats are received.
Compliance breaches regularly make the headlines. Compliance training is given much less attention, despite being a key tool in breach prevention and, in some cases, a mitigating factor when it comes to sanctions for a breach that occurs. In the section of the UK's "Code for Crown Prosecutors" which relates to corporate prosecutions, having a "genuinely proactive and effective corporate compliance programme" is a factor against prosecution of a company that has committed an offence. The existence of such a programme is also a relevant factor for prosecutors considering whether to grant a Deferred Prosecution Agreement (for more information about DPAs, see Deferred prosecution agreements toolkit).
A robust compliance programme also brings positive benefits of its own. A 2017 report titled "Solving the Compliance Conundrum" by learning analysts Towards Maturity summarises these as:
- Managing and mitigating risk and ensuring that staff work within regulatory parameters.
- Improving business performance through standardisation, transparency and keeping pace with change.
- Encouraging a better workplace culture.
While an effective compliance programme is therefore arguably priceless, the costs to an organisation of implementing one can be substantial. Towards Maturity found that, in 2017, organisations put an average of 81% of employees through some form of compliance training each year, at a median price of £14 per employee trained.
As the compliance burden increases, the costs of training are likely to rise. According to its report, most organisations surveyed by Towards Maturity wanted to increase the hours their employees were spending in training, from an average of 10.90 hours per year to 13.54 hours per year.
Given the cost of its delivery – and the price of its failure – how do organisations ensure that the training they are paying for is really doing what it is designed to do, rather than simply ticking a box for their regulators?
What does "effectiveness" look like?
Compliance training's effectiveness is difficult to quantify. Effective training should result in a "nil return" (the absence of a compliance breach), and hopefully some of the positive effects described by Towards Maturity, which are difficult to measure. But year-to-year, in the absence of breaches, how can a training programme's effectiveness be assessed, and even improved?
The UK's "Code for Crown Prosecutors" does not go into detail of what a "proactive and effective corporate compliance programme" looks like.
In the US, however, the Department of Justice Criminal Division has published some guidance on the "Evaluation of Corporate Compliance Programs", designed to assist prosecutors to form a view on the appropriate sanction for a company which has committed some sort of misconduct. This goes into more detail about the features of a compliance programme that could incline a prosecutor to treat a company more or less leniently.
While explicitly not a checklist or formula, the guidance may be helpful as a reference for UK companies, especially those operating internationally, when assessing the structure of their training programmes.
In relation to training, the guidance asks:
- Has the training been offered in the form and language appropriate for the audience?
- Is the training provided online or in-person (or both), and what is the company's rationale for its choice?
- Has the training addressed lessons from prior compliance incidents?
- How has the company measured the effectiveness of the training?
- Have employees been tested on what they have learned?
- How has the company addressed employees who fail all or a portion of the testing?
The guidance is not meant to be prescriptive: it is for a company to devise and implement the system that suits its organisation best, taking into account its own needs and recommendations as to best practice.
Best practice in compliance training
In designing its compliance training to ensure maximum effectiveness, an organisation may look at factors like different methods of measuring that effectiveness; the use of technology in delivering training; its format and frequency. Some views as to best practice in these areas are set out below.
Measuring effectiveness
While the US Department of Justice's Guidance itself does not go into detail on how best to measure effectiveness, it is possible to find some recommendations from (almost) the same source.
The 2017 Department of Justice guidance (updated in April 2019) was based on work done by Hui Chen during her time as a compliance consultant at the US Department of Justice. In 2018, she and Eugene Soltes wrote an article for the Harvard Business Review titled "Why Compliance Programs Fail – and How to Fix Them", which suggests that the answer to an effective compliance programme generally is better measurement. Research they cite found that almost a third of firms do not even try to measure the effectiveness of their compliance programmes, and only a third of those firms that do are confident that they are using the right metrics.
According to Chen and Soltes, when it comes to compliance training, a proper analysis should include assessment of what employees know before training, as well as after, so that the training's impact can be properly understood.
The article goes into the question of metrics in compliance programmes, and common pitfalls, in far greater detail.
Use of technology
Technology-assisted compliance training is popular, not least because it enables organisations to carry out measurement more easily, as well as confirm completion of training to regulators. In its 2017 report, Towards Maturity noted that 72% of the staff who participated in compliance-related training accessed technology to do so and that 42% of all online training was compliance related.
However, it also found that the majority of training technology formats had declined in usage since 2015 – most likely because of the difficulty of ensuring user engagement. This was the factor most frequently cited as a barrier to compliance training – far more frequently cited than the other factors of:
- Dull and boring, or previous bad experience.
- Subject matter experts overloading content.
- Lack of design expertise.
- Lack of credible design materials.
The result was a finding that only 77% of staff complete online training programmes.
The Towards Maturity report is not UK-specific. It analyses results of an international survey of over 250 ethics/compliance and learning professionals. While not focused on compliance training, two CIPD reports provide some UK-sourced statistics that support the contention that compliance training delivered through technology may be less effective than that delivered in other formats.
The CIPD's report on its 2015 Learning and Development survey recorded respondents' views that:
- While E-learning courses were one of the formats whose use was predicted to grow within respondents' organisations, they were one of the least effective methods of training.
- Face-to-face learning formats such as internal knowledge-sharing events were predicted to grow more than mobile device-based learning, massive open online courses (MOOCs) and gamified learning.
A 2017 Employee Outlook survey conducted by the CIPD produced similar results:
- Despite pure online learning (e-learning, virtual classrooms and MOOCs) being the most common type of training received, only 64% of employees who had had such training rated it as 'useful' or 'very useful'.
- Mobile-device-based-learning fared little better, with only 68% of employees giving it that high a rating. Higher ratings went to training formats that either combined technology with other types of training, or cut out technology altogether.
While the first survey was targeted at people in HR and learning & development roles (96% of respondents had a role in one of these departments, or a managerial role), the second was targeted at employees, and so despite the differing perspectives of the survey respondents, the view on this point was similar.
Given the convenience of using technology to deliver training, it is unlikely that an organisation would plan to dramatically reduce its use (although it might look to couple technology-assisted training with face-to-face training), or cut it out altogether. However, it may look to other factors, such as format or frequency of delivery, to improve its effectiveness.
Format
Matt Kelly, editor of the Radical Compliance blog and a former editor at Compliance Week, considers how to make training "stick" in a blog inspired by in-flight safety videos (it is much more entertaining than it sounds, as are the compliance memes that he has put together).
He suggests that, before designing a training course, a compliance officer should consider whether the training should result in employees following a specific procedure (that is, compliance) versus following a certain standard of behaviour (that is, ethics).
This will be determined by the subject matter of the training: is it teaching an employee how to interact with a machine or system; with other people (like foreign government officials); or in certain situations (like seeing someone bully a colleague)?
Once this question has been answered, and the audience for the training is known, a suitable training format can be chosen. An instruction video or podcast might be suitable for teaching employees how to interact with the particular system that they need to use when collecting personal data, for example. However, these formats may be less effective in helping employees decide how to deal with challenges that can manifest themselves in a variety of situations and require empathy and imagination to address, such as witnessing harassment.
Frequency
This Practical Law In-house blog from 2017 considers the importance of supplementing core annual compliance training with additional training in small, specific modules, to avoid falling foul of the "forgetting curve".
Towards Maturity found that using such "spaced learning" was a tactic that was highly correlated with both increasing efficiency of training (in terms of being able to provide regulators with a 'tick in the box') and effectiveness (in terms of building a compliance culture and influencing staff behaviour).
Sources
Employee outlook: Spring 2017 [online]. Survey report. London: Chartered Institute of Personnel and Development. With the permission of the publisher, the Chartered Institute of Personnel and Development (www.cipd.co.uk).
Evaluation of Corporate Compliance Programs, updated April 2019, U.S. Department of Justice Criminal Division.
How Good Training Finds Its Wings, 15 July 2019, Radical Compliance. With the permission of Matt Kelly.
Learning and development 2015 [online]. Survey report. London: Chartered Institute of Personnel and Development. With the permission of the publisher, the Chartered Institute of Personnel and Development (www.cipd.co.uk).
Solving the Compliance Conundrum, In-Focus Report, June 2017, Towards Maturity (authors Genny Dixon and Laura Overton). With the permission of Towards Maturity (www.towardsmaturity.org).
Why Compliance Programs Fail – and How to Fix Them, 1 March 2018, Hui Chen and Eugene Soltes, Harvard Business Review. With the permission of Harvard Business Publishing.