Security training is often considered a bit of a waste of time. Maybe this is unfair, but unsurprising in the face of the worst forms of training like flicking through the computer based training equivalent of a slide show or even the ritualized gotcha of the phishing test.
But, training our employees, vendors and customers on important topics to help them protect themselves is important. Even the correct strategy of creating ambient controls so that people are intrinsically protected by the platforms they use still needs some awareness training about the importance to cooperate with, or at least not actively resist, those controls.
I’ve worked in many companies and have also seen what 100’s of other companies do. I’ve been subject to much poor training but also some fantastic approaches over the years. I’ve had the privilege of working with some great people, teams and vendors who’ve done this really well. So, rather than present a critique of terrible security training, I will instead review some of the leading practices I see working in many organizations every day.
1.Computer Based Training (if you have to do it then do it better)
Some organizations are compelled to do regular formal training on a range of topics from privacy, security, compliance, through to resilience and crisis management. Despite more effective training approaches being available, there is often a legal, regulatory or contractual need to do the type of training where people sit through the on-line equivalent of flicking through a PowerPoint deck and then answering some questions and clicking an “I certify I’ll be good” checkbox. I’ve worked in regulated environments for most of my career whether energy, defense, finance or tech and so I have been subject to repeated anti-money laundering, anti-corruption, security and privacy training many times a year for decades. Amusingly, even when I ran a risk function overseeing parts of financial crime I still (for regulatory reasons) had to do the basic education and certification year upon year. Incidentally, and sadly, I saw one organization have to weaken their desktop security controls so the third party delivered security training modules (required by a regulator) could actually run.
So, what to do if you can’t escape this form of training:
Push Back Anyway. Sometimes the interpretation of the need for this specific approach to training is not quite as rigid as people think. There are plenty of ways to adjust scope, to appeal to regulators to satisfy the requirement in the spirit if not the letter of a potentially outdated rule. You can also reduce the scope of who has to do this, especially excluding people who in their roles already have professional certification hours and codes of conduct to attest to their knowledge.
Pre-test. Many requirements to do this training can be achieved, and the effort alleviated, by having people who have done the training before do a pre-test to show their knowledge is current without having to re-step through the training.
Certify Don’t Quiz. Sometimes it’s ok to simply certify you understand the training without a convoluted set of test questions. Although, test questions, can be effective in solidifying the knowledge of people who are genuinely new to the topic, so you should apply this carefully.
2.Ambient Controls (Solutions Not Just Policies)
It is a design failure if you are training people because they have to be part of the control framework of a system or have to act like a, so-called, “human firewall”.
A better approach is to keep implementing ambient controls - to look for the underlying security issue which needs solving - such that significant employee training isn’t needed.
Some progressive organizations have gone through all their training material (literally 100’s of modules, 1000’s of equivalent pages of policies and content) and looked at each element and asked: What failure of ambient control exists such that the training (or even policy) is needed? One organization eliminated over 50% of policies directed at people and consequently reduced training hours and content. In doing so they significantly improved control effectiveness by eliminating what was previously called “human error”, where people were repeatedly forced to be re-trained in the face of dismal processes.
To give one current example affecting many organizations, it is better to invest in implementing multi-party approval controls so a CEO or CFO can’t singularly authorize a multi-million dollar cash transfer in an email - rather than train and test all employees on business email compromises as the only line of defense. The same goes for moving away from passwords to phishing resistant authentication tokens.
3.Explain the Purpose of Controls
Even with this strategy of creating ambient controls, you will still want to provide some education about the controls and the reasons for their existence so people cooperate with them. This is also to ensure that if the controls do impede legitimate activity that there is a path for people to raise that, rather than try to work around the controls and create risk.
Explainability. Controls should always include some explanation or other user experience aid. This is especially the case with pop-up warnings when people attempt a prohibited activity or push up against a guard rail. For example, many organizations restrict certain websites for security reasons. If you as an unfamiliar user, or even just by mistake, hit one of these and the web proxy flashes up a block page that says something like, “SECURITY VIOLATION - YOU ARE GOING TO A WEB PAGE THAT IS BLOCKED YOU BAD PERSON - WE ARE WATCHING!!!” then you’re likely to immediately perceive the security team as an enemy. Whereas if the block messages says something like, “Hi, you probably didn’t know or did this by mistake, but the web site you’re going to is blocked because of [give some reasons]. To do this type of activity go to this [internal web site] where some other tools exist and if you need to understand this more then look at [this page]. If this is wrong or you have other questions please call or email the security team [here].”
Escalation Path. Building on this, all controls in some way should provide a path to raise concerns so people feel like they’re an active part of the control environment. With this in place I’ve found a number of times that the pressure to violate controls was because of external factors, sometimes from surprising places, rather than employee unwillingness. For example, in one financial organization I worked at we were very strict on portable media controls (USB Drives, CD/DVD, other media) and almost no-one had media permissions. Interestingly one of the biggest source of issues with this policy was various regional tax authorities wanting sensitive data filed by CD/DVD or other portable media. We had to comply but the escalation path got the security team in the loop early on this control incompatibility and we were able to engineer (over time) with those authorities a better (secure file transfer) approach.
Educate with Incidents. When a new control is implemented, explain the rationale with a horror story - possibly one within your own organization - and then people will know why the control exists or is coming.
Thank People. Keep a broad base of support for controls by using incidents at other organizations as case studies of why it can’t happen to you because of the end user cooperation with the control. For example, “Well done everyone for supporting the control in our organization that made [bad thing] that happened to [big competitor] not happen to us. Thank you.” I remember at one organization we had a spate of data leakage events due to e-mail address auto-complete issues that, thankfully, were more embarrassing than serious because of DLP defense in depth. We turned off auto-complete (this was years before more nuanced controls and warnings were usable) which wasn’t popular but in our risk calculation was necessary. What turned the perception around that this was only an irritating control, fortuitously, was about 1 month after our implementation, a big competitor and also, ironically, a large regulator had significant events due to the same issues. We were able to communicate this and all of a sudden people were not only more accepting of the control but actually self-congratulatory - and also we communicated the usability and other nuances that were coming to remove the irritation. We had similar experiences with removing desktop admin rights, keeping USB drives disabled and many other controls. If you can’t make controls fully seamless and transparent then moving end users from victims to willing heroes is useful.
Focus on New People. New hires from other organizations can be a constant control challenge especially if your organization is better controlled. Naturally, you can cover this in new hire orientation but another good way is to educate your help desks and IT support teams on how to explain the controls with case studies in their support scripts. Also, give the help desk a fast path to the senior people in the security team. In a prior role I made myself (as CISO) the point of contact for escalations for new senior hires who were hassling for things that were against our policies but had been “tolerated” by their prior organizations. Again, thanking people for their cooperation in advance to help protect the organization and our customers was more effective than strict policy pushes.
Day 100 vs. Day 1. Another opportunity to explain controls and for any required attestations is to eliminate a lot of content from new hire orientation on people’s first day. Instead, send people things gradually over their first 100 days and in the period after offer acceptance before Day 1. In one organization we also sent people a Day 100 e-mail with a bunch of reminders and fresh information. This was more effective because at Day 100 most people know what they’re doing, have experienced their workflows in depth and are more receptive to education.
4.Risk Culture / Escalation
Most security training is about 2 things, (1) Changing risk culture, and (2) Encouraging the escalation of new or latent risk issues so that they can be fixed (or subject to some other appropriate risk treatment). A lot of this training can be replaced by some “organizational engineering” that might include:
Embedding Security. The old cliche is true, security is everyone’s responsibility. Like other attributes of good security it's important that this isn’t a throwaway line but is actually consciously implemented. For example, hand-off into SRE, DevOps, development and other teams and support them by developing tools and processes to make this happen.
Explicitly Assign Roles. Designate specific roles in teams to champion security and other risks. These don't have to be deep security experts but they should, of course, have a broad understanding of security to be able to look for tooling, product and other potential improvements. The key skill is knowledge of their specific area and a sense of how to partner with the central security team.
Escalation. Put in place processes that drive the cultural change to encourage escalation and ensure your risk committees and other governance is set up to support this. One of the best examples of a practice to change culture I saw was a “Go flight” process to get people to open up and raise concerns. Imagine you’re sitting around a virtual or physical conference table as a group representing various functions. You are all about to approve a major decision or product launch. The boss says to everyone, “Are we all ok with going ahead with this vital and important project?” In a lot of organizations few people, even if they have concerns, are going to feel comfortable saying something. Yes, a few might but only because of years of conditioning. Now imagine instead, there is something similar to the NASA “Go flight” launch protocols where each person in the room is asked in turn, are you good to go? Even better if the leader expresses this as “What concerns do you have - everyone has to raise at least one - and do they merit pause?”
5.Gamification, Labs and Ranges
We’ve talked about reducing the need for broader training with more ambient controls. However, there is still a need for more directed role-specific training especially for security and operations teams. Training by simulation through virtual labs, cyber-ranges or live-fire training using attack simulation tools are very useful. The latter can also be used to test the correct operation of your sensory apparatus.
Attack simulation tools are just a special case of a long-standing technique of synthetic transaction generation that has been widely used in finance and other fields to inject specially constructed test events, in safe ways, into processes to test edge cases and the operational processes to deal with them. Such synthetic event generation, therefore, can be an even broader approach than what attack simulation tools might cover, especially in the context of business controls and guards around the behavior of AI tools and agents.
For labs and ranges it is useful to make the league tables of results transparent. This is not just to encourage competitive behavior but it also helps with discovering latent talent. Some of your best security people might not actually yet be in the security team. Some of the people who work their way up the leaderboards in tools like Immersive Labs, come from help desks, development or other engineering teams and these might be your next best core cybersecurity team members. Other latent talent in your organization can be similarly discovered by internal hackathons, open / gamified training environments, or simply keeping an eye out for the highly engaged people across the organization.
6.Tooling and User Experience Integration
The recurring theme of ambient control applies especially when looking at the tools and workflow people are using in their day to day roles. This can apply to developers (IDE integration, CI/CD pipelines, frameworks that avoid common security vulnerabilities, and well architected design patterns that reduce structural security risk), DevOps (infrastructure as code, infrastructure immutability patterns, multi-party access controls, centralized observability) and security (automated, if not autonomic, security operations tooling and practices).
It is vital to look at the design of controls in business processes and workflows. Look at consistency in user interfaces and design cues on data entry/validation to provide warnings of anomalous inputs. Watch out in taking this approach as it will drive a need for many small changes to systems. It could be 100’s of micro changes to screens, workflows, and UIs get stuck because most development and product teams are already overloaded with big work. So, for this and other purposes, reserve some development and engineering budget, say 5%, to only apply on small improvements in process, usability and other system deficiencies.
7.Drills, Exercises and Incident Learning
If the goal of security training is to sustain or improve security then implementing processes to routinely do this as part of the organizational culture is way more important than less effective episodic training or awareness communications. Such a culture of pervasive self-improvement (in the face of one’s own issues or learning from others' issues) comes not from education about how to have a better a culture, rather, it comes from actions that over time create that culture.
Some of the better actions to do this are drills, exercises and rigorous learning from incidents or close-calls that you or others have:
Incident Learning. This can be (blameless) post-mortems of your own incidents / close calls or looking at those of others. Incidentally, one of the major benefits of ISAC membership is to formally or informally see more incidents to study. When looking at these then make sure to do the 5 Y’s to really get to the root cause. For example, there was a breach because of a vulnerability in an application server. (1) Why? Because the server wasn’t patched. (2) Why? Because the application software wouldn’t work on the upgraded server software. (3) Why? Because the application hadn’t been updated in 2 years. (4) Why? Because the team that worked on this had been allocated to work on other priorities. (5) Why? Because no-one identified in the budget the need for preventative maintenance on that critical application. Tactical answer: see if you can deprecate the app and hence the server, if not then swing resources over to upgrade the app or determine another way to enable the server to be patched. Strategic answer: examine how applications, including this one, are identified in the application inventory as being critical and how maintenance budget is, therefore, sustained to reduce the risk of recurrence of such a deadlock.
Broaden Scope. When looking to learn from incidents make sure you are capturing not only the close-calls (aka near misses) that didn’t happen but nearly did, but the close-calls in terms of impact. Plenty of incidents happen but are written-off as uninteresting because they had little impact. But, how much of that was luck rather than planning? For example, many organizations have financial transaction incident loss reporting to be above some figure like $10k. This makes sense as you want to spend time looking at “big” incidents. But, how many of the incidents of this type are below the $10k because of luck. What if the loss was $5k just because of the happenstance that the breakage happened for an unusually low transaction and it could have just as well happened for something at $10M, in which case you really should look at the $5k loss and view it as a blessing that you got to remediate an identified risk for such a small trigger.
Drills and Exercises. The best training of all is a drill, exercise or even a live-fire event. Having drills and exercises that get as close to reality as possible and test your people as well as your systems is ideal. The downside of many organization’s approaches to drills and exercises is they are big and so take time to plan and prepare - which is sort of against the point. One of the better ways to counter this is to have lots of “micro-drills” as well as a smaller number of regular drills. Such micro-drills could be 15 minutes to, for example, test a means of back-up communications, exercise a call tree, or slightly longer like working out of a recovery site for 1 day a month. Micro-drills can be a bit more draconian, I knew one organization where leaders would occasionally walk the floors and tap people on shoulders and ask them where they would go in a disaster. Of course, this only happened once or twice before everyone made sure they knew the answer in advance.
8.Workforce Development and Feedback at Point of Need
Another means of improving organizational security competence is through the wider people management processes. Not creating new and separate security training, education and awareness but, rather, slip-streaming this into existing processes. In workforce development this can include:
Performance Feedback and Promotions. Make security specifically, or risk management more generally, a factor in how people’s performance is assessed and how different levels of promotion readiness are evidenced. Integrate specific capabilities into role and level descriptions and make sure this is consistent and fair with assessment rubrics. Depending on the organization it might be better to get more specific than just assessing “security” and instead be more explicit on elements that lead to good security such as maintenance, reliability and controls.
Community Development. Many organizations are challenged because the communities they hire from (technical, business, or otherwise) are insufficiently trained on security, or even just the importance of security. So, when they come into the organization they need more focus on these topics at boot camps and orientation or suffer an impedance mismatch when they land in a particular team. One of the better places to focus on more traditional security training is in these boot camps. But, looking at the broader problem you can seek to develop the wider community. A lot of organizations, including my own, have worked hard to create career certificates, University research and course development all the way through to improvements in professional accreditation and certification. There’s more to be done on wider professional integration, for example, the PCAST report I worked on has called for cyber-physical resilience topics to be worked into ABET accredited engineering qualifications so Professional Engineers understand the cyber-physical dimensions to what they are designing or certifying.
Team Capability Development. We continuously hear about the millions of unfilled cybersecurity roles, although I’ve yet to see a study that actually supports that near-constant claim. From this we are driven to believe the only answer to this problem is to create millions more cybersecurity professionals through a constant grind of training and skills development. We’ve been hearing this for years and despite some terrific work all round we don’t appear to have made more than a dent in the problem. I do, of course, agree we need more trained cybersecurity professionals, but this singular focus neglects the range of problems which become clearer when you look at this as a supply and demand problem. If we need 10x more cybersecurity people to fill all those roles perhaps we should 10x the productivity of the people we already have.
In Process Feedback. Provide people with the means to learn more when they hit a guard rail or hard rail . This should be multi-level. For example, you’ve just tried to install some software and hit a download or installation block due to insufficient privilege. One error message could be “Unauthorized Action - This Has Been Logged”, a better message would be “We do not permit the external download of software. Please go to our internal App Store [here] or chat [here] with the help desk or watch this 60 second explainer video.”
Event Specific Feedback. When a new project or program is launching, or a new event (e.g. M&A) commences then people are primed and focused on being the best they can be on the new thing. This is a great time for some quick / essential reminders on event specific policies, like M&A confidentiality, the use of the correct document sharing tools, and so on.
Leader Delivery. Security messaging delivered by the security team is unlikely to be highly rated because there’s no “surprise”. Any time security turns up to some event to brief about risks there’s usually a bit of a collective eye-roll. But, if the business head delivers that message with a bit of personal context like, “I used to think [insert your desired security control] was overly burdensome bureaucracy, but I tell you I got burned a few times personally, and I know our friends in security have worked hard to make this control highly usable so I ask us all as business professionals to pay attention to this - it’s core to our success.” A leader delivering that message at a sales leadership meeting, a product design meeting, or any other context is powerful.
Checklists. Many organizations use checklists. Where you have this culture and practice then don’t create your own, simply integrate the appropriate security steps into existing lists. When I’ve done this before I’ve found the checklist "guardians" push back a bit as they want to keep their checklists brief and very focused. This can be annoying until you realize that, mostly, they are right and you need their help to keep the security additions to be the necessary minimum so as to make them workable.
9.Training at Point of Maximum Receptivity
Train people at the point of maximum receptivity where they are feeling anxious (in a positive way) about some new situation. This could be:
Promotions. When people are promoted, and go through some next level induction training they are primed to be good and responsible corporate citizens. This is a moment to reinforce expectations. I’ve seen many organizations do this, and are careful to use it selectively, for the right type of reminders. This could be crisis management responsibilities, tone at the top reminders and how to exhibit the right behaviors all the way through to bringing them in the know on some previously sensitive incidents they should learn from.
Role Changes. The same applies for many types of role changes, whether or not it is a promotion. Such changes also give the opportunity for role specific messaging.
Team Restructuring. Likewise, team restructuring provides an opportunity for training reminders, tool refreshes and such.
Exits. Finally, and I see many organizations do this well, is to remind people of their roles and responsibilities on confidentiality as part of their departure from the organization.
10.Nudges
There is an additional layer of ambient control that can be designed into an environment through the use of nudge theory. The use of these behavioral psychology and economics techniques was all the rage a few years ago but has been somewhat discredited with some major pieces of research being caught up in questions of reproducibility. But, I do see many organizations continue to measurably benefit from these. In fact some of the approaches are sufficiently successful that they’re claimed to be obvious (in hindsight). Two examples I’ve seen personally were (1) solving a clear desk policy cooperation issue (in a high confidentiality environment) by putting in more secure disposal bins at the end of every desk aisle vs. a central floor location that required people to walk even for less than a minute. And, (2) in a risk assessment moving the end sign-off (“I assert I have represented the facts”) to an up front assertion (“I will represent all the facts”) and seeing an increase in the number of self-raised risk issues.
Bottom line: ambient controls should replace the need for much security training and what remains should be a function of the processes people operate within, not something they are dragged to. Concepts should be reinforced at key moments when people are maximally receptive - either at a moment when they’ve hit a guard rail, hard rail, made an error, or when they are changing roles or being promoted.
Comments