CISO ‘How to’ Without the Bull: Phishing Simulations

Welcome to the second of my ‘no bullshit cyber’ blogs, with useful tips for putting cybersecurity into practice. My name is Nick Harris, and I’m the CISO in Residence at Assured.   

Let’s talk about phishing simulations. 

It’s a marmite conversation in the industry. Are phishing simulations right or wrong? The argument against it refers wisely to the anxiety some people feel and the perception that phishing simulations are the security team trying to catch people out. This will destroy trust and discourage employees from coming forward with issues they are facing and questions they have.  

Noting Ebbinghaus’ Forgetting Curve (which suggests that little is retained after seven days), there needs to be a method of validating that cybersecurity awareness training is effective. The military spends large proportions of their time undertaking exercises under different degrees of stress and varied scenarios so they can put their training into practice, be well prepared for all eventualities they may face and ultimately remember it. I am in favour of applying the same logic to phishing simulations to reinforce learning and validate the training is effective.  

Randomise it 

I do not agree with a one-off phishing simulation, bespoke-built at a significant cost that hits the workforce for the sake of compliance. Nothing can be learnt from this, and the variables are so limited that there is no way to measure behaviour change.  

The only way to deliver phishing simulations is with a randomly selected set of templates (from a library) at a different time of the day to other employees. Therefore, assuming this is done monthly, no one will get a phish that’s the same as another person or the same as a previous month, and no one will get a phish at the same time as another employee.  

With all these random data points, the type of phishing template or the time of the day is no longer a factor, and we can now measure behaviours more accurately. Microsoft’s Attack Simulator misses the mark here and chooses a random time of day and random template, but sends the same template to everyone at the same time. 

Nonetheless, many platforms offer this capability, and staff can be auto-enrolled via Active Directory/Entra, so you shouldn’t have to lift a finger once it’s up and running. 

Prepare the ground 

Before any phishing simulation, the security team should prepare the way as much as possible. This can take the form of a company-wide roadshow: explaining the programme and providing training on what to look for at company town halls or all-hands, awareness webinars, employee induction sessions, generating poster campaigns, and sending advisory emails – and any other media your internal comms team helps you use. It’s important to warn everyone it’s coming, explain why you are doing it (to check everyone’s skill in being the strongest line of defence), and reassure them that your intention is not to catch people out and that you won’t penalise anyone who accidentally clicks. We all make mistakes.  

“Use specific modules where a gamified and interactive style will be far more effective than a PowerPoint briefing”

Take the opportunity to train employees on how to spot a phishing attempt and how to report it. Use specific modules where a gamified and interactive style will be far more effective than a PowerPoint briefing. In your roadshow, use recent, real examples applicable to the department you are briefing (e.g., attempts to change payroll bank details to HR, invoice fraud attempts to finance). 

Ensure a phishing alert button is rolled out in your email clients and integrated into the phishing simulation service. You must tell the employee they spotted a simulation and it’s not real; you don’t want the SOC responding to simulations. 

Fire evacuation 

Nathan Collings, an amazing practitioner in the industry, talks about how fire drill evacuations are done to exercise training but, even in practice, there are emergency exit signs everywhere, illuminating strips on the floor, and fire marshals to guide you. So why do cybersecurity teams send phishing simulations without this type of support? Even having prepared the ground, this approach undermines trust.  

The first few phish templates should be so obvious that they include clear messaging: “This is a simulated phish,” and offer clear instructions on reporting it in the same email. If training has been forgotten or was missed, this helps skill people and generates muscle memory for using your phish report button. 

With time, you can reduce the simulated phishing flags until they are gone entirely, at least for existing employees. New employees will need to start from the beginning to gain the initial skills. 

No penalty 

In my practice as a CISO, clicking on a simulated phish should never be penalised. This includes discipline, informing line management, calling out mistakes, or even auto-enrolling the clicker in training.  

“In my practice as a CISO, clicking on a simulated phish should never be penalised”

If someone repeatedly clicks, it’s likely an issue with the training delivery, which the cybersecurity team is responsible for resolving. The key is to find out why the training is not landing. In my experience, phishing delivered in a language other than an employee’s primary language is far more likely to be clicked on due to the nuances of the language. 

When someone clicks on a phishing simulation and/or enters their credentials into the fake site, I prefer the method of redirecting them to a red flag page with a message like “Don’t worry; we’re all human” and pointers to what to look out for next time. When someone clicks repeatedly, I spend time with them, build rapport, and coach them through a few examples. After you help them, they’ll be on your side. 

Click vs speed 

Measuring good performance is about measuring behaviour change. With repeated, random phish simulations, it should be easy to see the drop in the rate of phish clicks and the increase in the rate of phishes being reported. Don’t be afraid to shout out kudos to high performers. It should also be possible to track the speed of reported phishing simulations. We know that the faster they are reported to the SOC, the faster a real phish can be analysed and nuked from mailboxes before attachments are downloaded or credentials are entered into the harvesting site.  

If numbers stagnate or start to regress, revisit the engagement you took when ‘preparing the ground’ and reinforce positive practices with gamification. 

Gamify 

No penalties mean no negative messaging, but use positive reinforcement where you can. Call out top performers each month, individually or by department. This engagement reinforces the positivity of good behaviour, shines a spotlight on phishing, and presents cybersecurity in a great light. 

Explain the value 

Explaining the value of phishing simulations is vital. It justifies that the cybersecurity team is more than just a cost centre and can generate value. Use language that demonstrates the value of cybersecurity in a way that the business understands.  

Don’t say: “I’ve sent 847 phishing simulations over the last few months.” Instead, demonstrate the value by saying: “I have reduced the chances of an attacker defrauding us or gaining access to our systems by 34%.” 

Don’t point out that you’ve delivered six webinars and completed three phishing briefs for different teams. Instead, share that you’ve achieved an outreach to 95% of your workforce and made them more aware of how they can keep the business safe and secure.  

Wrap-up 

With any luck, you’ve taken some tips and tricks from here to deliver an effective phishing simulation campaign. I’d love to hear your thoughts on what has worked and where you have tried different equally valuable approaches, so do get in touch.