We now know that the best way to instill people with a healthy degree of "inbox suspicion" is to flood them with harmless but realistic-looking phishing messages. People who click the fake phishing messages are tracked and automatically educated with video-based training and exercises, thereby reducing the chances they will get fooled by real phishing messages.

Today, you can purchase "phishing simulation" services like this from a variety of vendors (including SecurityIQ from the InfoSec Institute), but how do you know if you are getting a good deal, if your investment is sound, or how your phishing simulation is performing against industry norms?

Strengthen security awareness with human risk management

Infosec HRM, powered by Right-Hand Cybersecurity, provides alert-based training nudges to minimize human risk at your organization. 

Demo Now

Fortunately, you can pull a variety of hard numbers from your phishing simulator to calculate and compare traditional business KPIs like return on investment (ROI) and emerging KPIs like "open rates." Today, we will learn how to calculate the ROI of a phishing simulation and a follow-up article will look at other effectiveness KPIs.

Calculating ROI of Phishing Simulation

Introduction to ROI

"Return on Investment" or ROI is the one calculation all managers depend on. ROI figures calculated on multiple projects let managers rapidly determine which projects provide the "best bang for the buck", and detailed analysis of each project's ROI calculations can help managers understand the risks involved in achieving the goals of each project.

As you probably already know, ROI is calculated by dividing a project's "return" by its "investment".

ROI = Return / Investment

"Investment" is, of course, the project cost. Project "return" is itself calculated by subtracting the value of a project's gains from its initial cost. This subtraction is performed so a project with an ROI of 0% indicates a project with no return after costs are accounted for - in other words a bad project!

ROI = (Gain_Project - Cost_Project) / Cost_Project

Calculating the Value of Phishing Simulation (Gain)

Figuring out the cost of a phishing simulation project is fairly easy. In fact, many phishing simulators use an "all in" software-as-a-service cost to make the calculation of this figure particularly easy. But how do you calculate the value of better-trained users?

The answer lies in expected loss-due-to-phishing and improvement figures. Since phishing simulation will be reducing your organization's expected losses due to phishing attacks, you can show that the value of your phishing simulation is the amount that it will reduce your organization's expected losses.

To calculate your organization's expected losses to phishing, you can adopt one of the industry's well-known figures, such as:

• $3.7M per year for an average 10K+ employee organization [1]
• $1.6M per year for an average 1K+ employee organization in a regulated environment [2]
• Approximately $2M for successful phishing attack, regardless of company size [3]

Keep in mind that these figures typically reflect organizations with limited security awareness training programs, and rarely reflect organizations with active anti-phishing programs (since cost-per-attack research is often conducted by phishing simulation firms after attacks on wide-open organizations). Now apply one or more figures to your organization, depending on your circumstances. For example:

Loss_{ExpectedDueToPhishing} = $1.6M * 300 employees / 1000 employees = $480,000 / year

Once you have a phishing-loss estimate, you can move on to expected improvement. The best number to concentrate on here is the "click rate", which tracks what percentage of end users are expected to click on a phishing message during the year. Industry click rate figures range from 12% [4] to 40% [5] so a beginning number of 20-25% is not unreasonable.

More importantly, you must also estimate the expected improvement of the phishing simulation. This can range from a pessimistic, academic-backed estimate of 30% [6] through another academic-backed estimate of 60% [7] and on to wild-eyed vendor estimates of 80-90% or more.

Improvement_{DueToPhishingSimulation =} 50%

Once you have all these figures, you can use them to calculate a gain value based on reduced expected phishing losses.

Gain_Project = Loss_{ExpectedDueToPhishing} X Improvement_{DueToPhishingSimulation}

For example:

Gain_Project = $480,000 / year X 50% = $240,000 / year

Calculating the ROI of Phishing Simulation

Once you have calculated the value of the gain you expect to realize by avoiding phishing attacks in your organization, you simply have to plug in the cost of the solution that will provide the phishing simulation and training to calculate its ROI.

ROI = (Gain_Project - Cost_Project) / Cost_Project

For example (with a $40,000/year solution yielding $240,000/year of protection):

ROI = ($240K/yr - $40K/yr)/$40K/yr = 500%

While it's unlikely that an anti-phishing solution for 300 users would really cost $40,000 a year, it's easy to see how and why so many phishing simulator projects get approved due to their high ROI's compared to other (less valuable) security projects!

In the second part of this article, we will show how you can demonstrate the value (or lack of value) of your existing phishing simulator.

See Infosec IQ in action

From gamified security awareness to award-winning training, phishing simulations, culture assessments and more, we want to show you what makes Infosec IQ an industry leader.

Demo Now

References:

1. http://www.csoonline.com/article/2975807/cyber-attacks-espionage/phishing-is-a-37-million-annual-cost-for-average-large-company.html
2. http://www.hipaajournal.com/cost-of-spear-phishing-8268/
3. Multiple Sources: http://www.coindesk.com/bitpay-sues-insurer-after-losing-1-8-million-in-phishing-attack/ , http://www.hotforsecurity.com/blog/top-5-corporate-losses-due-to-hacking-1820.html , etc.
4. Verizon 2016 Data Breach Report
5. http://lorrie.cranor.org/pubs/pap1162-sheng.pdf
6. http://lorrie.cranor.org/pubs/pap1162-sheng.pdf
7. https://www.cs.cmu.edu/~ponguru/eCrime_APWG_08.pdf