The ROI of Security Awareness Training

Chris Sienko: Welcome to another episode of the Cyber Work with Infosec podcast: the weekly podcast in which I talk with a variety of industry throught leaders to discuss the latest cybersecurity trends, how those trends are affecting the work of infosec professionals, as well as tips for those trying to break in or move up the ladder in the cybersecurity industry. Today's podcast episode is the audio version of a webinar we released on September 18th entitled 'The ROI of Security Awareness Training'. Hopefully the necessity of security awareness training for your employees is beyond question, but how do the benefits stack up against the costs and time requirements? To find out we'll be speaking to Michael Osterman, Principal Analyst at Osterman Research, along with Infosec's Chief Evangelist Lisa Plaggemier, about a new study conducted by Osterman Research. The study, using data from 230 organizations, will answer these questions and help you to quantify the return on investment for security awareness training in both small and large organizations. During the course of this webinar, Michael and Lisa will discuss how to calculate security awareness ROI at your organization, the opportunity cost of not having an awareness program and costs and returns of security awareness training. Now I'll send you over to the livestream webinar featuring Michael Osterman, Lisa Plaggemier and moderator Hunter Reed entitled 'The ROI of Security Awareness Training'.

Hunter Reed: Thanks for joining us on today's webinar, The ROI of Security Awareness Training. My name is Hunter Reed and I'll be helping moderate today's webinar. Today's speakers are Michael Osterman and Lisa Plaggemier. Michael is the principal analyst of Osterman Research, founded in 2001. Since that time the company has been one of the leading analyst firms in the messaging and collaboration space, providing research, analysis, white papers and other services to companies like Dell, Trend Micro, Convolt, RSA, Symantec, Malwarebytes and many others. Also speaking today is Lisa Plaggemier, Chief Evangelist at Infosec. Previously director of security culture, risk and climate advocacy for CDK Global, her career started in marketing with Ford Motor Company. Lisa has a track record of teams defying security to engage and empower employees to better protect their organizations. Before Michael and Lisa dive into research behind the ROI of security awareness training, I want to encourage you guys to stick around to the end of the webinar, we're gonna be discussing our new ROI calculator that you can use to leverage the research presented today to measure the costs and benefits of security awareness training at your organization. Now I'll turn it over to Michael first for some additional background on Osterman Research.

Michael Osterman: Well, Hunter, thank you very much and I'd like to thank everybody who's here with us today. I think we will be providing you with some very interesting information on the security awareness training market and maybe a way of thinking about the finances, the quantitative aspects of security awareness training that maybe you hadn't considered before. So just a little bit about Osterman Research. As Hunter mentioned we're an independent market research and consulting firm. Our focus really is on the way that people communicate and collaborate in the workplace. So we focus very heavily on primary research, really trying to find out the problems that organizations are having around security, you know how they manage information and so forth. The company was founded about 18 years ago and we're located in Western Washington. So let's get into the nuts and bolts of this. Why are organizations doing training? You know what is really motivating them? This came from a very recent survey that we did as a, really it's actually still an ongoing survey, but these are the results from the past week or so. What we've found is that the vast majority of organizations are implementing security awareness training really as a preventative measure. They see the problems with security, they hear about data breaches, they see malware infections and ransomware infections, and so forth. A lot of organizations really are understanding that they need to train their employees. What's really interesting about this market is that if you go back three or four years the preponderance of IT people said, no, this is a technology based problem, we will throw technology at the problem in the Cloud, on-premises and so forth, and that will solve security. What they found is that that hasn't been the case, they did address security very well from a technology perspective, putting in things like secure email gateway, secure web gateways, firewalls, lots of different technologies, but they didn't traditionally address the problem of the people, the people who would click on a phishing link, the people who would open an attachment from an unknown source. In the last few years we've seen a lot of organizations understand that you need to focus on the people because they really are the weakest link in many organizations. You can have all the technology you want, but something eventually is gonna get in to somebody's inbox and they're gonna click on it and so you do have to train people. So the vast majority of decision makers now really do understand that. They understand that they have to be able to prevent these kinds of things from happening at the personal level. About one out of five organizations we found in the survey are looking at things from a preventative measure, but also because they've been burned, they've had a data breach in the past, so they want to prevent future data breaches and we found that for a very small percentage it's really because they have been burned as a result of a data breach and they've had to take measures, really to train people to understand things better and really not be so much of a weak link. So one of the things we found in the survey is that there really are different types of training provided. For example, if we can go to the next slide? There we go. We find that it really runs the gambit, from those who do absolutely no security awareness training, at all, which is about one out of 20 organizations to those who do fairly comprehensive testing. They'll test everybody in the organization so that they can determine where the weak links are. They'll want to understand who is more prone to phishing attacks than others and so forth. Some organizations will preselect certain employees, maybe only senior executives, or maybe those who handle the most sensitive data assets, then send simulated phishing attacks to those individuals. We have really different types of training here. Employees, they'll view short, security awareness training videos, we have organizations that will take sort of a break room approach and gather people for a lunch meeting or a special meeting to discuss security, maybe once a month or once a quarter, something like that. So there really are different set of training requirements out there and a different set of training approaches. Certainly not everybody that's doing training, is doing training well, a lot of organizations are doing only periodic training, and so forth. As we'll see in the next slide the frequency of training varies quite a bit. We find it again, you know some organizations will do virtually no training, they will never tell employees about security awareness training or how to protect themselves from phishing and so forth. We have others that are doing it really on a fairly frequent basis, so more than six time per year. One of the things that we did in the survey is that we broke down the audience, the survey audience, into two groups, those that they have fewer than 1,000 employees, those that have more. Not surprisingly we see that organizations with more than 1,000 employees, those at the enterprise level, really are doing a better job at training, at least in terms of the frequency of that training. They are keeping people engaged more, telling them more about phishing, largely because large companies tend to be the primary targets, that doesn't mean they're the only targets, we see a lot of smaller organizations, mid-size organizations and and so forth that fall prey to these attacks, but large organizations with enormous number of data assets that can be stolen tend to focus more on security awareness training.

Lisa Plaggemier: Yeah, Michael, this is also a result of more companies taking advantage of those teachable moments, maybe a shift more to that best practice of taking advantage of in-the-moment training, event-driven training, to see that increase in frequency overtime, I think that might be a result, if we can infer that from the data, of having shorter training more frequently.

Michael: Yes, and I think that's a great point, Lisa. It's absolutely important to keep this front-and-center in people's minds, to train them on a frequent basis so that they really understand just how important this is. What we found, as you can see here in this chart, there are some organizations that will provide a security awareness training module, just when the employee joins the organization, and that's simply not frequent enough. You have to make this frequent so that people keep this as a sort of top-of-mind issue at all times since they're dealing with email. Because the fundamental issues, when you look at the number of emails that people receive, or that they send, it's once about every three or four minutes and particularly in the morning, people get a lot of emails. It's important to process these appropriately and especially when you're talking about things like business email compromise that are trying to fool somebody into thinking it's the CEO or CIO sending them a message, the top-of-mind issue here becomes very important. What we also found in the survey is that the amount of time spent on training is actually increasing. We found, as you can see here, comparing mid 2018 to mid '19, to mid 2020, we find that the number of minutes per user, per month, that's spent in some sort of security awareness training is actually going up. It's testament to the fact that the decision makers really do understand, and increasingly understand, the importance of good security awareness training and making it more frequent and allowing users to spend more time doing it. Again, we see larger organizations having their employees spend more time doing security awareness training than those in smaller organizations, but we do see the increase really across the board. And Lisa, I wanted to ask here too, I mean it's hard to define an optimal amount of training, but what would you suggest in the context of having more training over time, more minutes per month spent doing something in training?

Lisa: I think it really depends, I'm a big proponent of the event-driven training, right, so if you've integrated the training with any of your end-point protections and serving a training to people that's really relevant and in the moment. So when I was a practitioner, we didn't have that kind of integration, so I was doing it manually with email DLP, and even if we didn't have time to, really you get a lot of false positives in DLP unfortunately, depending on your implementation, in your environment, issues you have to deal with. So we would serve up the training within an hour or two of a DLP alert, we would offer training that wasn't mandatory, we would just say that we got an alert from your machine and while we're investigating whether or not there's really something there we have this training module that might help you, just in case. And we found that if we kept those really short, the completion percentage on those was extremely high, because it was specific to their, in this case it was specific to their role, their function and hit them at just the moment where they have triggered an alert, by exhibiting a particular behavior, sending out unencrypted data, or whatever it was, right. So I think you know you would see these numbers increase, not just, not having people take more training just for the sake of taking more training, by tying it to their specific behavior and their role and any alerts that they've generated, anything that they've done, anything, the more timely it is I think the more effective it is.

Michael: Yeah, that's a very good point, if you can tie it in to a specific event, something like you mentioned, sending out an unencrypted email and then providing training I think that really strengthens the understanding that that's something that really shouldn't be done.

Lisa: Absolutely.

Michael: So does training really work? What we found in our research is absolutely. We asked IT decision makers and security decision makers, just how capable are your users at recognizing various types of threats before and after they've gone through security awareness training? And as you can see here the after picture is dramatically better in the context of things like mass email phishing, targeted emails and social media and web scams that users, after training, are much more capable of recognizing these types of threats. And what's important to keep in mind here is that we're asking decision makers about the after picture, if you will, with training that in many cases is really not all that good, or all that frequent. So if we're comparing sort of the before picture with very good training, training that is sufficiently frequent, that is on point, that provides the kind of issues and feedback that individual users need, absolutely, you'll get a much, much more capable audience in your organization, after they've received good training. We also wanted to find out about the support for training and what's interesting is that there's a lot enthusiasm for security awareness training in some quarters, not so much in others. For example, if we look at the bars on the left, for groups of employees, for roles that are very enthusiastic about security awareness training and really supportive within the organization, certainly senior IT management is the most responsive here, they are the most enthusiastic about training, largely because they can see it's benefits. If you can get a user to not click on a phishing email, not to open an attachment, and you avoid a ransomware infection, if you avoid an malware infection or you avoid a data breach, IT is going to be most directly involved in that and so they can see the real benefit of having that training provided to their end users. Senior business management, you know they're fairly enthusiastic about it, they see the benefits as well, employees, not so much. We found that fewer than a quarter of employees are really enthusiastic about security awareness training, but the good news here is that nobody, really vigorously opposes it. There are very few employees that actually will oppose security awareness training, they're really not all that against it, but for them in many cases, it's just another thing to do. This really underscores the importance of making security awareness training engaging and fun and really applicable to the way that people do their work. People need to be able to see the benefit as it applies to them, as it applies to their department, and so forth. Lisa, I wanted to ask you about this as well, what would recommend in the context of providing training so that it is engaging and it really makes employees more enthusiastic?

Lisa: Yeah, I was just looking at this data and I think if you'd have done this five years ago, that, "somewhat against it, "or vigorously oppose it" number, might have been a lot higher. I think the quality across the market has gotten a lot better, there are a lot more engaging options out there to choose from. We see more and more clients, and I see more and more of my former peers looking for content that's a lot more entertaining and engaging and experiential types of training that's a lot more engaging. I think we as an industry have gotten a lot better, there's still a long way to go, you know there's still hackers and hoodies and some stuff out there that's a little bit too heavy handed and probably, maybe some stuff out there that's a little bit boring and not exactly entertainment. But I think, in general, as an industry we've gotten a lot better. I think that the other thing that occurred to me as you were talking, Michael, was that I've done some reading on the research that's been done on things like breach apathy and breach fatigue and how people sort of have a learned helplessness approach to all this, right. The headlines are happening so quickly and the stuff is making the six o'clock news almost everyday and people are feeling kind of overwhelmed and not well equipped, in some cases. When you really ask them, "What do you think you can do personally?" a lot of people will tell you that they still have this impression that somebody in the IT office or in security is somehow protecting me from myself or from these bad things happening. And they don't really understand the role they play so I think one of the most important functions of education in this realm is that feeling of empowerment, of really giving people the hard skills that they need, even if it's something as simple as using a password manager or enabling multi-factor authentication, that's empowering, when I realized that I've done something that's gonna make me less vulnerable, there's a positive message to that. And I think that's a much more attractive message, that positivity and empowerment is a lot more attractive than the hacker and the hoodie and the, hey, scary things are gonna happen to you if you don't listen up and you'll end up on the naughty list if you don't take your training and all those things. So I think as an industry we've gotten better at relating to people that way.

Michael: Yeah, good point. I think too, one of the things that will drive enthusiasm for end users is if they see how this applies to them personally, not only in their work, but if it prevents them from losing all their family photos on their home computer, for example, because of a ransomware infection. So I think if people can apply this personally, they will certainly become more enthusiastic and as I mentioned, I think that's why IT is so enthusiastic about training because they see the direct benefit to them. They see the fewer person-hours devoted to ransomware remediation, to malware remediation, to recovering, reimaging machines and that kind of thing. What we also found in the research is that budgets are increasing. We see certainly total security budgets increasing and we've been doing this research for many years and we find that the vast majority of organizations are, year-on-year, increasing their overall security budget, but we do find also that security awareness training are increasing quite a bit and actually at a faster rate, we do see that training budgets are increasing faster than overall security, in large part because many people here are late to the game, they've only recently, in the last two to three years, realized just how important training is and so their pouring a lot of resources into it. Whereas traditional security is just that, it's a little more traditional, it's a little more staid, and organizations are certainly adding to their capabilities, implementing new solutions and so forth. But they're really piling it on in the context of security awareness training and adding a lot of new capabilities and really focusing on this, we've found just in the fairly recent past. So why are security awareness training budgets increasing? We asked decision makers here, "what do you consider the important or primary reasons?" and we asked these decision makers to rate the reasons on a scale of one to seven. So we've taken here, the sixes and sevens, for 39% of organizations IT was requested by the security team to implement security awareness training, the security team really wanted users to be more aware and more capable in dealing with things like phishing and targeted attacks and so forth. In 38% it was requested by the IT team, 34% of organizations there was an executive emphasis on security awareness and one of the things we're finding, across the board in the context of security, is that a lot of organizations at the board level are really understanding security more. They are adding a SICO to the board of directors, they are providing regular reports from the security team, up to the board of directors at their regular meetings and so forth. So there's much more emphasis at the board level on security in general so that's tending to trickle down to the organization itself, where there's more of an executive emphasis on security awareness and the training that will enable that. In 29% of the organizations it was just a result of the overall increase in the security budget, so more organizations were adding to security, they've also decided to add to their security awareness training. And then finally in 29% of organizations it was requested by the security awareness training team. So a lot of reasons for the budget increases for training but certainly they're focusing on greater awareness of security and the need to train users more appropriately. Now one of the interesting things we found is that the vast majority of organizations, so basically 88%, they measure ROI for their technology based infrastructure. So when they implement anything from an antivirus solution to a sophisticated, advanced, persistent threat capability they do measure the ROI to really understand if they're getting value for what they're investing in the security infrastructure. We found that it was only 55% doing that for security awareness training programs and in part I think a lot of organizations find it difficult to measure ROI for something like training because it's hard to trace the result back to the training cost itself. Also security awareness training is a newer capability than technology infrastructure and so I think there's a level of immaturity here and more organizations overtime will begin to focus on the return on investment for their training expenditures, but we decided to do that here in the study, we really wanted to understand is there an ROI that you can quantify that would help to justify the whole notion of security awareness training. So in terms of modeling ROI, we developed a cost model so that we could really better understand the ROI associated with security awareness training and as Hunter and Lisa mentioned, there is a cost model that you'll be able to access to be able to do this yourself. Some of our basic assumptions, and we had just some overall assumptions for the model, we assumed that there was an annual, fully-burdened salary for IT staff members at $80,000, for just regular non IT employees it was 75,000, you know, 2080 hours worked per year. We also assumed that the productivity lost during downtime for an information worker was 70%, it's gonna vary widely, you can have an organization where if somebody's completely online all the time, their productivity loss is going to be approaching 100%, for some organizations it will be lower, in our research, in our experience, 70% is a relatively conservative number when things go down. For example, if somebody gets a ransomware infection and they can no longer access their work resources, the productivity suffers dramatically. We also found that the actual productivity loss from employee time spent in security awareness training is only about 15%, because very often the training doesn't involve pulling somebody away from their desktop or their laptop and sitting in a training session, very often it's just in the course of their work. Being tested on phishing attempts and so forth. So it really is not very disruptive in the context of productivity. We also estimated that the reduction and the effectiveness of major attacks from having good security awareness training after an employee has been well trained on detecting phishing, you know looking for ransomware attempts and so forth, is gonna be 90%. That sounds high but this is based also on empirical research where organizations have found that well-trained employees are dramatically less likely to click on that link or open an attachment when they shouldn't. Now getting back to the fully-burdened salary for IT staff members. This can vary dramatically, if you look at Glassdoor, for example, they're a good source of information on salaries for really a wide variety of jobs. The cost of an IT person is gonna vary dramatically, if you were in the lower-third of Manhattan, obviously your salary's gonna be dramatically higher than if you're in Wichita Springs, Texas. So this can have a significant impact on ROI because a lot of the cost that you're trying to avoid is having IT people remediate ransomware infections and reimaging machines, that kind of thing. So keep in mind when you go through the calculator that the cost of salaries is really quite a significant component in the overall cost of developing ROI. So let's look at the first cost, sort of routine security practices. We looked, and this is all based on the research that we did, the mean, monthly IT security person-hours that are spent disinfecting workstations and networks after a malware infection, after a ransomware infection and so forth, works out, and I won't go through all the numbers here, but it works out in smaller organizations to about $29.23 per user, per year. IT is spending time remediating the problems that occur when somebody clicks on that phishing link and bad things happen, when malware enters the organization. Larger organizations, because of economies of scale, because of greater efficiencies and so forth, that cost drops to $5.28 per user, per year. And again, keep in mind that these costs are gonna vary widely in your particular situation based on the salaries of your IT staff, the frequency of training that you provide, or the training that you don't provide, and so forth. We also found that after security awareness training in smaller organizations, the time spent, the cost associated with disinfecting workstations and networks, actually drops significantly, it drops to 21.75 per user, per year in smaller organizations, it drops a little bit in larger organizations, again because there have been greater efficiencies all along in larger organizations and there's been more training and so forth, that cost drops, but it drops by a smaller proportion. Lets look at a second cost, remediating major events. One of the things we assumed in the model is that there would be one, really major event, that you'd have one ransomware infection, that wipes out a very large segment of your network or in some cases shuts down the entire organization. If you look at, particularly in the government space, where you've seen a lot of municipalities, counties and even state-level governments that have been shut down for some period of time. If you look at the recent ransomware infection in Baltimore, last year's infection in Atlanta, there's been Jackson County, Georgia, lots of examples of municipalities getting shut down. This is a major ransomware event that really shuts down city government for awhile. They have to switch to paper-based recordkeeping, city services get stopped for quite some time, and so forth. So what we assumed was that for the typical organization there would be one of these really big events per year, and again your situation is gonna vary quite a bit, you might get two or three of these events per year. We've seen some organizations, particularly in Europe, that have seen these kinds of major events happen more than once a year. So if we look at the IT or security hours required to remediate just one of these major events, a ransomware or a malware infection, we're looking at a cost of $7.51 per user, per year, for smaller organizations. For larger organizations it's actually more expensive, 28.11 per user, per year, and that's largely because these kinds of events tend to be much more impactful for a larger organization, more data is stolen, more users are affected, and so the ramifications of this are really much more expensive than what you'd find in smaller organizations. So after going through security awareness training, and again, based on the 90% reduction in the likelihood of one of these events happening, we see basically a 90% reduction in the cost for both smaller and larger organizations. Again, your results may vary, but we do think these are very conservative and very real-world numbers because they're coming from the research that we did with decision makers in the security space. Let's look at the next cost, productivity losses. And this is really one of the most impactful costs of any kind of malware infection, ransomware infection and so forth. Basically the number of hours that 1,000 users would be down from the attack and we normalized all of these results on a per thousand user basis, really just to get a solid baseline for smaller organizations and larger ones. And what we found is that because of the greater impact of a major event, like a ransomware infection, on larger organizations, they're costs will be significantly higher than for smaller organizations. We found, for example, that for 1,000 users, there would be a total loss of productivity of 18,000 hours, in a larger organization, it would be closer to 10,000 hours in a smaller organization again, because of one of these infections, after ransomware, again using that 90% reduction, we find a dramatic drop in the productivity loss that results from one of these kinds of major events. Now one of the important things you need to consider is that productivity loss is not a cost for which you're gonna be writing a check and one of the things we found, we've been doing this research for a long time, one of the things we found is that some decision makers are a little bit resistant to considering productivity loss as a real cost of these kinds of events, of a ransomware infection, for example. In large part because when we don't write a check, you can just assume that employees are gonna make up the time on their own, they'll work from home, they'll stay in the office longer, but this still is a, even if that occurs, you do have productivity loss that arises from security incidents. So you absolutely need to consider it as a key component of any ROI calculation or even in a cost calculation. We also wanted to look at the cost devoted to training. And what we found was that the mean, monthly IT or security-hours devoted to security awareness training in smaller organizations, in smaller organizations it works out to about $44.61 per user, per year. In larger organizations it's quite a bit less, again, because of the efficiencies that they gain from having more users, having larger venues for being able to train employees, more efficiencies that they derive that way. We also took the cost of security awareness training, from a leading provider, and again your cost may vary, but we're assuming for smaller organizations they're gonna be spending $23 per user, per year, on the security awareness training that they provide. For larger organizations, because of volume discounts, that'll drop to 17.50 per user, per year. And then we also wanted to look at the cost of employee time spent in training. And again, there's not that much of a productivity hit, for individual users spending time in training because they're doing so in the course of their work, they're being tested on phishing, for example, which doesn't really have an impact, there's no real disruption in the way they do their work. So we're looking at a cost of 21.11 per user, in smaller organizations, a little bit higher than that in larger ones. So what we came up with is that the total cost for 1,000 employees, there's a typo in this chart and I apologize, this is actually the cost per user. What we found is that in smaller organizations, before training, smaller organizations are spending about $286 per employee, per year, after training that drops to $136 per user, per year. In larger organizations, because of the greater impact of major security incidents and so forth, that cost is about $489 before training, dropping to $110 after training, so a much more significant drop in costs for larger organizations. So the cost of ROI for smaller organizations is actually quite dramatic, I mean an ROI of 69% is really very good and this is really just at sort of the top level. Disinfecting workstations, the cost drops from about $29 to about $22 per user, per year. Remediating a major malware or ransomware incident goes from $257 down to about $26. Then we've just added in here the cost, labor costs of security awareness training, the cost of the training itself, the training modules and so forth. Then the employee time spent on security awareness training, so again, dropping from $286 to about $136, resulting in a return on investment of 69%. But if go to larger organizations we find really much more bang for the buck because of the greater cost of security incidents, that's the primary driver here. The greater cost in security incidents in larger organizations, we find a much more dramatic cost decrease for, in the after picture, if you will, for security awareness training and it results in a much greater ROI, 562% for larger organizations. Again, these numbers seem very high, but keep in mind that we used fairly conservative assumptions here, so the ROI can actually be significantly greater, for some organizations, it can be less for others, but we find that based on these conservative assumptions the ROI in security awareness training is actually quite significant.

Lisa: Yeah, my first impression too, Michael, when you showed me the results of your research was that the numbers seemed astronomically high and the first reaction that you have is to go back and look at all those assumptions, like you indicated. So I'm gonna give everybody a spoiler alert here that we'll have a spreadsheet available because I think what's really handy is for people to be able to tweak these numbers, as they see fit, for their own experience and their own organization and maybe costs or incidents or whatever that they've had where they could plug in their own numbers. But I think any which way, no matter how aggressive or conservative you are with the numbers, I can't imagine a situation where you can't show any ROI at all. It's really pretty clear once you start plugging numbers into the spreadsheet, that there's definitely an ROI there, that's for sure worth doing and definitely a tool you can use, you know if you don't have budget, to get budget. If you have budget but need more, if you need resources, I mean definitely a tool you can use to have those kinds of conversations with your leadership.

Michael: Yeah, and that's a good point, and again, going back to the high ROI numbers, it's one thing that bothered us initially as well. So we continued to go back to the model, over and over, just to make sure we hadn't made a mistake here, but the numbers speak for themselves. Again, if you never have a ransomware infection, you're ROI is gonna be significantly different but if you're an organization of any size you can expect some sort of an infection at some point, it's just a foregone conclusion, because just about every organization that doesn't have good training, that doesn't have good security, is going to have some sort of an infection and many of them that do have good technology are still gonna get infected at some point.

Lisa: If you've never had a malware infection, then I have other questions for you. That's pretty unrealistic now.

Michael: Oh, very definitely. So one of the things we did not factor into this cost model are the other costs that are associated with a malware infection, with getting shut down from a ransomware infection, and so forth. Again, the ROI calculations that we did are conservative and we intentionally didn't include a number of other costs. For example, if you don't have good training and you have a greater chance of data breaches, malware infections and so forth, there's a lot of other costs you have to consider. Regulatory fines, for example, if you breach sensitive or confidential data, if you breach personal data held by individuals, just about anywhere you can face regulatory fines. It's just one example, British Airways had a major attack, last September, where they suffered the loss of something like 380,000 records for their customers. The European Union fined them $605 per record for that data breach. So the cost of data breaches really is going up over time, in the context of the regulatory fines. We see this from the GDPR, where they've levied a number of fines, we're going to be seeing it in California from the California Consumer Privacy Act. And all 50 states now have data breach notification laws, so there can be fines, there can be other costs that go along with that. For example, there will be lawsuits, if you look at the Target data breach, for example, and again that wasn't directly related to a lack of security awareness training but a data breach is a data breach, regardless of its cause, you will inevitably spawn lawsuits from this so you're gonna have to be able to cost from those, you're gonna have to deal with the cost of judgements from those and so forth. There will be in many cases credit reporting costs, after a major data breach, it's quite common for an organization to offer free credit reporting services for two to three years for those victims of the breach, so that's gonna be a significant expense. There's gonna be a loss of revenue, we have done secondary research that finds that a lot of organizations that have suffered data breaches just have a loss of revenue. Customers stop doing business with them because they don't trust that they're data is going to be secure. Certainly there's gonna be a loss of brand reputation, if you look at any organization that has suffered a major data breach that has appeared in the news, there's a loss of brand reputation, there's a lot of damage that comes from that. We've also found that for many organizations their stock price drops significantly and a lot of third parties have quantified the cost here, and they have found that the stock price, even up to three years later, is lower than it otherwise would have been simply because of a data breach. And finally you're gonna have some lost employees, there's gonna be some people fired as a result of this. In the case of Target, they lost their CIO, and it's really quite a common occurrence to lose key people after a data breach, not because they leave, but because they're fired for not protecting the organization well enough, so that can cause disruption and it adds to the overall cost of the breach. So in summary, certainly what we found in our research is that good security awareness training can dramatically reduce the likelihood of cyber threats becoming successful. If you plug that last hole, if you train users well and teach them not to click on phishing links unless they know the source. Not to open attachments from unknown sources and so forth, you will go a long way toward protecting your organization and really enhancing security. And while the results can vary significantly as you'll find as you go through the calculator, the ROI from good training is significant, in almost every case. There can be a few rare cases in which you'd get a negative ROI or no ROI at all, but those cases are gonna be few and far between. Certainly you will have positive ROI in just about any calculation you run here. Okay, Hunter and Lisa, I'd like to turn the floor back to you now.

Hunter: Yeah, just before we get into some questions as Lisa alluded to earlier, this is our ROI calculator. Did you want to go into that a little bit, Lisa?

Lisa: Yeah, I didn't mean to steal your thunder, Hunter, with my spoiler alert, but I think if you check your email after the webinar you'll see that this is gonna be available to download and so then you can tweak the numbers, as you see fit, and put your own data there and it'll give you the result. You can see the data that's in the background as well, so this is fully configurable. You can use it anyway you'd like. I have some just some soft-skilled words of advice about using something like an ROI calculator as a basis for a discussion with your leadership. I think as security people we tend to spend a lot of time being shocked at the things we see all day and there's the tendency to want to go in with a little bit of shock-and-awe ourselves and the numbers, these numbers can be really, really impactful and telling, but I would caution you to make sure you use really reasonable numbers that you feel like you can successfully defend. So whenever you put calculations in front of a business person the first thing they're gonna say is where'd those numbers come from? Let me see your assumptions, how did you get to that number? They're not gonna look at that bottom-line number, they're gonna want to, potentially, want to pick through all the numbers in the calculation, and the calculations themselves to see how you got there. So make sure that you're really prepared to have those conversations and defend those numbers. Be prepared to walk them through you actual numbers, if it's possible if you take your actual numbers that you're able to find in your organization or some good guesstimates, maybe even step those down a little bit, be a little conservative, so that you can sort of say, well our experience indicates number X, but to be conservative I chose to use this number Y. You want to err on the side, as Michael said earlier, you want to err on the side of being conservative and get really comfortable with those numbers so that if your leadership or your controller or whomever, wants to talk through them with you, that you're well versed and can have that conversation and can walk through those in a way that you can defend and justify the numbers. As I said, you get to a certain point with some numbers that are so high that is this a 462% instead of 562%? Nobody's gonna argue with the fact that there's still an amazing ROI there. So I would pay less attention to how shocking that bottom-right number is and really dive in to the numbers in the spreadsheet so that you can have an educated conversation that you can defend your calculations and how you got there.

Hunter: Definitely, definitely a really powerful tool to have at your disposal. So let's get into some questions. There we go, we have a question about, from Bruce, he's wondering, I think Lisa you were talking earlier about having sort of a reactive approach to training, if something pops up, giving an employee a personalized training. He's wondering, should the training content be focused on scenarios raised from reacting or should we have a more proactive approach to things?

Lisa: So I think you need to do both, I ran a program where I had annual compliance training, it just ticked the box for compliance, it was completely mandatory and it was just the way life was, everybody had to take it, but I kept it relatively short, made sure I hit everything I needed to hit for compliance reasons, but that was it, I didn't try to change the world through my compliance training. Then I had additional training that was optional, that was more compelling and more interesting and funny and all those things and it covered a wide variety of topics. That was kind of run consistently all throughout the year and it took various forms, it could be funny videos, it could be game show, it could be a lunch-and-learn, all different types of activities. And then in addition to that I had sort of the just-in-time, teachable moment-type training as a result of DLP alerts or as a result of something we're seeing at the proxy or some sort of endpoint alert or a specific incident that we had and that training, I kept that short as well, but it was relevant just for those people who were involved in the incident or that triggered the alert, and most cases, I did not make that mandatory, it depended on what the situation was, but we had chosen, for email DLP for example, we had chosen about four or five different scenarios that we were seeing frequently, behaviors we were seeing from employees that we wanted to affect, so we chose the training modules that we would use for each one of those, they were three to five minutes, they weren't very long. And again, when we sent the email to employees saying hey, we got this alert, we're investigating, here's some training that might help you, just in case, we took that soft approach and the way that email was worded was to be helpful, and it wasn't, this is mandatory and you have to do this otherwise, you know blah, blah, blah, up to and including termination, all that heavy-handed language, we didn't use any of that. We had really high participation rates because people viewed us as being their to help them and it was enough, it sounds threatening enough when you say, we got an alert from your machine, most people even though we all sign, acknowledge every year that we know there's all kinds of technology on our computers watching what we're doing all day, most people are oblivious to that and when you say I got an alert from your machine, that in and of itself, is intimidating enough, or kind of shocking enough for the end user, you don't need to kind of layer on threatening language about training on top of that. So taking that approach that the training is there to be helpful, I worked for a SICO who used to say, I don't want to give training to people who don't need it because it eats up manpower, it eats up time, and it's a drag on the company, from a business perspective. So how do we make sure that we give training to people who really do need it at the moment when they need it, so that's kind of the argument for that event driven, those types of triggers for training, but I don't suggest you do that exclusively at the expense of all other types, I think you need to have a variety.

Hunter: Right, definitely, it's nice to have coming at it from all angles. Kind of piggybacking off of that, Gearish was wondering, how can we make compliance training making people more engaged and more enthusiastic to engage in those activities?

Lisa: I would say that just by the terminology itself, compliance training, it's really hard to get people enthusiastic about it, when it's mandatory, there's a little bit of psychology going on there where just because you tell me I have to do something I don't like it. And I've played off of that, in the past, where I've used a pre-test, which was actually training in and of itself, to say to people that hey, if you pass this pretest you don't have to take training, so then everybody raced to take the pretest and the pretest acted as my mandatory compliance training, not the training that they got afterward. So that was just a little bit of a reverse pyschology to kind of play off the fact that people don't like it, well if you don't like it how can I get you to maybe do, if I call it something else, not the training, can I get you to do that instead 'cause we kinda had this brand image that nobody liked our training, so I had to get really creative with ways to drive people to it. I think the quality of the content these days, you know for me that was four or five years ago now, when I kind of pulled that reverse psychology move, I think these days there's so much more content out there that actually could fulfill your compliance requirements that's engaging. I mean we have a life action series that I filmed this spring in Chicago, that's kind of like the Office or Parks N Rec, it's really funny and there's a lot of content in there that can help check the boxes. The animated stuff that everybody's doing has gotten a lot better and a lot more engaging, it's less preachy, it's more conversational. I think you also have to think about your training as just one of the tools in your tool belt, if you've got a compliance requirement, maybe rather than pushing hard on, hey, everybody has to do this, right, this is mandatory, if you make it part of a complete campaign, meaning if there's emails and articles and posters and lunch-and-learns, and everything else happening and the training is just one component of that, that might make it, that might help to generate a little bit more excitement, especially if you're using the training content that in and of itself is engaging. But as I said, I kind of ran two parallel programs, I know a lot of practitioners who do that, they do the bare minimum that they have to do for compliance just to meet compliance, and then if they're trying to really change the culture in their organization and get people engaged then they don't make those other training modules and activities mandatory, because just kind of have a knee jerk reaction when you say something is mandatory, it's kind of suddenly already less engaging just because you told somebody they have to do it.

Hunter: Right, and sort of circling back to the ROI, Michael, you sort of touched on this with looking at the certain legal fees or fines associated with data breaches, other data points that weren't included in this ROI calculator. Are there any other data points that Michael or Lisa, you can think of that could positively or negatively affect ROI?

Michael: There are a lot of things that are really tough to quantify, in the context of ROI, but that still have an impact. As one example, the hit to your reputation, if you look at companies like Target, Equifax, Anthem, I mean the list goes on and on in the context of data breaches. It's hard to really evaluate and quantify just how impactful that loss of reputation really is. Certainly you can do it in the context of things like stock price, a lot of organizations, as I mentioned, will suffer a drop in their stock price that really persists for a number of years after the data breach. You know what is the real impact on the ROI on the organization, again, even if you quantify the drop in the stock price, that doesn't necessarily have an impact on the day-to-day operations of the company in terms of security costs and employee costs, and so forth, but it does have an impact. There are things like loss of revenue, you know how many people won't shop at your store? Or how many people drop out of your supply chain because they don't trust you can manage their data properly? Again that has an impact on ROI, but again, it's a difficult thing to quantify because if somebody doesn't renew their contract with you, was it specifically because of the data breach or was it because of something else? Again, these are the harder issues which is why in the cost model we tended to focus on things that you could clearly quantify. The amount of time that IT was spending addressing problems that otherwise wouldn't have to be because of good training. Or the employee productivity losses, and so forth. But there are lots of different things that can occur, even to the point of employees sitting around talking about the data breach, it's the kind of thing that's water cooler conversation after a major incident happens in the organization, you do have productivity loss from that, but again, it's pretty tough to quantify.

Hunter: Definitely, and going back to the specific ROI calculator, I have a question here from Lew, he's saying, what if, knock on wood, a given organization does not have real numbers in terms of downtime, et cetera. Are there some industry standard averages that can used to present to senior management and thankfully they haven't experienced a breach or loss of productivity to malware, but they're very much proactively trying to prevent those types of things.

Michael: Yeah, that's a good question. The figures that we used that I presented on that slide of assumptions are what we would consider to be industry standard numbers. The 70% loss for example in productivity loss when your machine is down, while it's being reimaged and so forth, that's a pretty reasonable number. Again, it's hard to come up with a specific number because it can vary so widely. For example, if an employee has a ransomware infection and their machine is shut down for eight hours while it's being reimaged, maybe they go home and they start working on their home computer and they can once again be productive, but there is gonna be some loss while they're driving home and that kind of thing. So there's always gonna be some sort of a hit but it's gonna vary widely based on particular situations.

Lisa: Yeah, I think the calculator that is available for download, the spreadsheet, is pre-loaded with the numbers that Michael has used in his research so if you don't have your own numbers you can work with those averages or you can tweak those as you see fit. Is there anything else that you wanted to cover, Michael? I didn't have any other comments.

Michael: Yeah, I think we're good from my perspective, I think we covered the major points we wanted to.

Lisa: Okay, well as I said the spreadsheet will be available for download and feel free to tweak those numbers and educate yourself and have those conversations with leadership. We of course, we'll also email the recording of the webinar out, for anybody who's interested. I'm on LinkedIn probably way too much, having conversations with people but I'd be interested to hear if you have any questions about this content or your getting ready to have one of those conversations and you want to bounce ideas around or you've had a conversation using the ROI calculator, with your leadership trying to make the case for training and awareness, I'd love to hear success stories or otherwise, so follow us on social media or don't be shy to reach out directly to me if you'd like. Okay, with that we'll wrap it up. Thank you very much, Michael, your research is always really, really interesting and really well done and thorough and we appreciate it.

Michael: Well thank you so much, it was a pleasure to be here.

Lisa: Okay, have a good day everybody.

Chris: I hope you enjoyed today's webinar episode! Just as a reminder, many of our podcast also contain video components which can be found at our YouTube page. Just go to youtube.com and type in 'cyber work with infosec' to check out our collection of tutorials, interviews and other webinars. And as ever, search 'cyber work with infosec' in your podcast app of choice for more episodes. Thanks once again to Michael Osterman and Lisa Plaggemier and thank you all for listening. We'll speak to you next week.