The Phish Scale: How NIST is quantifying employee phishing risk

With the relatively recent uptick in phishing around the globe (due in part to Covid-19 and other factors), experts at the National Institute of Standards and Technology (NIST) have been working hard to create a new way to quantify phishing risk for organizational employees. This new way is called the Phish Scale. If phish and scales have you thinking more of the messy work associated with processing fish to eat, this article will give you a better smelling impression of the phonetic term. 

See Infosec IQ in action

From gamified security awareness to award-winning training, phishing simulations, culture assessments and more, we want to show you what makes Infosec IQ an industry leader.

Demo Now

What is Phish Scale?

Released by NIST in 2020, Phish Scale is a breath of fresh air in this age of ever-increasing phishing instead of the aquatic stench the name might suggest. Phish Scale was created as a method by which CISOs can quantify the phishing risk of their employees. It quantifies this information by using the metrics of “cues” and “context,” which makes the data generated by training simulations to be more insightful. In essence, it allows organizations to better categorize actual threats (for better detection) and to better determine the effectiveness of their phishing training program.

You may be wondering why this is a significant development — and it is probably more significant than you think for those that see its value in determining program effectiveness. Before Phish Scale, the traditional metrics organization used were click-rate, which is not always reporting rates and reporting times. By adding cues and context to the mix, organizations will have a more accurate view of where they stand regarding phishing detection.

Phish Scale

Number of cues Context Detection difficulty

Few High Very difficult

Medium Very difficult

Low Moderately difficult

Some High Very difficult

Medium Moderately difficult

Low Moderately to least difficult

Many High Moderately difficult

Medium Moderately difficult

Low Least difficult

Above is a visual depiction of the Phish Scale. It uses the metrics of the cues present in the phishing emails and the context of the information contained in the email about the organization which is referred to as premise alignment by NIST (simplicity is king so context it is).

Metrics: Cues and context

Cues refer to the characteristics of the phishing email that may tip off, or cue, the recipient into thinking that the email is legitimate. There are five types of cues to look out for, presented below:

Type of cue Cues

Error Spelling and grammar irregularities

Inconsistency

Technical indicator Attachment type

Sender display name and email address

URL hyperlinking

Domain spoofing

Visual presentation indicator No/minimal branding and logos

Logo imitation or out-of-date branding/logos

Unprofessional looking design or formatting

Security indicators and icons

Language and content Legal language/copyright info/disclaimers

Distracting detail

Requests for sensitive information

Sense of urgency

Threatening language

Generic greeting

Lack of signer details

Common tactic Humanitarian appeals

Too good to be true offers

You're special

Limited time offer

Mimics a work or business process such as a legitimate email

Pose as a friend, colleague, supervisor or authority figure

Context, or Premise Alignment, is the other Phish Scale metric. There are two methods to categorizing context. The first method uses three rating levels — low, medium and high for how closely the context aligns with the target audience. The second method uses five elements, rated on a five-point scale to measure workplace/premise alignment called the alignment rating. This helps the phishing trainer at the organization score the phishing exercise as being of low, medium or high difficulty based upon the data gathered of the phishing simulation. The Phish Scale implementor can choose either method they like and this article will focus on the five-element method.

The five context elements are:

1. Mimics a workplace process or practice
2. Has workplace relevance
3. Aligns with other situations or events, including external to the workplace
4. Engenders concern over consequences for not clicking
5. Has been the subject of targeted training, specific warnings or other exposure (not scored)

Only elements 1-4 are added up when scored with the fifth element being subtracted from the score. The five-point scoring system used to rate each element is based upon even numbers of 0-8:

8 = Extreme applicability, alignment or relevancy

6 = Significant applicability, alignment or relevancy

4 = Moderate applicability, alignment or relevancy

2 = Low applicability, alignment or relevancy

0 = Not applicable

Application

NIST tested Phish Scale by using 10 exercises on organizational employees. These exercises were emails that focused on different angles to trick the recipient. Below are the angles used in each exercise:

• E1. Safety requirements email
• E2. Weblogs (unauthorized web site access)
• E3. Unpaid invoice
• E4. Scanned file
• E5. New voicemail
• E6. Valentine “eCard delivery”
• E7. Order confirmation
• E8. Security token
• E9. Gift certificate
• E10. Adobe update

To highlight the disconnect between click-rate percentage and the actual difficulty level of detecting the phishing exercise, let’s take a look at how one exercise rated very difficult with few cues and high premise alignment, scanned file (E4). This exercise was conducted with 62 participants taking part.

Context element Alignment rating Actual click-rate percentage

Mimics a workplace process or practice 6  

Has workplace relevance 4  

Aligns with other situations or events, including external to the workplace 6  

Engenders concern over consequences for not clicking 4  

Has been the subject of targeted training, specific warnings or other exposure -2  

Total 18 19%

This phishing email exercise used a message referring to a shared scanning and printing device, a common device in organizational settings. Despite a high level of difficulty based mostly upon a mimicked workplace practice that aligns with workplace situations significantly, there was only a 19% click rate.

Phishing simulations & training

Build the knowledge and skills to stay cyber secure at work and home with 2,000+ security awareness resources. Unlock the right subscription plan for you.

VIEW PRICING

Utilizing NIST to categorize phishing threats

NIST has released the Phish Scale method for CISOs (and organizations generally) to better categorize actual threats and to determine if their phishing program is effective. It allows implementers to use other metrics aside from the traditional click-rate percentage to do this, which will positively impact cybersecurity in the face of an increasing number of phishing attempts.

 

Sources:

Categorizing human phishing difficulty: a Phish Scale. Oxford Academic Journal of Cybersecurity

4 Things to Know About the NIST Phish Scale, Mindpointgroup.com

The Phish Scale: NIST-Developed Method Helps IT Staff See Why Users Click on Fraudulent Emails. NIST