Here’s How Boards Should Measure Anti-Phishing Programs
In board rooms across the globe, directors are asking the question, “How is phishing affecting the organization and are we able to handle the risks?”
Fair enough. As anti-phishing programs continue to progress, it becomes important to develop and report against key measures of success. To do this, you need to look across a spectrum of data that reveals what your organization is experiencing today and how well it is prepared to deal with the phishing challenges of tomorrow.
Further, you must examine multiple levels of associated data sets. The organizing principle used within PhishMe® Professional Services is that of capability, along with the available solution sets from which to measure that capability.
As you can see in the model on the left, organizations vary greatly in their ability to resist a phishing attempt.
Each level, except for ‘No Capability /Red,’ will provide its own leading indicators or measures of success that can and should be shared with leadership and board members.
Let’s take a closer look at the types of data each level might provide.
Recognition Capability – is best measured through the repetition of phishing simulations across your entire user base. Leading indicators of success in simulation results include reductions in susceptibility and reductions in repeat offense rates across like/similar scenarios.
For the uninitiated, simulations involve phishing your own employees (after explaining the program to them) to condition them to recognize potentially malicious emails. The key metric here is your susceptibility rate, showing how susceptible employees are to different phishing tactics.
Reporting Capability – moves us further towards the ability to get “left of breach.” What we are looking for here is action: employees reporting emails they have learned to regard as suspicious. Success at this level will show overall increases in reporting, as well as a measure of resiliency against specific types and models of attack. Resiliency is measured as number reported/number susceptible in any given simulation.
Response Capability – is the real end game of an anti-phishing program and where we find the truest measure of success. At this level, it is critical that we take in, analyze and vet reported phishing data from our users. Measures of success include increases in reports and mitigation of malicious threats.
In addition to the development and measurement of these core competencies, we need to include an understanding of the full scope of the phishing problem specific to each organization. In other words, we cannot look at our capability results in a vacuum.
To that end, board room reporting needs to include data and analysis across the points below. Across the first three, you want to show what your detection and mitigation technology has found and secured. In the final point, you should address the nature of what your company has identified and mitigated, thanks to the improved recognition and reporting capabilities your anti-phishing program creates.
- Perimeter Security
- Network Security
- Remote Access
- Endpoint Security
- Desktop Firewalls
- Endpoint Security
- Reported Threat Analysis
- Capability Assessment
- User Identified Threats
- Categorization of Crimeware
- Analysis of Malware
Now that we have identified key components of reporting, let’s take a look at how these can be assembled for leadership and board level reviews. Remember, what can and should be presented will be based on the capability framework discussed earlier.
Below we share a basic model of reporting metrics that builds as anti-phishing programs mature . It should be noted that capabilities will exist across the levels above; where you have data and results, you should share them. Once again, let’s begin with key measures associated with simulations.
Leading Indicators of Recognition Capabilities:
- Repeat Susceptibility Analysis of Simulations – With this measure you are looking at your organization’s capability to reduce overall susceptibility. Analysis should include the following factors:
- Percentage of population found repeatedly susceptible. This percentage should be shown relative to current state and as a positive trend over time.
- Key Takeaway – Continued reduction in repeat offense at the individual level allowing for targeting of additional training within and without the core simulation program.
- Comparative Analysis of Most Recent Simulations – While not definitive, a comparative analysis of susceptibility within an industry and across the entire phishing landscape can add context most boards will want to see. What is your company’s ability to recognize threats that have made it past perimeter defenses?
As anti-phishing programs and capabilities mature, the focus of reporting can shift from susceptibility to resiliency. That shift will be important because some people will always be susceptible to phishing, whether acting on emotional triggers or falling prey to sophisticated tactics. However, if you counterbalance this risk with a critical mass of employees reporting quickly and accurately, you’ll reduce the risk of a breach. In all likelihood, you’ll only risk minor infections which you can treat at a much lower cost, financially and reputationally.
While reporting may maintain the repeat susceptibility analysis outlined above, the following adjustments and additions should be made.
Leading Indicators of Reporting Capabilities and Organizational Resiliency:
- Comparative Analysis of Most Recent Simulations – The comparative analysis should now include reporting data associated to the most recent simulations.
- Key Takeaway – You are looking for recognition and reporting to overtake susceptibility within specific phishing models and simulations. As you will see later on, simulation reporting levels based on active threat models can give you a preview of how well your company will respond when facing a particular threat.
- Increases in user reporting and mitigation of real threats – This is as simple as a raw number calculation of user reports and mitigation efforts over time.
- Categorization of threats and analysis of identified malware – This level of analysis helps you understand the magnitude of threats specific to your company and where to focus your efforts to improve recognition and reporting.Resiliency Trend over Time – In addition to specific comparisons, it is important to understand your overall resiliency over time. Naturally, we expect to see resiliency climb as programs mature.Up to this point, we have discussed reporting requirements in terms of simulation data and capability development. As an organization begins to respond to user reports of suspected phish and mitigate real threats, analysis of the scope of actual phishing attacks should be included in discussions with leadership teams and board members.Indicators of Anti-Phishing Program Success:
In the suspicious email analysis above, we are able to see the volume and type of attacks, as well as get a view into your organization’s ability to resolve phishing threats in real time.
Key points at this level of reporting:
- You can achieve a full understanding of the specific phishing threats your company faces.
- What types of phish and how many are entering the environment?
- What malware do we need to ensure we have solutions for?
- You can measure and address your company’s ability to recognize, report and respond to threats.
- How many threats have we mitigated ‘left of breach’?
- What do we need ensure our users can recognize and report on in the future?
In closing, it’s important that we stress the idea that reporting on an individual organization’s resilience to phishing needs to include data regarding the successes and gaps in proactive phishing defense from both the technology and human capability perspectives.
Remember – Gaining the full picture empowers improvements over time. That full picture should answer the following questions for leadership teams:
- What is our technology identifying and mitigating?
- What is making it past our perimeter, network and endpoint security?
- How capable are we to respond to attacks that have made it past those defenses via email?
- Are we increasing resiliency?
- Are we dealing with reported threats in real time?
- Where should continuing anti-phishing efforts be focused?
To distill: view and show the entire landscape in a transparent manner. This will boost the odds of getting your board’s buy-in. Measurably show how you have refined your strategies and tactics and you’ll build the support you need to help turn the tide.
Learn how reporting decreases susceptibility to phishing emails and lessens the time to detect a breach in our latest 2017 Phishing Resiliency and Defense Report.