It’s no secret that data is revolutionizing industries. Baseball managers have applied data to buck century-old beliefs about strategy (think Moneyball), anyone who has ever used Amazon.com knows that data has transformed retail, local law enforcement analyzes data to predict crime, and scientists are even using data to stop the spread of infectious diseases.
Most security awareness programs fail to gather metrics. Those that do typically measure inputs instead of outputs. What this means is that many teams are measuring items such as the number users who complete a CBT course or attended a lunch instead of the number of incidents related to a specific IT risk area. This is akin to looking at the number of times I visit a dentist each year instead of the number of dental incidents (cavities, root canals, etc.) and using that data as an indicator of good dental health.
Programs that collect meaningful metrics about behavioral change within their organization can make effective decisions and drive desired change with the data to back it up!
Traditional security awareness involves implementing a variety of security awareness initiatives and hoping for the best. Whether they are posters, knick-knacks encouraging employees to change their passwords, or annual classroom training, most of these initiatives fail due to a lack of measurable effectiveness (amongst other things), and offer no idea whether they are really improving employee behavior. It’s no surprise security awareness budgets are often limited, given the absence of proof that we are actually accomplishing our goals; thus, no justification for more resources. Collecting metrics offers not only information about the past effectiveness of our programs, but also a path toward continuous improvement and better security posture.
Every security awareness initiative you implement is an opportunity to collect information. Metrics measuring overall vulnerability to phishing emails are useful as a baseline to assess your readiness for a phishing attack, but offer much more insight. By measuring your susceptibility after each security awareness exercise you conduct, it gives you perspective of which concepts are working and which ones aren’t, allowing you to refine your techniques to improve the program. This is a great example of a directionally correct output metric.
Metrics shouldn’t end with something like the behavioral metric above; they should extend into the real treasure-trove which is your IR process. Using phishing as an example (probably one of the best to use, not because we’re PhishMe, but because it’s the most common initial entry point!) teams can measure outputs such as the change in the number of phishing related incidents, the time from incident to detection, and the number of user reported phishing incidents. All these metrics can show that behavioral change is happening (or not) and provide you with cost benefit data to support your initiatives.
Not coincidentally, collecting these metrics helps you follow the advice from our last post about keeping security awareness training fresh by mixing up the topics and methods you use. Simulated exercises using various tactics and at various times throughout the work week will not only make things more interesting for your users, you will receive valuable feedback about the areas where your users are most susceptible as well as when they are most vulnerable.
Your security awareness program should allow for collecting information about individuals, departments, etc. in your organization, and discerning which users are susceptible and which are more security-savvy. This knowledge enables you to tailor programs to users based on their level of knowledge. An at-risk user can be given more remedial training, while advanced users can be trained on more advanced topics such as conversational phishing. Testing user knowledge through a quiz and tracking their responses (shameless plug: PhishMe offers this), is a great way to gather metrics about user knowledge. The benefits from this are obvious. Training will be more engaging and you will be able to engage your human sensors beyond basic levels of security training.
Metrics that tell you which users are most knowledgeable about security can aid in incident response, if you encourage users to report potential security incidents or suspicious activity. If the IR team can prioritize reports from users who are known to be savvy security-wise increases the efficiency of response and remediation processes. Acknowledging users for successful reports provides positive reinforcement, and makes them feel like they are contributing to the overall security of the organization.
Even baseball managers (a notoriously stuffy, stubborn bunch) understand that collecting and analyzing data about players will help them prepare better strategies, and security awareness is no different – metrics will make your program more effective and impactful to the business.