Security Awareness Evaluation - Don't Fall for the "Streetlight Effect!"

A policeman sees a drunk man searching for something under a streetlight and asks what the drunk has lost. He says he lost his keys and they both look under the streetlight together. After a few minutes the policeman asks if he is sure he lost them here, and the drunk replies, no, and that he lost them in the park. The policeman asks why he is searching here, and the drunk replies, "This is where the light is." ⁽¹⁾

You have no doubt heard this joke before. It so clearly illuminates (pun intended) a certain psychological effect that it is actually named after the joke. ⁽²⁾ The “Streetlight Effect” is alive and well in cybersecurity, especially in the realm of social engineering and security awareness training.

Good and effective social engineering training includes the topics of mobile device security, browser exploits, cloud hygiene, personal information and social media hygiene, USB exploits, wireless network scams, cybersecurity responses and much more. Defense in Depth applies to security awareness just like it does to hardware, software and operating systems.

So what is the problem?

A disturbing trend in the cybersecurity market is to assume that email phishing click-through rates give an accurate picture of an organization's overall security awareness posture.  I believe this is simply not true. Phishing click-through rates only tell you how your employees are reacting to phishing emails, not to the broad scale of other social engineering exploits and con games. At best this metric falls short of giving a useful evaluation of an organization’s cybersecurity readiness; at worst it gives a false sense of knowledge and overconfidence, providing a huge blind spot that will likely someday be exploited.

How did this happen?

This problem has arisen because of the Streetlight Effect. It can be quicker and, in the short term, cheaper to buy or subscribe to software that measures email phishing click-through rates rather than implement broader spectrum social engineering evaluations. Making matters worse, some security compliance procedures accept phishing click-through as a sole measure of security awareness posture and some vendors of click-through evaluation software try to propagate this misunderstanding to their own benefit.

The simplicity of making quick and easy “magic metrics” by using click-through measurements is appealing and thus quick to gain buy-in, but it gives you a dangerously small-scope look at your security awareness.

Using tools to evaluate your organization’s phishing click-through rate is both extremely valuable and important. The issue is that we must be aware of the scope of each tool set that we use.

Solutions

One of the first steps to having a competent security awareness posture and an effective social engineering defense is to obtain targeted security awareness training that covers the entire spectrum of social engineering exploits, utilizing modern adult education principles and science. Quality security awareness training is literally the best bang for the buck you can get in cybersecurity. ⁽³⁾⁽⁴⁾

Targeted training puts the right kind of topics and case studies in front of the correct audience, whether it is C-level employees, information workers or IT specialists such as administrators or developers. Adult learning science makes the training enjoyable and greatly aids the attendees in knowledge retention and application of the material.

Even without a full social engineering metrics program in place, we know that security awareness training is an effective tool. Even generic training evaluated on partial-scope metrics return a 37-fold return on investment. ⁽⁵⁾

Cadre Information Security offers a series of new security awareness training options that are audience-targeted, implement adult education science, use broad-spectrum topics, utilize a custom library of case studies and are efficiently delivered.

The second step is to acquire effective big-picture metrics and test your full social engineering security posture. Cadre has a number of services and products that are designed for any sized organization in almost any market.

References

1. David H. Freedman(2010). Wrong: Why Experts Keep Failing Us. Little, Brown and Company. ISBN 0-316-02378-7.
2. Kaplan, Abraham(1964). The Conduct of Inquiry: Methodology for Behavioral Science. Transaction Publishers. p. 11. ISBN 9781412836296
3. C.J. Kelly (2006) Awareness Trumps New Security ToysComputerworld p44
4. Howard Solomon (2016) Best spending value is on security awareness It World Canada 382124
5. Maria Korolov (2015) Does Security awareness training even work? CSO IDG Communications