Comments on: Does Security Awareness Work (pt. 2)? It all Depends on What You Mean by “Work”

By: Randolph Smith

Randolph Smith — Wed, 14 May 2008 13:07:34 +0000

Consider that if a security awareness program is delivering useful results, then the target audience should be able to:
1. Know what is expected of them
2. Distinguish between acceptable and unacceptable behavior
3. Take appropriate actions

It is good that you point out that constructing valid experiments is not as easy as we would hope. Yet, good experiments are necessary to answer the most important question about an awareness program – Was it worth the effort?

Sometimes just asking the right questions gets useful answers to the first two elements. Whether or not people will do the right thing when confronted with a security challenge is the heart of the matter. Regardless of what they know, what people actually do in a given situation is the true measure of an awareness program.

If we are not willing to observe and measure real behavior, we will never know if the messages ever had the desired effects.

By: Gary Hinson

Gary Hinson — Wed, 14 May 2008 04:40:37 +0000

Personally, I’m interested in (a) promoting information security awareness, making people more aware of the security issues they face at work and at home, (b) modifying individual behaviours, for example taking security into account when doing things and hopefully avoiding overtly risky activities, and (c) making this a widespread change, in other words a cultural shift embedded in the organization. Measuring and demonstrating the effectiveness of the awareness program is a separate issue. Tests, surveys and trials (such as the self-phishing or pen test ideas you mention) certainly generate information and, if properly designed, can generate valid statistics, but there are many other ways of measuring and reporting, for example counting the increased number of calls to the IT help desk or information security team, or page views on information security’s intranet website, directly relating to a recent security awareness initiative. If you don’t mind spending some $$ to get the stats, you could even conduct behavioral assessments using observation and recording of employees in real-life or experimental conditions – there’s the whole field of behavioral and sociological sciences at your disposal. The bigger question, though, is why do you need stats? A truly effective security awareness program should be self evidently effective. There should be a buzz around the place when new awareness topics are covered. Managers, staff and specialists should be talking about the program, doing the things it is suggesting, and ideally coming up with their own ideas (good communications are not broadcasts but two way – like comments on blogs!). Security policies should be up to date and referenced. People should be using and updating security-related standards, guidelines and procedures. Employees should be getting feedback when they ‘do the right thing’ as often as they are chastised for doing something insecure. People should be looking forward to the next awareness topic, and keen to get involved in the seminars, competitions and other learning opportunities. Information security should be nearer everyone’s ‘front of mind’ than if no awareness program was in place. Perhaps MRI scans would give you the stats you seek!!

Kind regards,
Gary Hinson

By: Carnival of the Security Catalyst Community - April 22, 2008 | Scott Wright's Security Views

Tue, 22 Apr 2008 14:31:41 +0000

[...] (click HERE) This is the second post by Sam in recent weeks on examples of real security awareness campaigns, [...]

By: Scott Wright

Scott Wright — Tue, 22 Apr 2008 13:16:53 +0000

As with any awareness campaign, the act of communicating content itself is only a small part of a successful campaign or program. As Sam points out above, things like budget, frequency and topics are considerations that ultimately contribute to changes in an organization’s outcomes and business performance.

One of the things I believe is most important is creating “sticky messages”. A great book that helped me understand the essentials of sticky message content is “Made to Stick” by Chip Heath and Dan Heath. The subtitle is, “Why some ideas survive and others die.”

Without a memorable message that people can act on, success is unlikely, no matter how you measure it. With a good message, the effectiveness of penetration can be pretty obvious.