ROSI: Security Returns?

Two of the more controversial topics in information security are return on security investment (or ROSI) and the related subject of security metrics. I will talk to ROSI in this column and metrics in the next one.

There are a number of opponents to the ROSI approach. One is Jos Pols who, in his recent article “The Fallacy of Information Security ROI” in the February 2008 issue of the ISSA Journal (membership required to access link resource), claims that one cannot have a return where there is no income. In my opinion, this is an overly restrictive view of the meaning of the word “income.” The avoidance of potential losses redounds to the bottom line, as does revenue, so that a cost saving is a return on an investment, just as much as a corresponding revenue enhancement would be.

Pols prefers to use the term “insurance” in referring to spending on security. He asks the question “How do you value what was not lost?” While I am not saying that it is easy to measure the losses from a potential security breach, I do believe that one can come up with workable estimates of the magnitude and probability of losses and then calculate an expected loss number as the product of estimates of the size of loss and probability of occurrence. Better than point estimates is the inclusion of probability distributions and Monte Carlo simulations. You can read how this may be achieved in Douglas Hubbard’s excellent book, How to Measure Anything: Finding the Value of Intangibles in Business (John Wiley, 2007).

Donn Parker, for whom I have the highest regard, also argues against the ROSI approach. In an article in the May 2006 issue of the ISSA Journal (membership required to access link resource) with the title “Making the Case for Replacing Risk-Based Security” he states that “ … security based on risk management, risk reduction, and risk assessment is a failed concept.” He argues that it is not possible to measure the probability or the magnitude of infrequent but very damaging security events. But Hubbard specifically addresses IT security and provides confirming evidence as to the viability of the probabilistic approach from his work at a major government agency. On page 47 of his book, Hubbard writes: “When we say that security has improved, we generally mean that particular risks have decreased … a reduction in risk must mean that the probability and/or severity (loss) of a certain list of events decrease.”

So what is the answer? In one sense, detractors such as Pols and Parker are correct in questioning the feasibility of measuring what might be lost as a result of a security breach. It is neither a simple nor obvious task and requires some measure of training (or “calibration,” using Hubbard’s term). However, I think that it is not only possible, but imperative, to come up with reasonable estimates of the probability and magnitude of potential losses in order to achieve reductions in risk. Not doing so precludes an important method from the risk manager’s toolbox. I address this whole issue in my chapter on ROSI in the book Managing Information Assurance for Financial Services (IGI, 2007). In the chapter, I demonstrate that, if one does in fact derive reasonable estimates of potential gains and losses of various approaches, one can then go on to optimize the mix of security measures to be taken.

Risk reduction and avoided losses are not easy concepts to apply, but that does not mean that they should be discarded. A relatively small effort in this area can lead to major benefits and savings. Look into it. You’ll be pleasantly surprised.

4 Comments

  1. Alex Mar 10, 2008 at 11:01 am | Permalink

    Excellent lead article, Warren! If this is indicative of the quality we can expect here, then I’m all for the new venture.

    RE: “it is not possible to measure the probability or the magnitude of infrequent but very damaging security events.”

    Interestingly enough, Protiviti (I’m not affiliated) just released a report that does just that. On page 3 of their “Flash Report” on the Societe Generale – they offer the following:

    “The frequency and magnitude of the losses listed in the chart on page 2 implies that a loss of more than US$1 billion is occurring approximately every 18 months. And of these 11 incidents that each exceed $1 billion in trading losses, more than half (six) have been attributed to a “rogue trader.”

    I’m very glad to see that you linked to Hubbard, btw.

  2. stacy Mar 11, 2008 at 12:30 pm | Permalink

    While I agree with pretty much everything in your article, I would still argue against ROI simply because it is the wrong term. I prefer the term Cost/Benefit, the same factors go into calculating it; you just avoid arguing over whether or not there is any “return”.

  3. David Gutiérrez Mar 11, 2008 at 7:09 pm | Permalink

    i’m sorry to say, but I have yet to see one of that so called “workable estimates of the magnitude and probability of losses and then calculate an expected loss number as the product of estimates of the size of loss and probability of occurrence” that today are nonexistant or useless; note that I’m not saying that we don’t need them, but they’re not at the maturity level we need. Perhaps we should invest more time into developing the models we will use to start collecting useful data which, in a few years, will help us to do all this work.

  4. Alex Mar 12, 2008 at 3:13 pm | Permalink

    @David Gutiérrez

    There was recently a thread on the securitymetrics mailing list where some companies were refuting that exact assertion. This is also why I’m betting Warren links to the Hubbard book, it’s an excellent read on the subj.

One Trackback

  1. […] Link to what he writes here. […]

Post a Comment

Your email is never published nor shared. Required fields are marked *

*
*