Disclaimer: The opinions of the columnists are their own and not necessarily those of their employer.
Kenneth F. Belva

Bejtlich and Business: Will It Blend?

As I read Are the Questions Sound? I laughed, I cried but CATS was still better….

Richard tries to compare the world of finance to the world of InfoSec after he takes the following notes on a Wall Street CISO’s InfoSec comments:

The present author was confronted with this list, exactly as it is, by the CISO of a major Wall Street bank with the preface “Are you security people so stupid that you cannot tell me….”

This particular CISO came from management audit and therefore was also saying that were he in any other part of the bank, bond portfolios, derivative pricing, equity trading strategies, etc., he would be able to answer such questions to five digit accuracy.

Richard’s post replies with his InfoSec thoughts on Business, Accumulation, Assumptions and Accuracy as ways of contrasting our field of InfoSec to that of finance. In his post he discusses InfoSec ROI, the Black-Scholes model, properties of money and more.

I showed Richard’s blog response to some friends of mine who work in financial risk management as well as equity sales to get other informed opinions. Our conclusions were:

1. Either the CISO does not know what he’s talking about at all or Richard clearly misinterpreted him
It’s most likely the later: Richard misinterpreted him. Richard’s focus on five digit accuracy is misleading. What the CISO is saying is that within finance, one has the ability to model the upper and lower bounds of risk quantifiably to the point of five digit accuracy. This is not the same as predicting a particular outcome with certainty. All the CISO was saying is that Wall Street has sophisticated models to help guide their financial decision making process: he’s asking, “Why can’t risk be quantified in the Information Security field like it can be in the field of finance?”

2. Information Security does not have an ROI and that security mechanisms are not business enablers
My friend in the financial risk department read Richard’s statement that “Security does not have an ROI” and he laughed. He commented, “Just let some hackers change some numbers in a banks financial system and you’ll see that security has ROI.” That’s a finance guy talking, not an InfoSec guy. Also, the paper I co-authored with Sam DeKay rebuffs the claim that “Only security vendors make money from security”: in addition, see my reply to Richard on security as a business enabler.

3. Some quotes are just wrong or nonsensical because they take the five-digit accuracy too literally instead of using modeling to understand risk :

  • “Assumptions make financial “five digit accuracy” possible.”
    • Actually mathematics make five digit accuracy possible: I can assume anything
  • “If financial five digit accuracy were possible, no markets could be sustained.”
    • 1/3 = 0.3333333333 – Hey! I’m past five digit accuracy and the markets are still going!
  • “If trading houses all figure out how to make money with five digit accuracy, their advantage is not going to be sustained because no one will want to trade with anyone else — they’re all want to take the same positions.”
    • See 1/3 point above… People take different positions because each assess that the future will play itself out in a different way: modeling and quantifying risk is just one expression of trying to understand what has happened and how what *may* happen will effect their market positions

Let’s return to the question I assume the CISO is trying to ask:

“Why can’t risk be quantified in the Information Security field like it can be in the field of finance?”

Here are a number of ways to respond:

  • InfoSec can be quantified but it is a young field and we do not have mature models
  • InfoSec can be quantified with existing models but is not worth the time and effort to adapt it to each environmental architecture (especially due to the volume of changes within a particular architectural landscape)
  • Quantified analysis is not as meaningful as qualified analysis when describing InfoSec events [Note: This is my personal belief and I think it’s one reason why DHS uses colors and words instead of numbers.]
  • Not all risk can be translated into numbers (unquantifiable risk)

My only question: after Richard moves to GE, will he blend?

8 Comments

  1. Richard Bejtlich Jul 13, 2007 at 11:14 am | Permalink

    Hi Ken,

    Thanks for your thoughts. Here’s a brief response.

    1. That is a good clarification — but again I contend that assumptions matter here.

    2. Apparently your friend doesn’t understand ROI either. Please ask him to describe why preserving integrity is NOT loss prevention.

    3. See point 1.

    Your unnumbered points are good too.

    I’m blending just fine, thanks. And no one I’ve encountered in security here believe ROI for security exists either.

  2. Kenneth F. Belva Jul 13, 2007 at 11:28 am | Permalink

    Every finance person I showed your article to said that information security has an ROI because it saves money from potential loss.

    Finance people usually discuss ROI in this way:

    “Acronym for return on investment. ROI refers to the amount of profits or savings a business will realize from any given use of money.”

    See:
    http://www.google.com/search?hl=en&q=define%3AROI&btnG=Google+Search

    ROI does not necessarily mean revenue derived from a particular activity (see definition above). Loss prevention has an ROI: investing $10 to prevent a loss of $100 has an ROI of $90.

  3. Richard Bejtlich Jul 14, 2007 at 3:16 pm | Permalink

    Good grief — did any of these “finance people” take any economics or business classes in college?

    Do you recognize that the so-called “definition” you cited is from here

    http://technology.findlaw.com/law-technology-dictionary/roi.html

    and that is a “law” site? Try looking at ALL of the other definitions — none of the others mention loss prevention as ROI.

    Next you’re going to tell me that these “finance” people think they “save” $10 when they buy a $100 shirt for $90. Keep following that plan and they’ll be millionaires no doubt!

  4. Kenneth F. Belva Jul 15, 2007 at 12:15 pm | Permalink

    Another blog post on security ROI pointed me to ISACA. Here is the ISACA .pdf link on ROI/ROSI:
    http://www.isaca.org/Template.cfm?Section=Home&Template=/ContentManagement/ContentDisplay.cfm&ContentID=28173

    In it the ISACA author(s) write: “Downtime assessment can provide an important postmortem analysis of lost productivity during a security incident. Productivity loss must also be considered in calculating the ROI of security solutions.”

    So according to ISACA, security solutions have an ROI. They ALSO have a ROSI. The next sentence after the one quoted above is:

    “IS auditors should be aware that there are number of ways in which lost productivity can provide meaningful estimate of risk exposure, any of which could be used to calculate ROSI.”

    I’ll continue to use ROI in the manner described above when is it appropriate to help facilitate and communicate information security decisions. When it no longer becomes appropriate to use ROI in terms of savings I’ll stop. It’s a pragmatic choice: it shows the ability to communicate effectively in the same language that is being used by others; and it shows open mindedness.

  5. Chris Jul 15, 2007 at 6:13 pm | Permalink

    In both the cost avoidance and the “value generation” scenarios, you are talking about a stream of cash flows.

    If you spend money on a security project, those flows will include the cost of the project and the expected benefits it brings by reducing other costs. Calculate the NPV of the flows under the “do the project” and the “don’t do it” scenarios, and whichever is higher is the one you go with.

    Clearly, calculating the cash flow out to five decimal places is preposterous, just like calculating the bump in sales an advertising campaign will bring is. However, nearly every firm in the country (even GE, I hear :^)) makes decisions about spending ad money.

    What is the big deal here?

  6. Kenneth F. Belva Jul 15, 2007 at 7:54 pm | Permalink

    @Chris

    Why security ROI touches a nerve with some InfoSec professionals is beyond me.

    Perhaps such a discussion is worthy of a bulletin board or wiki of some type for those truly zealous in one direction or the other in regards to InfoSec ROI.

    If so, I’d be more than happy to let them come to a consensus for our field that we can all standardize on.

    Until then I do not plan to spend too much time on InfoSec ROI as I have other projects to which I must attend.

  7. Richard Bejtlich Jul 15, 2007 at 10:32 pm | Permalink

    Chris,

    http://taosecurity.blogspot.com/2007/07/no-roi-no-problem.html

    If you still believe what you posted you should check back in with a business class.

    Ken,

    The “nerve” is my utter dismay when I see terms hijacked and distorted to serve an agenda, or to speak the language of a manager who doesn’t understand the scope of his job.

    I would be happy to be quiet when the term ROI is never associated with security again.

  8. Kenneth F. Belva Jul 20, 2007 at 6:00 am | Permalink

    Please see the following blog post with an email from Dr. Lawrence Gordon noting that Security ROI is possible:
    http://www.bloginfosec.com/2007/07/18/email-from-dr-lawrence-gordon-security-roi-possible-but-not-optimal-use-other-metrics/

    Thanks,
    Ken

Post a Comment

Your email is never published nor shared. Required fields are marked *

*
*