6 Theories of Probability and 6 Reasons Why They Matter to ISRA

Similarly, when testing personal or intersubjective probability estimates, the auditor could request evidence that the person(s) providing those estimates have been sufficiently calibrated so that their estimates fall within, say, a 90% confidence level. If the estimator(s) have not been calibrated, then the auditor could reject those portions of the ISRA based upon the uncalibrated personal or intersubjective probability estimates.

(4) The theories clarify the scope of different interpretations of probability. Again, the frequency approach defines “probability” in the context of a long sequence of events; it is controversial whether the frequency interpretation can even be applied to a single event, such as this server, this application, this vulnerability. Indeed, the problem of the single-case probability was one of the reasons, if not the reason, why Popper originally developed the propensity theory.

(5) The theories provide insight into the common worry about ISRA. When critics of risk-based security talk about the “lack of actuarial data” for calculating the Annual Loss Expectancy (ALE) for information security incidents, they are what Ian Hacking calls “frequency dogmatists:” they are implicitly presupposing that the frequency theory is the one and only way to understand probability (Hacking 2000, p. 140). As the above discussion should make clear, however, the frequency theory is not the only game in town.

(6) The theories can be used in a complementary fashion. There is no need to adopt a “one-size-fits-all” approach to interpretations of probability; one can instead take an eclectic approach and use different interpretations can be used in different contexts (See Hacking 2001, pp. 140-141; Gillies 2000, pp. 180-205). For example, one approach would be to use the frequency interpretation where feasible, but then use the personal or intersubjective interpretation (with calibrated experts) when that is not feasible.

References

Alberts, Christopher and Audrey Dorofee. Managing Information Security Risks: The OCTAVE Approach (Boston: Pearson, 2003).

2 Comments

  1. Russell Thomas Sep 8, 2010 at 10:23 am | Permalink

    Great post, Jeff.

    One thing I’ll add is to counter the criticism that ISRA relies on *predictions* of the future, which is another way of saying “knowledge about the future”. Most InfoSec people, in their gut, feel that such knowledge is unattainable or infeasable.

    But ISRA is really not about predicting the future or having highly certain knowledge about the future. Instead, it’s benefit is to help us ORGANIZE OUR UNCERTAINTY. It’s the systematic treatment of uncertainty and ignorance in all it’s forms, with a goal of promoting continuous learning and adaptation.

    Russell Cameron Thomas

  2. Jeff Lowder Sep 8, 2010 at 2:10 pm | Permalink

    Thanks, Russell. I’m glad you liked the post!

    Regarding the issue of ‘predicting’ the future, I think I agree with your point, but I would word it in a slightly different way. I would say that risk analyses do make ‘predictions’ about the future, but these predictions are hedged in various ways. For example, personal probabilities and intersubjective probabilities represent our degrees of belief (and, accordingly, our uncertainty) regarding various information security-related hazards. Additionally, as my discussion of single-case probabilities hopefully makes clear, frequency probabilities typically don’t make a prediction about a single event. On the other hand, estimated relative frequencies do … estimate the actual relative frequency in the real world, and hence the corresponding ‘actual’ frequency probability. Thus, for example, an ISRA may not provide an inductively correct argument for concluding that this web server will be attacked at this time, but it may be able to show that some system will be attacked at some time during a given time span. In that sense, I would say that ISRA does make predictions. This does not deny what I think is your point, however, that the criticism of ISRA falsely assumes that ISRA is committed to making a series of predictions about single events.

    Jeff

Post a Comment

Your email is never published nor shared. Required fields are marked *

*
*