Is Risk Assessment a Snake-Oil Discipline?

I do a lot of risk assessments, 2-3 a month. They are obviously not going to identify every possible threat, vulnerability, and countermeasure. In most cases, many of the risks that are detailed in the report are known by the IT department of the organization. But some are not. And I try to be careful and point out that I am not giving them an exhaustive list — even when this is precisely what they ask for. It is simply impossible.

Time after time, I have found that one of the big reasons for bringing in an outside consultant to perform a risk assessment is precisely because the IT department knows that they have certain risks but want to document the need from a third-party. This helps them justify to management taking on projects that help strengthen security measures.

I’m a big fan of Marcus and his wacky opinions. I think he’s right on a lot of things, but at times overstates his case to prove a point. I think this is one of those times. Like a great many things, risk assessments can be good or bad. But that’s not a reason to stop using them.

I’ve no clue the precise way in which computer security professionals use risk assessment, but it’s unlikely to be anywhere near as precise as what insurance companies do.

The insurance companies use past history to predict future probabilities. They’re NOT predicting whether YOUR car will be stolen (or in an accident). They’re predicting the probability that it will be stolen (or in an accident). It’s not an educated guess so much as a scientific conclusion. If I see a coin come up heads 6 times in 9 tosses, I might bet that it is slightly biased in favor of heads and bet that it comes up heads on the next toss. I’d call this an educated guess. If I see the coin come up heads 512000 times in 1000000 tosses, I’d expect similar results in the next 1000000 tosses. This is a bit more solid. And if I merely predict that there will be fewer than 600000 tosses in the next 1000000 tosses, it’s an extremely safe bet (EXTREMELY). Insurance companies have the advantage that things don’t change all that quickly (usually) in the areas they are insuring. In City A, a certain percentage of cars will be stolen in a typical year. The percentage in City B is smaller, so the insurance premiums are lower (or at least that PART of the premium is lower). If more cars are stolen in City B one year, the insurance company will lose money there (or will have smaller profits), and they will adjust their premiums for next year (taking into account not only the recent events in City B but the trend they are a part of). The world of Information Security may change too fast for everything to be accounted for in a risk assessment (and the sample size may be too small to do more than make educated guesses). Additionally, insurance companies know the value of what is at risk. If this car IS one of the ones that is stolen, the company will be out a maximum of D dollars. This isn’t to say that a very unlikely but widespread event can’t be VERY costly for insurance companies. (There are plenty of companies that sell no new homeowner policies on the gulf coast.)

You have to consider who is going to be consuming the risk assessment. Ranum is a very technical person and I’m sure he’s thinking that the consumers of the report will be very technical as well. The fact is that many consumers of RA’s will be professional managers at the VP or C-Level who will in turn have to condense the report down to one or two PowerPoint slides for the Board of Directors meeting. I believe you are on target with using RA’s to bring awareness to management. The presentation must be tailored to the proper group. In my experience it takes a VP or Officer’s financial signing authority to bring a consulting firm in and the COO or CFO is not going to be interested in hearing about SQL Injection or specific vulnerabilities. They want to hear a general exposure report and the compensating controls in place to reach an executive decision as to how much residual risk they want to live with. A CTO, CISO, or CIO might want to hear what Ranum is suggesting, but most RA’s are being driven by the SOX compliance division under the CFO or through the Legal or Compliance departments, not by IT and the final report will be tailored to suit the department paying for the engagement.

I’ve written about this quite a bit, but the main problems following the Parker/Ranum mindset are:

1.) Neither really understand probability theory – so neither understand risk, and neither are qualified to be authoritative on the subject. Carl M’s mindset about probability theory is typical, even Ranum himself claims to ‘have had a couple of statistics classes in college’ so he knows risk analysis can’t be done. Um, bullcrap. When people say these sorts of things to those trained in stochastic methods, or in front of those who use them everyday in real science, they don’t seem to realize how stupid it makes them look.

2.) Neither really understand risk management (that circle thing graphic you have there is case in point), so neither are really qualified to be authoritative on the subject.

3.) I don’t think they realize how absurd the alternative they propose is. Take that quote from Parker. OK, Donn, how are we going to understand what those ‘practical’, ‘doable’ security things are? Oh, that’s right, it will be risk management. It’ll be done by you licking your finger and sticking it in the air to make the decision, but it’s still (albeit sloppy) risk management based on your (not mine, my business, or my bosses) risk tolerance.

Me? I’d rather have decisions based on logical frameworks (i.e. not risk=vulnerabilityXcontrol/impact) with real probability theory rather than something based on how Donn Parker or Marcus Ranum or some committee in Europe – who has no idea what my threat landscape looks like – try to make blanket decisions for me because they are some flavor of information security demagogue.

Alex writes: 1.) Neither really understand probability theory

WTF? 3 semesters of stats with straight A’s and what do you think, I was asleep? That’s what I get for trying to be modest.

“stochastic methods” sounds really fancy but any of us can get into throwing terminology about. How’s this for a better argument:

If what you’re claiming to do is science then your methods must have predictive power. Indeed, one of the key attributes of science is the ability to measure and quantify, then predict with some degree of accuracy that a change in one place will produce a result in another. “Risk assessment” in computer security is more like vigorous hand-waving than science, if you look at it from that perspective.

Scientists often use statistical methods to look for correlations that can be explored experimentally. We all know (right?) that statistics can’t predict the future. They can, however, help you identify the knobs that you can turn in an experiment to see whether you’ve found the right knob. Computer security isn’t doing that; practitioners who are fond of risk assessment models jump from a measurement to a conclusion – one of the most basic mistakes in statistics and science – without being able to tie things back to a theory with predictive power. Put another way, if risk management were actually a science, one could measure the impact of a firewall, or a policy with more precision than simply “it would be nice.”

The use of statistics for risk management of problems in banking and underwriting works because you’re dealing with very large data-sets that are well-understood and quantified. Unlike in computer security, for example, there are excellent and detailed data-sets about the relative ages of drivers in automobile accidents – in fact the insurance industry has such detailed data-sets that they can correlate between owning a 2-seater vehicle and the likelihood of speed-related payout. But we can’t come close to doing that with computer security for two reasons: 1) The numbers just aren’t there. We’re left with handwaving B.S. like “80% of attacks come from the inside” – a nonsense number someone pulled out of their butt on a friday, which has achieved credibility through repetition among security practitioners. 2) Security is a dynamic environment. When you’re up against an intelligent enemy, your ability to fall back on how past statistics indicate the likely future is extremely impaired. This is why military commanders seldom get away with telling the enemy, “based on past performance, we win.” Our enemy innovates. Consequently, a Mac might be “safer” today and less safe tomorrow. Meanwhile, teen-age male drivers are not likely to get dramatically worse or better in the next decade.

Statistical models allow you to make some guesses about likely long-term trends and to game them to overcome short-term fluctuations. That, in a nutshell, is how insurance works. It’s also how Las Vegas works. In computer security, it is those short-term fluctuations that concern us.

some flavor of information security demagogue.

And you, sir, merely come off as defensive of your dogma.

mjr.

You have to consider who is going to be consuming the risk assessment. Ranum is a very technical person and I’m sure he’s thinking that the consumers of the report will be very technical as well. The fact is that many consumers of RA’s will be professional managers at the VP or C-Level who will in turn have to condense the report down to one or two PowerPoint slides for the Board of Directors meeting.

That’s actually one of the key points in my argument against the usefulness of risk assessment. In short, it becomes an exercise in bullshittery aimed at management that is above the detail level. Information security practitioners use the folderol of risk assessment to manipulate the perception of those managers by tweaking the inputs to give the desired outputs that will “help management get it.”

I am not averse to manipulating clueless senior executives – but I just cut to the chase, when I do it.

mjr.

My “mindset” about probability theory? I was just trying to help Daniel see that there MIGHT be a difference between what insurance companies do when insuring a car or home and what information security professionals do when putting together a risk assessment. I made no pretense to know anything about information security (so I probably should have kept out of the discussion entirely). I was just imagining him stopping by my office and asking if I had any views on this. It’s not clear (to me) that I wrote anything more than a VERY BRIEF illustration of the kind of analysis that goes into insurance calculations. Everything I said about possible differences with information security was in an “it MAY be that” sort of way — meant to allow Daniel with his understanding of information security to decide if there is any real difference.

Other than my simple (and simplistic) example, I don’t see that I really wrote anything about probability theory. I mention this only in case anyone thought it was intended to be more than it was.

I suppose that the one thing I said that could be called a “mindset” is my attitude that tiny samples give less useful information than very large samples (and that’s more a FACT than a mindset – I’d hate to think that anyone was using statistics and didn’t have that mindset). Calling a prediction based on a tiny sample an “educated guess” and a prediction based on a very large sample something “more solid” was meant to help Daniel understand that what insurance companies do is “more solid” than what most people would mean when they use the term “educated guess”. I think that insurance companies would cringe at the use of the word guess, but this is semantics. I was merely trying to point out that there are educated guesses and there are EDUCATED GUESSES.

Carl M,

I didn’t read what you wrote that way. For example:

“If I see the coin come up heads 512000 times in 1000000 tosses, I’d expect similar results in the next 1000000 tosses. This is a bit more solid. And if I merely predict that there will be fewer than 600000 tosses in the next 1000000 tosses, it’s an extremely safe bet (EXTREMELY).”

This implies a necessary level of precision that not only doesn’t exist in most real world situations, it’s not necessary for accuracy in the vast majority of cases. THAT is the mindset that is built on a frequentist approach and is, frankly, the bane of decent analysis and what causes people to suggest we make UNeducated guesses and faith-based decisions.

In retrospect, I was WAY too harsh, and I apologize. But I hope you’ll understand that it’s only because I’m so sick of people suggesting that the frequentist approach is the only valid statistical approach, and subsequently recommending that the industry regress back into witch-doctorism (and continue the manipulation Ranum discusses, above) rather than have a hand at quantitative analysis and scientific method.

Oh, and the fact that “educated guess” and “probability statement” are not synonyms and most people act like they are. “Guess” is one of those words InfoSec folks like to throw around without thinking about what it means, addressing the root cause of their uncertainty, and/or accounting for it in their data.

@Carl

Good points about the maturity of the information within the insurance companies.

@Alex

Carl is a math professor; I’m not sure you want to go down the path you’re on. I can assure you that there are only two options with respect to your disagreement — either 1) you’re right and you’ll agree after more discussion, or 2) you won’t agree because it’s you that is wrong. :)

At any rate, I’d very much like to hear your views on risk management. I say this because you have out of hand dismissed two of the most experienced and respected infosec professionals in the world on the topic. I see you left a website. I’ll check it out.

Feel free to expound on your comments though — I’d love to see what true metrics you think are possible to be used in this interesting field. I’m actually reading a couple of books on infosec metrics right now related to eliminating the need for “guessing” when judging risk.

Anyway, nice to have you here and I hope to hear more about your ideas.

@Daniel

Relax. Apart from his trying to read between the lines of what I first wrote, any disagreement between Alex and me appears to be semantic. (Keep in mind that I remain ignorant of the application of risk analysis to information security and have said nothing about it.)

@Alex

I am usually pretty careful about what I say. For example, I didn’t say that the precision in my 1000000 coin toss example was “necessary”. It was meant to be an extreme example taken in contrast with my 6 of 9 example (which was fairly extreme on the low end of the spectrum). As I already said, I was using these examples to help Daniel see that not all educated guesses are created equal. (My post wasn’t meant to be a treatise on probability theory.)

Interestingly, I checked the dictionary definitions of “educated guess” and “guess” before making my second post in this thread. We might as well avoid semantic arguments if possible.

Guess: an opinion that one reaches or to which one commits oneself on the basis of probability alone or in the absence of any evidence whatever.

Clearly, leaving out the “or in the absence of any evidence whatever”, we are talking about this sort of guess. Unfortunately, the word “guess” conveys the “in the absence of any evidence whatever” to many people. So, one could certainly argue against the use of the word guess in this context. But, as I said, this is semantics. (Side note: this is the same issue that scientists must deal with when talking about things like the THEORY of evolution.)

@Carl,

So it would seem. My assumptions were WAY to hasty. Thank you for being understanding. RE:

Guess: an opinion that one reaches or to which one commits oneself on the basis of probability alone or in the absence of any evidence whatever.

To me, the difference in belief statement vs. guess has to do with the quality of evidence (data). Probability suggests that there is some evidence, no matter how small or useful. Guess suggests that there is no evidence at all to prove or disprove the probability statement.

Semantic, to be sure, but “guess” carries such a negative connotation as to render a probability statement as useless. IMHO, that’s unfair.

“If what you’re claiming to do is science then your methods must have predictive power. Indeed, one of the key attributes of science is the ability to measure and quantify, then predict with some degree of accuracy that a change in one place will produce a result in another. “Risk assessment” in computer security is more like vigorous hand-waving than science, if you look at it from that perspective.”

It seems to me that you’re confusing scientific method with “science” there. Science has no predictive power on it’s own in some anthropomorphic manner, it is simply a study about a body of knowledge. Scientific method is the process used to measure and analyze in the context of a model or theory in order to make a belief statement, no? Then, in turn, that belief statement can be tested for accuracy and/ or precision (the ‘predictive power’ you’re claiming, I guess).

I’ve suggested before that you read ‘Jaynes – Probability Theory, the Logic of Science’. I’m happy to hear you have an aptitude in traditional statistics – frankly, you’ll probably be in a much better position after you read it than I will ever be – based on your capability to digest and use advanced subjects.

Bottom line – I believe if we continue to work through what we’ve got now, establishing the “laws” of security and risk (from some of your writing and inspired by http://www.overcomingbias.com/2008/01/beautiful-proba.html) we, in the end, will be no worse off than Paleontology, much of Astronomy, and similar disciplines. At worst, we’ll continue towards something akin to the best of meteorology or economics. Which is a hell of a lot better than the witchdoctory we do now.

“Computer security isn’t doing that; practitioners who are fond of risk assessment models jump from a measurement to a conclusion”

I’m not sure I understand what you’re saying here. Are you saying that scientists who use probability theory shouldn’t be using their posteriors to draw conclusions? That probability theory and scientific method has no use in decision making? Or that the current methods for risk assessment methods you’re familiar with suck it?

If it’s the latter, then you’ll hopefully understand that’s why I get so upset when industry pundits write these sorts of statements. You’re making a huge generalization here. We (myself and those who couple of dozen of folks who are trained and doing these things as part of their jobs) just might have something different, no? In probability theory we might say that you aren’t accounting for the uncertainty you have around your observational data :)

So understand how we perceive the arrogance of the statements you’ve made in the past – we’ve vetted our approach against dual PHDs in stochastic methods from various universities. We’ve done our homework, our methods are in use and we are seeing the value. I’m not claiming they are perfect, it’s a new approach that needs more and more vetting. But when someone with a lot of personal brand equity and small sample size error comes and craps out a big blanket statement that denigrates what we think actually has value? Yeah, we’re going to be a little defensive.

“The use of statistics for risk management of problems in banking and underwriting works because you’re dealing with very large data-sets that are well-understood and quantified. Unlike in computer security, for example, there are excellent and detailed data-sets about the relative ages of drivers in automobile accidents – in fact the insurance industry has such detailed data-sets that they can correlate between owning a 2-seater vehicle and the likelihood of speed-related payout. But we can’t come close to doing that with computer security for two reasons: 1) The numbers just aren’t there. We’re left with handwaving B.S. like “80% of attacks come from the inside” – a nonsense number someone pulled out of their butt on a friday, which has achieved credibility through repetition among security practitioners.

Again, I’m not sure I understand the points you’re trying to make here. Are you saying that it is impossible to account for the noise in data? Then someone needs to tell the rest of the scientific and assurance worlds they can’t use noisy data either. Are you saying that traditional actuarial/statistical approaches don’t work because they can’t account for noise in data? Then I kind of agree. Are you saying that you’ve read a bunch of Taleb and you buy into everything he says? That you don’t agree with Cox? Then we’ll have to agree to disagree on an epistemic level, and discuss the value of analysis and modeling in more practical terms.

But you should understand that our approach to risk analysis initially came from a very, very large Insurance company. So when you make large generalizations about “why underwriting works” I do question the usefulness of your prior information. We know the beautiful chaos that is business decisions based on probability first hand – the size of the estimates, the uncertainties accounted for. My familial background is strong in Bayes Theorem (my Dad being early on in NMR) so claiming that probability theory cannot help science cope with similarly noisy data isn’t going to resonate, either (sorry for the pun). If anything, I’ll continue to assert that we should pursue analytic functions using the best that science and probability theory have to offer us – even if my current model turns out to not be the best one mirroring reality. We should account for our noisy data in the same way that they do.

You’re a self-accliamed Feynman fan. Seek what he would do with noisy data and uncertainty. I don’t think he’d as carelessly dismiss the stochastic approach as a useful tool as you have. http://www.springer.com/west/home/statistics?SGWID=4-10128-22-16878799-0

To that extent we have the tools in probability theory. We know how to best use those tools thanks to scientific method. We don’t all have the laws and subsequent models. Our data could be of better quality. But having industry pundits crap all over the fledgling efforts that have been made and telling us to just give up – yeah, I guess that’s what history proves we should do.

“2) Security is a dynamic environment. When you’re up against an intelligent enemy, your ability to fall back on how past statistics indicate the likely future is extremely impaired. This is why military commanders seldom get away with telling the enemy, “based on past performance, we win.” Our enemy innovates. Consequently, a Mac might be “safer” today and less safe tomorrow. Meanwhile, teen-age male drivers are not likely to get dramatically worse or better in the next decade.”

Paraphrasing a statement you made above: We all know that analytic functions don’t predict the future, right?

The purpose of analysis is to help people make decisions (synthesis, as Kant puts it). Any probability statement is a belief statement about current state or some past/future state based on evidence. It is a belief that is hopefully tested by scientific method (implementation of Bayes and Jaynes’ desiderata is said to be an analogue of scientific method). Expressed in that belief is not only the probability that it is wrong, but the uncertainty (key) surrounding the data and the concluding belief statement. If your problem is with the accuracy of current belief statements from lousy risk models – I can’t agree more. If you’re saying (as it appears to me you have) that it’s impossible to create accuracy using scientific method and probability theory – I’m going to continue to argue with you until you or someone else has strong evidence that Bayes/Cox/Jaynes’ theorems are wrong.

But even if you do end up creating a revolution in probability theory and show Cox or Bayes to be wrong, then hopefully you will arrive at a conclusion that the journey is worth the effort. I’ll personally celebrate because we’ve arrived at something more useful.

“And you, sir, merely come off as defensive of your dogma.”

Guilty to my own shame. But aren’t we both? The only question is which dogma is beneficial – one that suggests we apply scientific method to models and theories until we have some level of accomplishment – or the dogma that insists upon outright rejection and suggests we just put our faith in their personal knowledge instead?

I’d rather use science than shamanism with you (or Donn Parker, or the PCI council, or whomever) as the shaman – even as much as I’ve been taught to respect your past work and knowledge by people we both know. And that’s what pisses me off Marcus. You’re smarter than this. You’d never stand for someone telling you to drop science and put your faith in them to be your priest. And frankly, if you were in my position I doubt you’d be as charitable.