SAIJE Thoughts

Or  Edgar Allen Poe Teaches Risk Management

 One of the comments on my Much Ado About Mapping  post asked the following question, “…thoughts about if following frameworks actually leads to the most effective security…  I wonder why infosec hasn’t applied some rigorous scientific methods for risk analysis and reduction.”

My thoughts:

Frameworks when applied like a check list, are relatively easy when compared to rigorous, scientific methods. In the

absence in many organizations of dedicated security professionals, the checklist approach may be a relatively simple approach to improve security, but as with my mapping example, there is a danger when blindly implemented. Specifically, do you know what you are protecting or are you just following instructions. There is a quote to the effect of “You cannot manage what you don’t know you have” Check lists can be good - pilots use checklists before operating a highly complex machine, but I hope that they know what each step means before implementing. So the key to a framework implementation that actually improves security involves:

1. Knowing your beginning and end states and

2. Understand the actions you are taking.

 Of course the cynic in me cites the lazy man approach - are you going to do what’s already been done by someone else or are you going to study what you need to know?

In addition, the ability of your average professional to implement “rigorous scientific methods” may be rather limited as that is not their field of expertise. I think to implement something like this you would need a seperate class of individual to quantify the risk for the security professional to implement. And you do hear terms such as CRO. The areas where risk is relatively well defined (banking, insurance, etc) have whole areas dedicated to risk and they do it well because otherwise large sums of money would be lost. In the security field, I don’t think that many companies fully understand the risk in those terms, like banks and insurance companies do, although I think that is changing to some extent. 

There are actually some impressive scientific methodologies out there and good examples of quantitative risk analysis. One interesting one I came across is the Algebraic Specification of Network Security Risk Management  provided you are  mathematical genius. I am not. A simpler, and so in my case, more interesting post is on the blog Technology Reflections.

 Another key problem is the wide variety of variables that one would need to fully provide a quantified risk analysis. Many of us are simply limited in our capacity to run simulations that run mutiple independent variables that would allow for some sort of discernible outcome. There is software out there to do the work, but one still needs to understand exactly what they are measuring. A practical method to run multiple variables over many iterations is called the Monte Carlo method and the link has a brief explanation including the math behind it, although I prefer to use an excel plug in.

There are some decent processes that anyone can use for risk analysis. A process is provided within NIST publication SP 800-30 and the draft 800-39 and in the issue of Control from the ISACA that I reviewed in my last post there was some decent discussion of risk. I think a hybrid approach where one does the best one can on a qualitative and quantitative risk assessment and then implements a set of controls likely works well enough.

So I guess my thoughts are:

• Frameworks are a good means to implement risk reductions schemes, but it may limit your actual understandings of the risks you are actually reducing and without some sort of analysis you may be implementing more (or less) than what you need, particularly as one shouldn’t spend more on risk reduction than the cost of the actual risk.

• We just haven’t acheived critical mass yet in terms of understanding the impacts of the risks yet, which would push for a more cohesive set of risk practices and professionals like we have in the insurance and banking industries.

• Apply a best effort risk analysis before implementing a given framework or set of controls.

• Give me a checklist.

Comments welcome.

This entry was posted on Monday, March 3rd, 2008 at 5:33 pm and is filed under COBIT, ISO, IT Governance, NIST. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.