SAIJE Thoughts

I can’t think of a witty title

Once you’ve determined the need for a SAS70 audit, the next key step is selecting an External Auditing Firm for your SAS70 engagement. The reason why you want this selection so early in the process is that this is not your typical external audit. In this case, the external auditor becomes a member of your team BEFORE the actual audit. They should help in determining the scope of the audit and the set of controls. Many detractors of the SAS70 audit state one of two things:

• A SAS70 audit is no good because the audited company determines their scope and controls so a lot is missed.

• A SAS70 audit is no good because the auditor only reports on what controls are present, there could be a lot that is missed.

By selecting an ethical and experienced SAS70 auditor, both these statements are false. While the company still is the final arbiter of scope and controls, the auditor as part of the team will highlight deficiencies in both places and can even include those items as part of the report. By doing this, your company will develop a stronger scope and set of controls.

Some may ask, “Why do this, it sets the bar higher?” That is a good question. Consider it like getting a degree. Not only does the degree count, but WHERE you get the degree counts as well. A Mail Order degree counts far less (if at all) as a degree from a certified institution. In addition, by having an auditor that is part of the team and has experience in SAS70, not only will you check the block in terms of customer needs, you also may improve your corporate operations.

 OK the short list of items to select an External Auditor:

• Must come from a CPA firm (or at least a CPA, but really you want  a firm)

• Should have several years at least of SAS70 experience.

• Able to provide references for SAS70 engagements.

• In their proposal, they should outline that they will help determine and/or review scope and controls or be part of the initial team before the audit period starts.  This addresses the issue I wrote about above and adds value to your SAS70.

• Should have some information systems and information security professionals on the team (CISA/CISSP/etc). While SAS70 is not a security audit, many of the controls selected relate to security. ISO 27001/2 (old ISO 17799) experience a plus.

• Must have Sarbanes Oxley auditing experience.

• COBIT experience/certification a plus.  (COBIT and ISO experience is good as these are common best practice control sets for IT Governance and Information Security)

Note that the selected auditor does not have to be the same external auditor you may already have on contract to provide services for your firm. In fact, many CPA firms may not qualify for the above items.

Obviously your company should interview more than one firm. In addition, finding the right firms to ask may be difficult. Search the web for firms. The big 4 all do these audits, but they could be cost prohibitive for smaller firms, many regional CPA firms also provide this service.  Ask other companies you do business with if they have contracted with an auditing firm for SAS70 engagements. Finally if you contract for third party services from another company (hosting for example), there is a good chance they have undergone a SAS70 audit and you may be able to find your auditor that way.

If all else fails, ask me, I have dealt with several reputable firms for SAS70 engagements.

Or  Edgar Allen Poe Teaches Risk Management

 One of the comments on my Much Ado About Mapping  post asked the following question, “…thoughts about if following frameworks actually leads to the most effective security…  I wonder why infosec hasn’t applied some rigorous scientific methods for risk analysis and reduction.”

My thoughts:

Frameworks when applied like a check list, are relatively easy when compared to rigorous, scientific methods. In the

absence in many organizations of dedicated security professionals, the checklist approach may be a relatively simple approach to improve security, but as with my mapping example, there is a danger when blindly implemented. Specifically, do you know what you are protecting or are you just following instructions. There is a quote to the effect of “You cannot manage what you don’t know you have” Check lists can be good - pilots use checklists before operating a highly complex machine, but I hope that they know what each step means before implementing. So the key to a framework implementation that actually improves security involves:

1. Knowing your beginning and end states and

2. Understand the actions you are taking.

 Of course the cynic in me cites the lazy man approach - are you going to do what’s already been done by someone else or are you going to study what you need to know?

In addition, the ability of your average professional to implement “rigorous scientific methods” may be rather limited as that is not their field of expertise. I think to implement something like this you would need a seperate class of individual to quantify the risk for the security professional to implement. And you do hear terms such as CRO. The areas where risk is relatively well defined (banking, insurance, etc) have whole areas dedicated to risk and they do it well because otherwise large sums of money would be lost. In the security field, I don’t think that many companies fully understand the risk in those terms, like banks and insurance companies do, although I think that is changing to some extent. 

There are actually some impressive scientific methodologies out there and good examples of quantitative risk analysis. One interesting one I came across is the Algebraic Specification of Network Security Risk Management  provided you are  mathematical genius. I am not. A simpler, and so in my case, more interesting post is on the blog Technology Reflections.

 Another key problem is the wide variety of variables that one would need to fully provide a quantified risk analysis. Many of us are simply limited in our capacity to run simulations that run mutiple independent variables that would allow for some sort of discernible outcome. There is software out there to do the work, but one still needs to understand exactly what they are measuring. A practical method to run multiple variables over many iterations is called the Monte Carlo method and the link has a brief explanation including the math behind it, although I prefer to use an excel plug in.

There are some decent processes that anyone can use for risk analysis. A process is provided within NIST publication SP 800-30 and the draft 800-39 and in the issue of Control from the ISACA that I reviewed in my last post there was some decent discussion of risk. I think a hybrid approach where one does the best one can on a qualitative and quantitative risk assessment and then implements a set of controls likely works well enough.

So I guess my thoughts are:

• Frameworks are a good means to implement risk reductions schemes, but it may limit your actual understandings of the risks you are actually reducing and without some sort of analysis you may be implementing more (or less) than what you need, particularly as one shouldn’t spend more on risk reduction than the cost of the actual risk.

• We just haven’t acheived critical mass yet in terms of understanding the impacts of the risks yet, which would push for a more cohesive set of risk practices and professionals like we have in the insurance and banking industries.

• Apply a best effort risk analysis before implementing a given framework or set of controls.

• Give me a checklist.

Comments welcome.