(adapted from a presentation I did at the 2011 Oceania CACS Conference)
As a customer, sometimes it’s useful to know the things you should be generally concerned about in terms of auditing, i.e. what are the normal areas to be concerned about, what should trigger red flags etc. – because the reality of auditing a service provider is that typically, they know you are coming, and you don’t have the ability to review material in the same way as a formal audit, or internal audit would.
1. SLA’s: Review these carefully. In particular, pay attention to the exclusions, the areas of applicability, how they will be measured, and the penalties assigned. As mentioned in other posts, one of the key things about SLA’s is that a good many service providers don’t see them as terribly binding, and are very optimistic.
2. How exposed are you to failures within the Service Provider infrastructure? i.e., can your services still run if their AD server fails, are you depending on Single Points of Failure of the network infrastructure, are you reliant on their DNS systems, load balancers, mail routers etc, even if you haven’t specifically decided to use these services? Are these services covered under your SLA?
3. Where and how is data hosted in the service? Are systems backed up, and do they have off-site secure backup?
4. What is the stability of the company? Are they acquisitive, are they ripe for takeover, are the services you are using core to the company? How is the share price, and recent advice to the market?
5. Do they have security policy, and is it compliant with ISO 27001? Do they have any evidence of operation of the policies you can vet? (note, many service providers may legitimately not provide you with logs or details for confidentiality reasons, but it helps to ask)
6. Data Centre – do they run their own, and if they do, what controls are in place around it, and can you inspect the site? If a third-party provider, still worth seeing if you can site visit, but otherwise you can ask for the SLA with the DC provider, and make sure it’s reputable via point 4.
7. What are the notification arrangements in the event of a security, or other important incident? Is this 24×7 (if you need it) and if so, are your staff ready to take that call?
8. Is the Service Provider dependent on any third-parties (typically at least network providers). If so, how does the flow-on SLA from that provider look, and are the arrangements in place to meet your SLA? Is the Service Provider liable if a third party doesn’t meet their SLA? Do they have backups and alternate arrangements to support failure of third parties?
9. Make sure to review the proposed service in the vendor selection process, and ensure the requirements are relevant to the service being provided. Some companies have no choice due to regulation, but uninterested or overly protective internal security or internal audit teams can come in with stiff requirements at times, well beyond what is justified for the service.
10. Make sure that the delivered service is still up to scratch after the contracts have been signed, and service has been deployed! In a lot of cases, I’ve seen initial vendor selection criteria quite tough, for a service that dropped most of the important things (like IDS, firewalls, backups, DR environments) because they were too costly in the end.