Understanding Vendor SOC Report Controls and Evaluating Results

Reviewing a SOC report can feel like a daunting task, especially if you’re unsure of what parts of the SOC report are considered key components, what best practices to follow, how to assess the controls, evaluate results and why vendors write the control activities the way they do. The best way to tackle such a colossal task is, of course, to start at the beginning.

The Key Components of SOC Report Controls

First, let’s look at the key components of SOC report controls and where they originate from — the auditor or the vendor: 

1. Control objectives. Control objectives and activities are determined by the vendor. Their purpose is to assist the vendor in achieving a secure environment. For SOC 2’s, control objectives and activities are designed to satisfy the required common Trust Service Criteria (Security, Availability, Confidentiality, Privacy and Processing Integrity).
   
   
2. Control testing. Testing of controls is performed by the audit firm. The firm designs and conducts tests through interviewing, documentation review or inquiry (interviewing key staff), in addition to observation (watching a task be performed), inspection (reviewing reports and other documents) or less often re-performance of the activity or computer assisted testing.
   
   
3. Test Results. Are the controls working as designed? The two most common results are either “no exception noted”, meaning that the control is working, or “exception noted”, meaning the control did not work as designed each time it was used.

How to Properly Evaluate Vendor SOC Report Test Results

Lucky for the users of the reports, the actual testing of the controls is conducted by the audit firm, but it is the report user’s responsibility to review and interpret the results of the testing. 

Step 1

Review the control objectives and activities to verify that they’re applicable to the service(s) you consume from that vendor, while also reviewing the adequacy of the controls. 

Pro tip: Make sure that you’re reviewing the correct SOC report, as some vendors have multiple SOC reports for various products. 

There are guidelines for control objective and activity design and, if necessary, auditors will assist in the creation of controls. Ultimately, the depth of the objective is determined by the vendor, but keep in mind: Not all control activities are created equal. 

Let’s look at an example of why the strength of seemingly similar activities may vary:

Control Activity Version #1 

“Administrative access to the company’s local area network (LAN) is authenticated via user account and password, and logical access is removed at termination.”

Control Activity Version #2 

“Administrative access to the company’s LAN is controlled by Windows Active Directory, requiring unique user IDs and passwords. Complexity settings require passwords to be at least 12 characters long. Password history is set to 24. Passwords must be changed every 30 days and users are locked out after five failed login attempts. Logical access is removed at termination. Active Directory access is automatically updated and terminated employees are removed each day.”

While both control activities are referring to “Logical Access,” the second option provides more detail and depth, hence providing you with a better understanding of the controls the vendor has in place (as a side note, reviewing control activities can also provide insight and assist you in designing a more comprehensive and layered approach to your own internal controls).

Step 2

Once you have completed the initial review of the controls, you now need to look for exceptions. 

Questions you should ask yourself during this step include: 

1. Were there findings from control tests? Many SOC reports contain findings from control tests. Most exceptions are instances where the control did not function as designed each time it was used.
2. Are there duplicate exceptions? SOC 2s often reuse control activities to satisfy multiple common criteria. Due to this, one instance of a control not functioning properly may cause multiple control activities to have the same exception. Take this into account when looking at the total number of exceptions within a report.
3. What is the impact of the exception? Potential for unauthorized access? Violation of internal procedures? Potential for a separation of duties violation? Does it even impact the part of the service you use?
4. Are there mitigating controls in place? An exception is negative, but they do not always result in heightened risk. A scenario is outlined below.
5. Did management provide a response to the exception? Management should provide responses to control exceptions explaining what occurred, whether any mitigating controls were in place, and how the control activity has, or will be adjusted to properly function in the future.

Using the example of the control above:

In the SOC report for ABC company, there was one exception surrounding logical access and the result of testing was: “Terminated employee was not removed from XYZ application at time of termination.” 

Management provided the following response: “Although the terminated employee was not removed from XYZ application, access to the application requires authentication to the LAN through Active Directory, the terminated employees access was automatically removed from Active Directory through the termination process, as such access to XYZ application cannot be accessed.” 

In this example, the vendor has mitigating controls in place: The automated Active Directory termination process. So, although the impact is potential for unauthorized access, after reviewing management’s response and taking the mitigating controls into consideration, the residual risk is probably pretty low. 

This process needs to be applied to EACH and EVERY exception in the report. 

Possible Audit Outcomes for Multiple Exceptions

In the event there are multiple exceptions surrounding a specific control or combination of controls, the audit firm will state in the assertion that there is a “significant deficiency” and expand what control(s) the deficiency(ies) applies to. 

• Significant deficiencies typically result in a “qualified opinion” of the report, which is an auditor's opinion that the controls are fairly presented, with the exception of a specified area. Unlike an adverse or disclaimer of opinion, a qualified opinion is generally still acceptable to most auditors.  

• Material weaknesses are the worst form of exceptions. A material weakness will result in either a qualified opinion or adverse opinion. An adverse opinion is a professional opinion made by an auditor indicating that a company's statements are misrepresented, misstated, and don’t accurately reflect its performance and health. To break it down further, a material weakness means that a misstatement could occur and your confidence in your vendor could be in question.
  
  When reviewing the exceptions, make sure you ask yourself the questions we covered above and break down each exception just as we did in the example.

Evaluating a SOC report can be intimidating to say the least. While there are many parts of the report you’ll need to focus on, paying close attention to the key components above will prove beneficial to your organization. Understanding your vendor’s strengths and potential weaknesses will ultimately support your organization in the assessment of your relationship and help you move forward with either confidence or caution. 

Get a breakdown on the different terms and definitions that you need to know when evaluating vendor SOC reports. Download the dictionary.