In late December of 2022, the email arrived from the auditor. A deep breath and a mouse click later and the word “Congratulations” jumped from the screen. Backblaze had passed our SOC 2 Type 2 examination with no exceptions noted. The examination covered both our Backblaze B2 Cloud Storage service and our Backblaze Computer Backup service.
It was the end of an important milestone that had begun years ago, and we wanted to take a few minutes to look back and share the lessons we learned along the way as we created and built a successful SOC 2 certification program. Whether you’re interested in how we made the journey, or how your organization might follow in some of our footsteps, we thought the things we learned along the way were worth sharing.
Background
SOC stands for System and Organization Controls, with SOC 2 being a voluntary compliance standard for service organizations developed by the American Institute of CPAs (AICPA). The standard is based on the Trust Services Criteria (TSC) which specifies how organizations should manage and protect customer data. The specific criteria you will use are based on your organization’s business operation and practices. An outside auditor reviews and tests your practices and procedures to ensure you are complying with those criteria.
“When it comes to selection of criteria and implementation of controls, there is no one-size-fits-all approach to identifying the scope, as it is critical for a company to first understand what controls are applicable to their products and services, and how they would fit within their very own environment.”
—Evangeline Cheung, VP and Associate General Counsel, Backblaze
Given the uniqueness of the criteria each organization will use, we are not going to get into the mechanics of SOC 2 criteria selection here. Instead we will focus on the process you can expect as you go through your SOC 2 journey.
The Importance of SOC 2
Over the past several years, many organizations have started asking, and even requiring, their vendors to be SOC 2 compliant and verified by a third party auditor to ensure the vendor is providing a service which adheres to a defined set of industry best practices for data protection.
While Backblaze utilizes data centers which have a current SOC 2 report and/or other similar certifications such as ISAE 3402, ISO 27001, or ISO 20000, we are seeing an upward trend of customers and prospects asking for a SOC 2 report covering the Backblaze service and platform. This makes sense—while everyone is comfortable their data is safe in the data centers we use, they want to ensure our cloud storage platform and its associated applications are also safe. To address their concerns, the compliance group under our legal department kicked off our SOC 2 program.
Getting Started
There are many tasks you do at the beginning of any project, such as conducting a kickoff meeting, creating a project plan, and so on. We’ll focus on a handful of things you will need to do for your SOC 2 project.
Executive Buy-In: Okay, every significant company project needs this, but given the resources and support you will need for project success, this is a must. You will not be able to fly the SOC 2 certification under the radar of your CEO or CFO.
Stakeholder Buy-In: While a SOC 2 evaluation affects nearly every department in your organization, technical operations, information technology (IT), and engineering are the most impacted. Without buy-in from those departments to provide the necessary resources to create, document, and follow the required policies, procedures, and controls, you will not get far.
Seek Out Experience: Ask key stakeholders and others in their sphere if they’ve been through a SOC 2 or a similar certification before. Experience in the process is helpful, and having that knowledge with respect to your infrastructure and internal processes will provide you with meaningful inputs and feedback as you define your policies, procedures, and controls.
Build a Strong Core: Forming a core team with the key stakeholders is one of the most impactful steps in the SOC 2 process, as it helps provide visibility on the status of the project and identifies any roadblock issues.
“Having alignment cross-functionally through communication and transparency is key to the success of our SOC 2 program. Not only is getting buy-in from leadership key, but it is just as important to ensure that any process change is transparent to the rest of the organization and that input from process owners are thoughtfully considered as new controls are being introduced and implemented.”
—Evangeline Cheung, VP and Associate General Counsel, Backblaze
Outside Help Is Okay: Another source of help to consider (and budget for) is an outside consulting firm. This can be very useful, especially if your organization is new to the SOC 2 framework. Choose a consulting firm that understands and supports your objectives and is familiar with your business, preferably with references for having worked with similar firms.
Choose an Audit Firm That Knows Your Business: The audit firm you select is one of the more important decisions you’ll make. We reviewed and interviewed several firms before selecting Schellman as our auditor for our SOC 2 efforts. They had completed hundreds of audits for SOC 2, ISO 27001, PCI-DSS, and so on, and they had worked with Lumen Technologies (CenturyLink), Iron Mountain, and others on the data center side, and Litmus and others on the Software as a Service (SaaS) side. That breadth and depth of experience was a great fit for us.
Tools of the Trade: A SOC 2 examination is a large multi-departmental project. While some organizations have managed the project using spreadsheets, the complexity often leads you to look at solutions built specifically for SOC 2 and similar certifications. The category is known as Governance, Risk, and Compliance (GRC) with offerings ranging from a few hundred dollars to $50K+ a year. For Backblaze, we wanted a system that could be used for multiple types of certifications, that way we could leverage the work we did for one certification towards the next one. If you are new to SOC 2, you may want to start out with a simple, purpose-built solution. If you do, ensure that your data can be exported as needed should you decide to upgrade later on.
Don’t underestimate how long the “getting started” stage will take. Activities like selecting an auditor, choosing a consulting firm, and selecting your tools can consume months. So, start your preparation work early!
Your Path to SOC 2 Type 2
One of the decisions you’ll have to make early is where to start. The three basic steps are as follows:
- SOC 2 Type 1 Assessment
- SOC 2 Type 1 Audit
- SOC 2 Type 2 Audit
If your company is new to SOC 2 and audits in general, then starting with an assessment makes the most sense, but where you start is up to you. We’ll dig into each of the steps below.
SOC 2 Type 1 Assessment Preparation
The assessment step starts with you educating your auditor about your organization. Typically the auditor will provide you with a long list of questions about your organization, how it operates, what equipment you use, what type of policies and procedures are already in place and so on. You need to be brutally honest here as many downstream actions will be based on this information. For example, based on the information you provide, the auditor will work with you to define the scope of the assessment; that is, the systems and services that will be reviewed. If you leave an important system out and the auditor finds it later, that’s—well, it’s not good. On the other hand, giving the auditor everything, whether it matters or not, can lead to an expansive, overly intrusive audit.
You’ll also spend much of your preparation time understanding the SOC 2 framework and determining what evidence you are going to use to address the SOC 2 criteria. As noted previously, the criteria and controls which apply to your business will be unique to you, although basics like risk management, disaster preparedness, encryption practices, and so on will apply in varying degrees to everyone.
We used the word evidence above; you will become very familiar with that term in the process. Evidence is the proof you need to provide the auditor to prove that your organization does indeed meet the criteria that is applicable to your organization. Evidence comes in many forms: policies, procedures, tickets, scripts, and so on.
You’ll find some evidence is useful in helping comply with multiple criteria, and you’ll find that some criteria can take 10 or more pieces of evidence to address the issue at hand. Understanding the mapping from evidence to criteria and keeping track of the evidence you have and where it applies are two of the biggest challenges in your SOC 2 project.
SOC 2 Type 1 Assessment
The actual assessment will typically be a couple of weeks long. The auditor will review your evidence and interview key employees about that evidence. Think of an assessment as a dress rehearsal. You should be ready, but the process is flexible enough for you to ask questions and fix things along the way.
The two most important learnings of an assessment are first, to determine the sufficiency of your evidence, and second, to determine how your company’s employees do in the audit process. We’ll talk more about sufficiency in a bit, because the second point is often overlooked. For example, if during the interviews your IT manager is a wall-flower—or worse, combative—in front of the auditor, you have some work to do beyond getting the evidence right.
Evidence sufficiency is a subjective term that ranges from the concrete to the creative. Sufficiency is also related to context or use. For example, a list of employees with hire dates is sufficient when you need to demonstrate who was hired in the last three months. But if the list does not have terminated employees, it does not help identify who should have access to your systems. Do you want two lists or just one? The assessment period is the time to pose and answer such questions.
After the assessment is complete, you’ll get a report outlining how well you did. It should contain a pass or fail on each of the points of focus within each criteria group. At this point, you’ll need to address how to fix the failed items and how you are going to move forward towards an actual audit.
SOC 2 Type 1 Preparation and Audit
The SOC 2 Type 1 audit is based on a date in time. The audit is all about proving to the auditor that:
- You have all your policies, procedures, and controls in place.
- These policies, procedures, etc, are sufficient to meet the criteria you’re addressing.
- That you have a defined cadence of when various controls will occur.
- You have documented how you will prove you have exercised the various controls, or you have actually taken the action and have documentation.
As an example, you have a checklist in place for new hire onboarding. The checklist has sufficient inputs from all departments involved in bringing a new person onboard. Each quarter the human resources (HR) manager will review all new hire checklists to ensure compliance with the controls in place. You have evidence of the completed HR manager’s most recent review via a ticket in your service management or other activity tracking system.
Preparation for a SOC 2 Type 1 audit is about cleaning up any missing or incomplete items (policies, procedures, controls, etc.) found in the assessment, and taking a deep breath before you plow forward towards the audit. The assessment itself can be exhausting, especially if such a task is new to the organization and the people involved. This is a good time to assess whether you had the right employees to answer questions on the subject at hand. Were they too senior or junior? How well did they answer the questions? If you need to make changes or coach up your folks, now’s the time.
You should have at least a quarter between completing the assessment and starting the SOC 2 Type 1 audit. This gives you time to test your controls, at least the quarterly ones, and have them ready as evidence for the audit. The more “we just finished that yesterday” policies, procedures, and untested controls you have when starting the actual audit, the less prepared you will feel. The auditor may also want to dig deeper into those items to make sure they do, in fact, address the criteria appropriately and you are ready to act on them when the time comes. In short, the more evidence you have that demonstrates you have done a given task, the better off you will be.
Type 1 Versus Type 2
While the SOC 2 Type 1 audit is about a point in time, the SOC 2 Type 2 audit is an evaluation of how well you document and maintain your controls over a specified evaluation period. The evaluation period is at least six months and usually no more than one year. And it is not a one-and-done thing. You will be audited at least annually to maintain your SOC 2 Type 2 certification.
The difference in the evaluation period between a Type 1 and a Type 2 is the primary reason not to jump from an assessment directly to a Type 2 audit. If you jump straight into Type 2 and you have missing or insufficient controls, you won’t know until the Type 2 audit itself, and it is too late at that point. This could lead to an exception or worse for a given criterion.
You may be able to pass a SOC 2 Type 2 examination with a limited number of exceptions, but they will be listed in the SOC 2 Type 2 audit report for all to see. You will be able to respond to any exceptions found, with your response being part of the final report. Still, it is not a good look regardless of the circumstances. Doing the SOC 2 Type 1 audit first allows you to determine whether your controls are sufficient before placing them into practice. This will minimize potential exceptions in your SOC 2 Type 2 audit that are based solely on insufficient controls.
Staggering Reviews
One mistake that is easy to make is to have all your quarterly reviews done on the same date each quarter, or even in the same month. This is especially vexing to everyone when the reviews are piled into the last month of a fiscal reporting period. Spread out the reviews of the various controls. They can be done anytime as long as they meet the cadence you specified. It’s perfectly fine to have a quarterly review on the 15th of February, May, and so on.
One way reviews can be scheduled is by using the GRC application we mentioned earlier. The nice part of using the GRC application here is that the review can be tied directly to the control, which in turn is tied to the criteria you are attempting to satisfy. The evidence gathered in the review can be captured (or linked to) in the application, then, at audit time, the review and supporting documentation are readily available.
SOC 2 Type 2 Evaluation Period and Audit
For a SOC 2 Type 2 audit you will have to demonstrate that you performed and recorded the actions specified by the policies, procedures, and controls you devised to meet the SOC 2 criteria over the evaluation period. Here are a few examples:
- You have a requirement to document the code changes, additions, and deletions for each production product build. A build typically occurs once a week, but not always. You have a change management system which documents everything you need and includes any sign-offs you captured as part of your process. You also document the weeks when there was no build. The auditor will ask for your build documentation for several different weeks during the evaluation period. This could include weeks you did not do a build. How many different weeks and which weeks they will ask for is unknown until the audit itself.
- Your risk management plan is required to be reviewed by the risk management officer once a quarter. You’ll want to have a tracking ticket showing the action was completed and, within that ticket, a note or other correspondence that discusses the findings along with any follow up actions from the review.
- Your risk management plan is required to be reviewed each year by your executive staff to ensure all appropriate risks are being surfaced and addressed in the plan, and that all risks are correctly rated. The review is documented per your risk management procedures. If the date for this review falls outside of the evaluation period, make sure you have a previously completed review ready to show the auditor if asked. Saying, “We haven’t done one yet,” is not the best answer and will only cause the auditor to dig into your risk management policies and procedures to ensure you will be ready when the time comes.
At its core, the SOC 2 Type 2 is about demonstrating your ability to consistently enact and follow industry best practices across your organization over a period of time and then demonstrate that to the auditor.
Consistency Matters
During the initial SOC 2 Type 1 assessment you will meet the actual auditors who are doing the audit. There are usually two or three auditors, each focusing on a different area where they have some expertise. As you work with each of these folks, you need to decide if this is the auditor you’ll want to use in future audits, including SOC 2 Type 1, and so on. After completing the assessment, the auditor will have a decent understanding of your organization and its quirks and capabilities. Swapping out auditors or even audit firms between the different SOC 2 phases means you’ll be starting from nearly ground zero each time.
The only downside to wanting to use the same auditor for each SOC 2 audit is you may have to wait for them to have a future hole in their schedule to conduct the next audit. Still, the consistency gained is worth the wait if, each time, you can have the same auditor with prior knowledge of how your company works.
Summary
The entire SOC 2 process, from the initial assessment through annual SOC 2 Type 2 renewals, adds rigor and consistency to many of the processes and procedures you already have in place. It also helps you identify deficiencies and correct them along the way. You don’t have any deficiencies you say? Well good on you, but keep an open mind as you go through the process—just in case.
Another Beginning
Thanks for joining us as we celebrate our first SOC 2 journey. In the end there was little tomfoolery, no bloodshed, and no one got lost under a mound of paperwork. Hopefully there were a few nuggets of useful information that can help you along the way on your own SOC 2 odyssey. Of course, as this SOC 2 Type 2 journey ends, we start a new one, as each year we will be audited to ensure our continued compliance. Onward.
Open Sourcing the IPO