Security operations · 4 MIN READ · GREG NOTCH · DEC 19, 2023 · TAGS: Analyst report / Expel report / MDR / Tech tools
Research from SANS uncovers the ways orgs measure and assess their security programs. It’s safe to say that more work needs to be done.
Ask 20 different security leaders about the methods and tools they use to assess their security maturity and guide their programs, and you’ll get at least 20 different answers. Many factors come into play, including size of the organization, the roles covered by the security team, and the experience of team members, the threat profile of the business, and how security leadership reports to the C-suite and the board of directors.
While it’s difficult to quantify the different frameworks, tools, and techniques that security teams use, we wanted to get an idea of what that landscape currently looks like. Having that knowledge in hand will help us understand how security teams design and lead their programs, showing us where and how we can assist them. So we partnered with the SANS Institute to design a research study to analyze the wide range of security operations center (SOC) practices and create a report that outlines the current state of the SOC within many organizations.
The report is titled Frameworks, Tools and Techniques: The Journey to Operational Security Effectiveness and Maturity, and it’s out now.
In working with SANS, we specifically wanted to:
- Determine if frameworks are used to define, measure and assess SOC functions and, if so, which framework(s) are preferred
- Assess SOC metrics currently in use and the presence of any policies and training, as well as respondents’ sentiment regarding efforts to improve cybersecurity
- Capture respondents’ self-assessment process for their organization’s security program maturity and examine the security program components that contribute to maturity
- Learn if benchmarking is performed and whether KPIs are useful and effective in driving improvements in security processes
Some of the results we found were encouraging. Some were concerning. Take a look.
Lots of orgs are using frameworks, and lots of those orgs use the NIST CSF
The survey found that nearly 70% of respondents currently use a framework to help define and measure policies, processes, and controls, where only 22.1% don’t. The remaining 8.5% are unsure whether they currently use frameworks or not. We’d love to see that ~30% get more clarity and put a framework in place.
Overwhelmingly, almost three-quarters (74%) of respondents employing a framework use the National Institute of Standards and Technology Cybersecurity Framework (NIST CSF)—almost twice as many as the next top contenders (ISO 27001, NIST 800-37, and MITRE). These orgs likely prefer the NIST CSF for the same reasons we love it: it’s flexible, straightforward to measure against, and can help security teams get a sense of where they stand and what they should do next. As I said in my webcast with SANS’ Dave Shackleford, the reason people like the NIST CSF is that it has a beat and you can dance to it.
If you’re looking to get started with the NIST CSF, take a look at our “getting started” tool. And when NIST releases CSF 2.0 in 2024, we’ll release a new and improved version, too. Read our recent blog to know what to expect. Stay tuned.
Security teams use multiple metrics to measure and improve SOC maturity
Two-thirds of respondents currently use metrics to assess operational security performance. Just under 22% don’t, and another 11.8% aren’t sure. SANS thinks that could be due to them holding job roles with little to no exposure to security metrics. Still, though, I’d like to see that 34% have a better understanding of those metrics or get them into place. The old saying goes, you can’t fix what you can’t measure.
For those that are using metrics (fortunately, the majority), the top three used are security incidents (74%), vulnerability assessments (58.5%), and intrusion attempts (43.9%). Other popular metrics on the list include uptime, vendor/third-party risk ratings, and, of course, the mean-time-to-detect/resolve/contain numbers.
This is an area where we excel in supporting customers. We measure performance continuously, not only to ensure we’re providing strong outcomes, but also to identify areas where we can improve our own performance. Expel analysts regularly provide recommendations to customers to improve their internal processes, address gaps, and strengthen specific areas to improve their cybersecurity posture. And Expel Workbench™ also provides resilience recommendations to continuously improve, as well as dashboards that show how customers are improving their security posture and metrics over time.
The use of preventative measures could be a lot better
The survey asked respondents whether they perform cyber-readiness exercises on a routine basis. Sixty-one percent of respondents indicate that they do, around 30% don’t, and the rest aren’t sure. The respondents that do perform cyber-readiness exercises use penetration tests and tabletop exercises (tied at 73.7% each) and incident response testing (71.7%). Disaster recovery tests (56.1%) and red/blue/purple team exercises (38.6%) round out the top five responses.
Thirty percent don’t do cyber-readiness and about 9% don’t know if they do. That’s alarming. We’d hoped those numbers would show a lot more companies maturing their readiness programs, since cyber-readiness is critical for identifying (and ultimately rectifying) weak spots and gaps in security controls. We encourage customers to conduct cyber-readiness exercises—such as tabletop exercises—often, as the threat landscape is constantly evolving.
Expel provides a fun exercise for teams who need a little bit of guidance getting started with tabletops with our Oh Noes! game. Oh Noes! is a tabletop exercise in the style of Dungeons & Dragons and Shadowrun that guides players to role-play through cybersecurity incidents specific to their org. It’s for security teams who don’t have a formal tabletop exercise in place and provides them with a fun and potentially familiar framework.
What I’ve outlined above is really just scratching the surface of all the findings from the SANS research report, so go ahead and download Frameworks, Tools and Techniques: The Journey to Operational Security Effectiveness and Maturity. And if you’d like to learn more about how we can help you address your own SOC effectiveness, drop us a line.
