Security operations · 4 MIN READ · DAN WHALEN AND PETER SILBERMAN · AUG 1, 2022 · TAGS: MDR / Tech tools
Building a security company and a corresponding product suite is a lot like building a house. If the foundational building blocks you’ve created are solid, you should have a sturdy, reliable structure. And if you decide to expand your house in the future, those great foundational building blocks will help you build out faster while maintaining the top-notch structural integrity that you developed in the first place.
Since we started our journey at Expel, we’ve believed in this. That’s why we’ve invested in creating processes and tech that ensure our Expletives aren’t burning out as we grow and that our customers don’t see a decline in the level of service they expect.
One of the features we recently built and released that helps us do all of this is something we call Alert Similarity. What is Alert Similarity, how did we get the idea for it, and how does it benefit our team of analysts (and, of course, our customers)?
How it started
Our bots, Josie and Ruxie, process millions of alerts each day, and thousands more show up on our analysts’ screens. Given this volume, it’s no surprise that many look similar to one another. For example, in a given week, our analysts typically review a few hundred alerts related to suspicious logins. If you see enough of this activity, you start to recognize common patterns and similarities (incidentally, humans are pretty good at pattern recognition).
So we asked ourselves: Is it possible to teach Josie and Ruxie the same trick? What if we were to think about alerts and their corresponding evidence as documents? Could we compare similar “documents” and apply what a human did with one “document” and suggest or recommend a next step?
Our hunch: comparing alerts to past activity (and corresponding outcomes) can provide valuable situational awareness.
Imagine that our analysts identify a security incident on a Monday morning. Chris in accounting fell victim to a phishing attack ー their credentials were stolen and had to be reset. On Thursday night, a different analyst is reviewing a new alert for Alice that looks similar. The situational awareness of what happened a few days ago and what the outcome was helps the analyst make the right decision for Alice’s incident.
This is a simple example, but here’s where it gets really interesting: Imagine you’re a security provider employing a distributed team of analysts who monitor many different environments, nearly a hundred different security technologies, and respond to many different kinds of activity. The pattern recognition that analysts are innately good at starts to break down with constant context switching ー you can only expect a person to commit and recall so much information. By teaching our bots to recognize and surface similar alerts and their associated outcomes, we can let our analysts focus on what they do best: judgment and relationships.
The result? We can improve quality and scale at the same time.
This is how our Alert Similarity experiment was born.
What is Alert Similarity, anyway?
In short, Alert Similarity is a feature of Expel Workbench that helps us make high quality decisions at scale. We accomplish this by applying document similarity techniques to the security alerts we process – think of this as teaching our bots to recognize patterns and similarities between alerts. As a result, Ruxie can helpfully surface relevant historical context during alert triage, including a recommended action based on the decisions we’ve made for similar activity in the past.
Fast forward to today: What started as an experiment quickly turned into a unique and valuable Expel Workbench feature.
How Alert Similarity benefits our analysts
There are three specific ways that Alert Similarity benefits our team of analysts:
#1: Instant suggestions based on past similar alerts
One of the biggest benefits of Alert Similarity is that our analysts now receive dynamic suggestions in near-real time about new alerts based on similar alerts that our team has seen previously. Not only does this give our analysts more context when evaluating the right next step to take in an investigation, but it also gives us the benefit of personalizing our response to a specific type of alert in a customer’s environment. For example, a customer might want PUP / PUA alerts categorized as policy violations instead of unwanted software. Our system then automatically learns to treat future, similar alerts as policy violations, automatically “suggesting” this to the analysts who manage that particular customer environment.
#2: Enhanced quality control
We pride ourselves on being at the forefront of quality control in security operations. This is incredibly important because, as we add new products and offerings, we want our analysts to learn from their mistakes as we scale our business. With Alert Similarity, we can look for clusters of similar alerts and identify when they result in different outcomes. We then push them into our quality control review process, where we review the alerts with a second set of eyes to determine if the correct actions were taken and whether there are any process improvements to be made.
#3: New and improved detections
With Alert Similarity, we now have a way to compare data we collect from customer environments against historical alerts that we know turned out to be true positive incidents. This capability helps with detection research and engineering. For example, if we compare a new event (or even an event from the past that wasn’t deemed alert-worthy originally) to our known true positives, it can give us the insight we need to determine if we should write additional detections for customer environments.
How it’s going
Our Alert Similarity feature launched in February 2022. The feedback we’ve heard from our security operations center (SOC) about it so far is positive and early metrics show that the feature is working the way we’d hoped – it’s offering our analysts valuable suggestions about new alerts in the moment they’re being reviewed, and it’s saving our team time as they make decisions about the right next steps in an investigation. Take a look at our first read-out:
We’re also sharing some of these same metrics with our executive leadership team and board.
Interested in creating your own version of Alert Similarity?
In just a few months, our analysts – and in turn, our customers – are already benefiting from the introduction of Alert Similarity.
That doesn’t mean we’re done. In fact, we’re continuing to work on ways of improving the feature and using similar techniques to drive other features and use cases.
Want to learn more about exactly how we built and tested Alert Similarity, and get some tips on how you might be able to develop something similar to benefit your own SOC? You’re in luck – we created a technical walk-through.
