Security operations · 6 MIN READ · DAVE JOHNSON · SEP 14, 2023 · TAGS: MDR
Logs are a necessary and useful component in any cybersecurity practice, but when and how you use them can significantly change your security outcomes.
🎵JavaScript… snap… snap…
JavaScript… snap… snap…
JavaScript… snap… snap…
JavaScript… snap… snap…
You put the malware into my hosts.
You make my alerts sky-high, when the campaign starts
JavaScript into my RAM (yeah yeah)
Goes a knock knock knock til my agents fail
But somethings pinging me, something ain’t right
Expel told me what you did last night.
They left me sleeping in my bed, I was dreaming,
Because I wasn’t worried about you for once instead!🎵
Okay, before I continue butchering this ‘80s classic, let’s talk about something real quick.
LOGS! (No. Stop. Go back to ruining the hits, please?)
Whether you love them or strongly dislike them, logs are a necessary and useful component in any cybersecurity practice. But when and how you use them can significantly change your security outcomes in drastic ways—ways that many of us, myself included, are only just starting to wake up to in recent years.
Like many of you, I come from the old school, when firewalls were primordial and computers screamed at you as they connected to the internet. BBSs (bulletin board systems) were the Facebooks of their day, and WiFi was just a twinkle in Hedy Lemarr’s eye. The periodic monolith parties were pretty legit though, I must say.
In those days, security had very little strategy. We didn’t know what we were protecting, let alone what the threat surface really looked like. If you want an idea of what security was like, go check out The Cuckoo’s Egg by Cliff Stoll from your local library. In it, Cliff—a career astronomer—tells the story of how in 1986 he used his knowledge of physical constants and the accidental invention of computer-based honeypotting to figure out how to trace a malicious threat actor logging into the computers at his lab (and others).
To add some sense of scale, a total of about 400 disparate machines were involved in this compromise. The activity was alerted initially by a 75-cent accounting error. Think about how hard it is, even with all the right tooling, to find that many compromised hosts in the average environment today. Cliff essentially did this with a dot-matrix printer readout and some physics. (And if you’ve already read the book, here’s an update that Dr. Stoll did at SANS DFIR a few years back.)
This is what we imagine that situation could have looked like, with today’s technology and Expel.
(Click image to enlarge)
Cliff had “zero budget, zero expertise, and zero mandate” to do security. Nobody did. He did the best he could with what was available, and in many cases, I’d argue we’re still approaching the problem in a similar fashion. Most of us don’t have everything we need for the totality of the work in front of us, and that’s often the nature of defense. The attacker usually dictates the rules of engagement and we respond, doing our best to shore up defenses and detection methods. Or we modify our approach in anticipation of their attack.
The importance of log data
More recently, we’ve collectively succeeded or failed based on the quality and quantity of our log storage solution. We used logs for everything because it’s all we had. So we did what anyone would do in that scenario—we created tools to either generate better quality logs, or handle them more easily, or derive value from them in a more consistent fashion. The more stuff you have to dig through, the better and worse your security strategy. This is what’s meant by “double edged sword.”
But logs weren’t designed with security alerting in mind. They were designed for auditing or development tasks, and using them for alerting creates the utilization and functionality challenges for triage, detection, and response that we’re all pretty familiar with. The perspective I had until a couple years ago was, “this is the way we’ve always done it, and it’s better than doing nothing. No method is perfect.” Which is technically right on all counts, if not for one thing: there’s a more modern approach that delivers much better outcomes.
Let’s borrow another famous quote from Dr. Stoll:
Data is not information, information is not knowledge, knowledge is not understanding, understanding isn’t wisdom.
What good is log data if you can’t use it quickly or effectively? And even if you could, would it provide the context you need or would that have to be built separately as part of the analysis? The quality of your outcome depends on your ability to use the raw materials and tools. But there are limits to even what raw ability can compensate for. And then, to bring us back to one of the original premises of this article, when and how matter a lot, too.
A colleague and I recently concluded that using logs for initial alerting was akin to using archaeology to stay apprised of the daily news. Artifacts can give us a ton of detail, of course, and that can be useful for constructing complex models of past activity and predicting future trends. But the time needed to derive value on an artifact, and the appropriate time to use it, may not be during emergency triage or a defensive scenario where you need the absolute latest information in the most consumable fashion. Time and place matter.
That core concept is one of the many reasons why Expel consumes security signal across your security stack the way we do. API-based alerts have the detail that responders need in a much more contextual and immediately consumable fashion. They’re also faster to consume than logs and are less prone to signal loss or detection failure when a log format changes in transit or due to vendor updates. This translates into fewer false positives and puts more time and freedom back into SOC analysts’ hands so they can better assess what happened and tell that story in a way that’s immediately useful. We reclaim the time we’d have otherwise used for maintaining, analyzing, and updating log-based detections and put that back into our service in the form of more relevant detail and faster response times (with a near-zero false positive rate). We’re confident this is the future of security analysis, at least from a signal perspective.
Let’s say for a moment you agree and believe it will work for your organization. You’re probably thinking, “this is cool, but there’s still a ton of useful things in logs.” We agree wholeheartedly. This is where time and order of operations is paramount in extracting the most value from the data you have, and where “applied wisdom” shows its real value.
Doing something in a “new” way doesn’t mean you have to change everything about how you do it. Often, when you’re learning something new, you’re reusing old concepts that you already have firmly ingrained to help you master the new ones. Expel is no different.
The first part of what we do with this newly refined, modern API-based triage approach to security alerts is stop escalating false positives to customers. Because of the choices we made about the approach and SOC design, we have the time to properly review alerts before we determine if they are actually malicious and require escalation to you. By reducing the low-fidelity noise, just by nature of our model, one of the most common things we hear from our clients is that they experience a drastic reduction of False Positives, along with a reduction in “non-contextual” analysis, right after implementation. (Which only takes a few hours/days depending on customer preference.)
The logs help power the “evidence” part of the investigative process, where we remove false positives, assess the scope, and write the story of how the incident happened. The initial alert or related alerts have a lot of rich detail, but not always everything we need to determine root cause or total scope of impact. We make it easy for our analysts to query your log sources, your SIEM, and your individual tools to get the detail necessary to describe everything that happened.
See the difference? We’re moving up the analysis process stack. We’re still doing the same kinds of things in regard to log-based analysis, but that’s not where we start. By starting with the higher fidelity security alert, we get the reclaimed time and applied wisdom to determine when to switch to enrichment, aka “OG” mode. Now the quality of signal-based intelligence tells us the best places to conduct that type of deep analysis rather than trying to apply it to everything everywhere, all at once. We also have even better tools designed to help us get the logs we need as the analysts need them. This saves our customers quite a few conversations with their retained or in-house incident responders, allowing them to spend more time preventing this problem from happening again.
Just to be clear, there’s definitely a place for SIEM in this world. We can use your SIEM, if you have one, like the other data sources we collect from, and in some cases we can use custom detections you’ve already built out. See this post for more details.
To quote Cliff Stoll one last time (as he quotes international chess grandmaster Arthur Bisguier) the best way to win at chess is to “always make the best move.” Duh, right, but we think we can help you achieve exactly that, 24×7 and at scale, without you having to work late nights, weekends, and holidays anymore. Let us handle that part.
If you have questions about how Expel can improve your use of log data, drop us a line.
