Dr. Strangelog or: How I learned to stop worrying and love alerts

· 4 MIN READ · DAVE JOHNSON · DEC 15, 2022 · TAGS: Cloud security

Cybersecurity isn’t always rational. It’s certainly not always black and white. Instead, it can be a madcap adventure complete with a colorful cast of characters, each with an agenda (that may or may not be obvious).

Unlike in the movies, we practitioners don’t get retakes until everything looks perfect on screen. Timing, situational awareness, team communication, practice, and knowing which buttons to push often mean the difference between success and failure.

But what if we could improve our odds of success? And what if the tools we already have installed behind the curtain could help?

Scene 1: The camera pans to you, The Security Professional, sitting at your dual-monitor-lit mid-century modern desk. It’s cluttered with vendor swag, paperwork, books you’ve been meaning to read, and a mug of lukewarm coffee loitering at arms reach. Sound familiar?

Now imagine there’s a continuous-feed dot matrix printer to the left of your desk that’s printing every line of every log generated by your environment. Millions, potentially billions of lines per day.

Somewhere in that fast-growing stack is the information you need to protect your environment, assuming you not only know what to look for and where, but also that you’re able to intuit or draw correlations between seemingly innocuous and unrelated items extremely quickly. Is it a benign audit log? Or a malicious indicator of compromise (IOC) designed to fly under the radar?

Now let’s try to solve this problem as-is. You could hire more people to read every single thing on every line (assuming you can find that many people). But that’s difficult (and expensive), and so is finding ways to help them communicate the vast sum of knowledge they’re absorbing.

Fine. We can’t hire enough people. Let’s build a state-of-the-art computerized system that can read and ingest everything printed on the paper and sort it all, like some sort of security information event management (SIEM) system… that’s the ticket!

Now we have a complicated SIEM device, sitting on our desk scanning logs, that also needs to be tuned and maintained. Every time a vendor changes a log format, the system is non-functional until you adjust your ingest template. We also need to teach it what to look for and constantly prune false positives.

This is starting to get out of hand, and we haven’t even started talking about the actual nuts and bolts of doing security itself. We’re just trying to make the system functional.

There’s got to be a better way…

What if—and this may sound insane, but bear with me—the solution is incredibly obvious? What if the data we started with had an innately higher level of fidelity and security context? That’s where we’re trying to get, isn’t it? What if, instead of looking at logs all day, you had a much higher level of refined alerts to dig through? That’s a much better place to start, with potentially a lot less effort and noise. There will still be challenges, but at least we’ll be further down the path. We’ll be shortening our journey to security maturity significantly.

This approach embeds a lot more initial context, stability, and fidelity, assuming the technology is advanced enough to generate that rich security signal (which is commonly the case with on-prem solutions).

While better tactically, this isn’t a universal fix. The cloud, for example, is a bit different. It hasn’t matured to a point where native alert signal alone is solid enough to rely on entirely (at least not yet). So we incorporate alerts, events, and some logs to help fill gaps in monitoring. All this data helps us understand what the cloud thinks is security-relevant and also lets us monitor gaps commonly exploited for compromise. Catching low and slow attackers living off the land is a particularly great use case for this method, and is especially germane these days.

OK, so what if it’s somehow fancier or better? What practical application does simply “doing better” translate to in real life?

In a word, “time.” Time is a non-renewable resource. By starting with a better security signal product to begin with, Expel is able to use time a lot more wisely.

Remember that stack of green bar paper that was filling up your desk? Look again.

What was once a pile has been entirely replaced by a single well-designed form that just says, “Here’s what happened, why it’s important, and how you fix it.” Full stop.

That’s literally itーjust the relevant information you need to take the next steps to ridding your environment of malicious activity. It’s a document you can trust, too, because you can verify and validate every piece of information in it. We call it an incident findings report, but our customers call it “absolute magic” (actual customer quote). Does that sound like something that could improve your current ability to accomplish your daily security tasks and critical business objectives?

That sounds cool. But can it fix the problem and help me prevent it from happening again?

Definitely. Since stopping the spread of malicious activity is paramount in limiting scope of impact and improving mean time to remediation (MTTR), we can programmatically isolate hosts and block bad hashes on your behalf. If it involves compromised user credentials, we can shut that down in its tracks. That way you not only save time by preventing further spread (only a few minutes for us to implement those actions), thus reducing overall impact, but you also reduce subsequent remediation efforts.

We can do all that because we aren’t just log streaming in one direction. We’re directly connected via API and can use your tools’ own native capabilities to remediate automatically.

We know this innovative philosophical approach works, and so do our customers. And don’t worry, your logs will still be extremely useful during enrichment to complete the previously mentioned findings report, specifically when determining root cause. We don’t let good data go to waste. The primary difference is simply in the approach and the significant improvement in the end result. Alerts drive the posture and immediate detection and response, and all the other relevant data, including logs, help us tie it all together for you. That’s a big part of how we do what we do so well.

Scene 2: You’re back at your desk, which is now clean. Incidents have been identified and remediated. The trusty Expel incident report is the only thing you need to handle what once took an entire desktop of security flotsam to tackle.

What are you going to do with all this newly rediscovered desktop real estate???

You open a drawer. Out comes a dusty file folder titled: “Projects I’ve been meaning to work on.”

And now…the fun really starts. But that’s a story for another time…