Security operations · 5 MIN READ · ANTHONY RANDAZZO · OCT 15, 2019 · TAGS: Cloud security / Get technical / How to / Managed security / SOC
Gone are the days when we only had to protect some physical servers and all of the associated networking gear used to route traffic to and from those servers in our data centers.
Fast forward to today — most companies are running at least some of their workloads in the cloud. Today we’ve got virtualized servers, abstracted services and a simplified networking layer all managed via an API. Amazon Web Services (AWS) offers lots of security services to help protect their customers’ data. One of the more well-known services for detection and response is Amazon GuardDuty.
If you’ve heard of Amazon GuardDuty but aren’t exactly sure how to get the most out of it, then this post is for you. I’ll talk about how Amazon GuardDuty works, share the kinds of threats it’s looking for, show you some sample alert investigations and offer a couple tips for how to make more sense of the signals you get from GuardDuty.
What is GuardDuty in AWS?
Amazon GuardDuty is a continuous threat monitoring service available to AWS customers that works by consuming CloudTrail logs (AWS native API logging), Virtual Private Cloud (VPC) flow logs and DNS logs. Fortunately, CloudTrail logging is enabled by default — and you don’t even have to pay for VPC flow logs or Amazon Route 53 (AWS DNS) to benefit from GuardDuty as long as you’re using an AWS DNS resolver (versus using something like Google or OpenDNS). However, having VPC flow logs enabled will provide defenders another tool in their toolbox to use when investigating potential security incidents (more on this later). Now, if you consider what visibility AWS has into its customer’s data and services, then GuardDuty’s use of these three datasets make sense.
Now let’s look at what types of alerts GuardDuty might generate for us by using flow, DNS and API activity logs.
As of today, there are 54 unique GuardDuty findings (more commonly known as rules). These are all based on easy to understand logic or basic anomaly detection. Each finding has the following naming convention:
ThreatPurpose:ResourceTypeAffected/ThreatFamilyName.ThreatFamilyVariant!Artifact
We’ll focus on the ‘ResourceTypeAffected’ portion of that convention because this is the most important section to understand when you’re reviewing GuardDuty alerts. Today, this field will consist of either ‘IAMUser’ or ‘EC2’. All Elastic Cloud Compute (EC2) rules are based on VPC flow logs or DNS logs, while the ‘IAMUser’ rules generate alerts from CloudTrail API logs and possibly in conjunction with flow and DNS logs. We’ve found the ‘IAMUser’ rules to be quite valuable, as they indicate authenticated access into your AWS account.
Because of AWS’ visibility into our data, many of the ‘EC2’ rules are based on an AWS-curated threat lists of atomic IOCs such as domains and IPs. These are then filtered against the flow and DNS logs. But there’s a problem: just like with most security tech, we don’t have any visibility into the threat lists that AWS is using and how they match up (or don’t match up) with the threats we’re concerned with for our own org.
However, there’s a silver lining here: You’re able to provide your own threat lists from third parties and even automate the ingestion of these lists into GuardDuty. This can be IOCs you’ve identified internally to your org or external feeds you subscribe to or consume through something like a Threat Intelligence Platform.
When considering the pyramid of pain — a model focusing on how best to disrupt attackers — many of these GuardDuty alerts correspond to the bottom of the pyramid. When consuming lower fidelity alerts, we recommend enriching those with additional context to provide analysts with more decision support. Here at Expel we take advantage of some third-party enrichment services, such as passive DNS, WHOIS information, OSINT and other data to get a better understanding of the potential threats associated with these IPs and domains identified by GuardDuty.
Investigating GuardDuty alerts
Now that we have an idea of what to expect in a GuardDuty alert, let’s take a look at a couple different example alerts. Expel uses the AWS API to consume our customers’ GuardDuty alerts directly from their AWS Accounts and then we normalize the GuardDuty alert data in Expel Workbench for our analysts.
NOTE: These alerts were generated with GuardDuty’s built-in ‘generate sample findings’ regression test.
Here we get a pretty straightforward explanation in Expel Workbench that our EC2 instance is making connections with a known Tor exit node. Given what we know about these EC2 rules, this alert was simply generated from the VPC flow logs based on an AWS threat list for known Tor exit nodes.
This is where those VPC flow logs would really come in handy. Flow logs contain not only the source and destination IP and ports, but also how much traffic was actually passed. Depending on the function of your EC2 instance, this can paint a pretty telling picture as to whether there might be an instance compromise or not.
If we need more answers, then you’ll need a snapshot of that EC2 for a deeper dive or potentially having some other endpoint software such as EDR running inside of that instance. The latter will depend on your organization’s risk tolerance for security software running on your production (AKA revenue generating) assets.
Next, let’s look at another alert that looks similar at a glance but has drastically different implications.
This alert isn’t quite as straightforward as the previous, but if we revisit what we know about GuardDuty alerts with ‘IAMUser’ as the ResourceTypeAffected, then we know this originated from a CloudTrail log(s). This is where we might sound the alarm. What you’re looking at is someone with API credentials making successful API calls to your AWS account from the Tor anonymizing network. Unless your org has some privacy averse AWS admins or developers, then there’s little reason for you see this particular alert.
Now you need to do some analysis. Determine if an AWS Identity and Access Management (IAM) user or role was compromised as this could help you determine how those credentials may have been compromised. The key difference between these two are that User credentials are permanent whereas role credentials are temporary (lasting between 1 – 12 hours). A user compromise might imply leaked API access keys on GitHub or something similar, while a role compromise will generally implicate some other deeper-rooted issue in your environment such a Server Side Request Forgery (SSRF) vulnerability.
Lastly, let’s look at a real-world GuardDuty alert.
In this alert, Expel analysts identified the anomalous detection of an AWS service user (a Continuous Integration/Continuous Delivery IAM user) making a suspicious API call, ListAccessKeys, that should never be attempted by that user given its purpose. Fortunately, GuardDuty has some insight into what API calls a user or role normally makes. This threat actor was able to initially compromise a less privileged user access key for the AWS account and then the attacker pivoted with a variety of methods to expand access and privileges into other IAM users and roles.
4 things to remember when reviewing Amazon GuardDuty alerts
- GuardDuty alerts are generated based on VPC flow logs, DNS logs, and CloudTrail API logs.
- Currently, there are two primary classes of GuardDuty alerts: alerts based on DNS or VPC flow in and out of your EC2, and alerts that are generated from suspicious IAM (authenticated) API activity.
- Many of the GuardDuty alerts are generated based on threat lists of known malicious domains and IPs. Like most security technology, these threat lists may or may not be what you care about in your org’s threat model — consider enriching these alerts with additional decision support such as passive DNS, WHOIS data, or other IP reputation.
- Keep a close eye out for IAM-related GuardDuty alerts, as this implies there’s an authenticated API session to your AWS account.
