Security operations · 3 MIN READ · AARON WALTON · JUN 13, 2024 · TAGS: MDR
In his Kernelcon 2024 keynote address, John Hultquist, VP of Intelligence Analysis at Mandiant (a Google Cloud company) pointed out that attackers’ use of artificial intelligence (AI) will not always be obvious. In most cases, attackers will primarily use AI to operate more efficiently. Because of this, defenders are less likely to be able to observe exactly where (and how) attackers leverage AI.
When John said this, he likely did not imagine attackers accidentally sending their phishing code scripts to their victims…
Lucky for us, that’s exactly what happened. Expel’s phishing team recently intercepted an email from a (likely inexperienced) attacker that appears to contain the AI-generated code the attacker used to create and send said email.
Let’s break down what the attacker sent and what it can teach us about adversarial use of AI for phishing in the future.
TL;DR
The script provided by the attacker is fairly simple. We believe that the attacker likely received it from someone else who had generated it using ChatGPT or another LLM. It has some clear indicators of being generated: every variable is sensibly and clearly named, and every method contains a comment explaining the functionality of the method.
The Basics
The script is nothing spectacular, but it handles basic features such as dynamically generating the phishing attachments.
The main method of the script allows the attacker to specify their targets using a text file and specify a template for the malicious attachment. In the below code, “list.txt” would contain a list of email addresses.
recipient_file = “list.txt”
html_content = load_html_content(“letter.html”)
The attacker can then specify their “sender_name” and “sender_email” for the phish and pass this to a function called “send_emails” which is designed to handle the rest.
Their “send_emails” function relies on the SendInBlue API (sib_api_v3_sdk) to send emails. (The attacker also included their API key–whoops!). Quite simply, the function sets up the API calls, loads the files, generates the emails and sends them out.
def send_emails(recipient_file, html_content, sender_name, sender_email, use_multi_threading, attachment_file=None):
configuration = sib_api_v3_sdk.Configuration()
configuration.api_key[‘api-key’] = ‘<redacted API key>’api_instance = sib_api_v3_sdk.TransactionalEmailsApi(sib_api_v3_sdk.ApiClient(configuration))recipients = load_recipients(recipient_file)
subject_index = 0
def worker(recipient):
nonlocal subject_index
headers = generate_unique_header()
subject = subjects[subject_index]
send_email(api_instance, recipient, html_content, headers, sender_name, sender_email, attachment_file, subject)
subject_index = (subject_index + 1) % len(subjects)
Specifying the target
The code contains a routine which prepares the attachment to be specific to the target. The script will read the list provided by the attacker and extract the individual targeted usernames and their domains. The domain will be used in multiple parts of the code to personalize the payload for the target, for example, with the subject line.
subjects = [
“[DOMC]: Remittance Note for DOMC -$19,200.08 Batch No.”,
“[DOMC]: ACH Deposit Completed for your Invoice – 29,300USD”,
“[DOMC]: Notification of EFT Remittance”,
“[DOMC]: EFT Payment Advice”,
“[DOMC]: Payment Notification from vendor”,
“[DOMC]: ACH Payment Advice Note 05/16 -“,
“[DOMC]: Remittance Advice attached – ID”
]
The code replaces the placeholder “DOMC” with the user’s domain which is extracted from the target’s email address. Though simple, this allows for some more targeted phishing. The attacker also attempts to randomize the email subjects by appending 5 characters to the end of the email subject lines.
random_string = generate_random_string()
subject_with_random = f”{subject.replace(‘DOMC’, formatted_domain)} {random_string}”
The Future of Phishing
In the last few years, we have seen many threat actors take up phishing. These actors were enabled by enterprising threat actors who manage Phishing-as-a-Service (PhaaS) platforms. These platforms enable threat actors to manage sophisticated campaigns with the latest adversary-in-the-middle techniques. The script we reviewed in this blog represents in a concrete way how AI can further enable cyber crime—namely, providing attackers with the basics for running a targeted campaign. In this example, an actor leveraging this script would only need to supply a target list and an API key, and the script does the rest. For us, the trend is clear: phishing isn’t going away and will only grow as a problem.
Phishing is a major risk to businesses, frequently leading to account takeover, the deployment of malware, and entry points for ransomware gangs. Businesses need to continue to strengthen their security and consider additional technical solutions to the phishing that targets their users and organizations. We recommend building defense-in-depth, ensuring that multiple mechanisms exist to stop and prevent cyber attacks.
Going forward
AI is able to supplement the skills of both defenders and adversaries alike. We expect both inexperienced and seasoned threat actors to enhance their capabilities to conduct cyber attacks. This example shows how one threat actor is using AI to do their dirty work, highlighting that more actors are entering the scene empowered by AI and reminding us to ensure our defenses are ready for whatever the future holds. And at this point, it looks like it holds a lot of phish.
