Introducing Endgame Red Team Automation

Organizations and practitioners often struggle with a practical way to validate the effectiveness of prevention and detection products, services, or homegrown capabilities. Fortunately, MITRE's ATT&CK™ matrix partially addresses this challenge by identifying almost 200 different kinds of adversarial behavior.  As we collectively recognize the need to go beyond malware-only testing, ATT&CK™ is increasingly the standard by which adversarial behavior is assessed. However, it is essential that organizations realize that their coverage of techniques in ATT&CK™ is a process and not a destination given that new tactics are regularly added. For vendors using ATT&CK to express coverage, detections must be robust enough that an adversary can’t simply rename their binary to evade discovery - ATT&CK™ is the counterpoint to those brittle indicators.

As useful as the ATT&CK™ framework is conceptually, it only solves a portion of the problem: organizations and practitioners still need to test their defenses against each of the behaviors detailed in the ATT&CK™ matrix. ATT&CK™ can be overwhelming because of the wide array of techniques and, within each technique, sometimes many ways an adversary can implement the technique. This complexity has prompted MITRE to add sub-techniques later this year.  Beyond a basic lack of time to perform assessments, the absence of a turnkey validation toolkit with sufficient detail to account for the range of adversary behavior further limits an organization’s ability to carry out comprehensive assessments.  

Endgame’s Red Team Automation (RTA) begins to fill this gap, joining a small number of similarly useful tools like Red Canary’s Atomic Red, Uber’s Metta project, and MITRE’s own Caldera. Endgame Research created the RTA framework for internal experimentation and automated testing of some of the preventions and detections we deliver to customers in the Endgame endpoint protection platform.  We are sharing the RTA framework publicly to help organizations accelerate and enable the assessment process and highlight detection coverage and gaps. Organizations can then focus more confidently on monitoring high real-time detections in their enterprise and prioritize filling critical gaps in coverage.

RTA Overview

RTA is a set of 38 scripts and supporting executables that generate reliable artifacts which correspond to techniques in the ATT&CK™ framework.  Initially, RTA provides coverage of 49 ATT&CK™ techniques and will expand over time. To help get started with RTA, I’ll provide an overview of the technology in RTA, followed by few examples to demonstrate how to start using RTA today. Importantly, we’ve structured RTA to help expand defensive capabilities and support the broader open source community. To that end, I’ll also address how you can contribute techniques to RTA through the GitHub repository.   

RTA differs from other adversary simulation capabilities in some significant ways.  When we started, none of the other frameworks existed publicly. We continue to develop RTA instead of adopting another framework because we’ve found Endgame RTA to have low overhead and is simpler to use and extend.  We expect some practitioners will find RTA useful for the same reasons. This blogpost won’t focus on comparing these tools, but will point out some substantive differences so readers can assess which adversary data-generation toolkit may be best for them.  

To begin using RTA, it is important to understand some of its key features. Some scripts work with a compiled executable, “myapp.exe”, which is capable of creating a local web server for hosting malicious downloads, manipulating timestamps of a file, or generating a user-defined beacon. The remainder of the repo includes dozens of Python-based scripts with the capabilities to:

• Use native tools to download and execute remote files
• Perform anti-forensics operations such as deleting volume journals
• Perform lateral movement to a target system and take actions
• Setup both common and uncommon persistence mechanisms
• Perform one of several UAC bypass techniques
• And many more...

One way that RTA differs from solutions like Uber’s Metta is that RTA doesn’t directly include the ability to configure a playbook of several sequential adversary behaviors. Enabling this natively is an option for future development, but can now be accomplished with some light scripting. Presently, we prioritize covering the depth and breadth of ATT&CK™ over sequencing or APT simulation features.

Figure 1 contains an excerpt of one of these RTA scripts. It is a Python script which wipes the Windows Security event log, corresponding to File Deletion in ATT&CK™. While we chose the Python language for convenience, there’s no technical barrier to changing that language to something else in the future.

Figure 1: Contents of Script to Clear Security Event Log in Windows

The Endgame RTA repository currently has 39 scripts in addition to the “myapp.exe” executable. Over the coming months, we’ll share additional scripts to expand this coverage across even more of the ATT&CK™ matrix. We also actively encourage community engagement to contribute to and expedite the creation of a complete collection of ATT&CK™, and are currently accepting pull requests. RTA is intended to be a tool for the community and by the community.

Figure 2: Current ATT&CK™ coverage using Endgame’s RTA

The RTA repo also includes a file containing global configuration settings and utility functions, “common.py”. This can be imported into a new script to take advantage of variables or functions as well as modified to create new variables or capabilities. By updating the LOCAL_IP variable, which is set to localhost by default, RTA scripts can be pointed to hosted payloads on a remote system. By updating the CALLBACK_REGEX variable, the URL associated with some types of beaconing simulation can be changed. This Python file also contains globally-available functions like logging messages during execution which can be presented to the user, evaluating whether a system is 32 or 64 bit; or identifying writable directories.

At this time, RTA has very few dependencies aside from Python 2.7. To get the most out of RTA, which demonstrates several living-off-the-land techniques, we recommend that users download and extract to Sysinternals suite to the “bin” subdirectory along with Microsoft’s commandline transform utility, “msxsl.exe”. The RTA README contains the relevant links and instructions.

Putting RTA into action

To demonstrate how RTA can support coverage assessments against specific behaviors in ATT&CK™, I’ll walk through a brief scenario. During initial compromise, attackers often incorporate a range of tactics such as combining phishing with a macro-less Word document attack. For example, a recent sample of malware leverages multiple techniques and features to eventually deploy credential-stealing malware. How exactly does this work?

First, a phishing lure is sent that contains a Word document and, instead of implementing macros (consider blocking macros) which are embedded entirely within the document, it contains a link to a remotely-hosted file. This is a different approach but not exactly a new one. It works because the DOCX file fetches external resources once opened, grabbing an RTF file from the Internet and opening it locally.

The RTF file is itself weaponized to execute the built-in “mshta.exe” (MSHTA) application and download an HTA file from the Internet, a generic behavior we should universally be on the lookout for. This HTA file contains an obfuscated VBscript with an embedded PowerShell script. You might call it “script-ception” given the number of layers. PowerShell is then invoked to run the script and download a binary which is classified as a “credential stealer”. Figure 3 contains a high-level representation of this activity.

Figure 3: Visual representation of malware execution process

Let’s quickly map these generic tactics to the ATT&CK™ matrix:

1. Phishing (Pre-T1144) messages with malicious attachments to deliver a Word DOCX file through your email gateway
2. Exploitation of a vulnerability (T1068) within the RTF file downloaded as an external resource of the Word DOCX file
3. Use of the native MSHTA utility (T1170) to pull down an HTA file containing obfuscated (T1027) VBScript (T1064)
4. The VBScript invokes PowerShell (T1086) which downloads and executes a malicious compiled executable
   1. The request occurs over HTTP, so commonly used ports (T1043) applies
   2. The download is from an external network location, so remote file copy (T1105) is relevant

That’s not a small number of techniques but I think we can handle this. Let’s start with the first behavior we see at the endpoint. A user receives a DOCX attachment. Upon viewing the attachment, Microsoft Word then grabs an externally-hosted RTF. Unfortunately, few organizations have a firm understanding of how normal this occurrence is. Because it utilizes native functionality of the Word application to fetch external references, there’s a chance that this is going to be too noisy to generically prompt an alert. Fortunately, there are RTA scripts for simulating unusual network activity associated with Microsoft Office and several otherwise trusted applications.

Microsoft Word is also the parent of MSHTA. This native Windows utility is used for downloading and executing HTA files, which is a suspicious generic methodology. In most cases, this application should not be a child of Microsoft Office. This RTA creates a local webserver long enough to download and execute a benign HTA file - a harmless way to prove that you can detect this widely-used generic technique, no matter which payload is ultimately dropped and executed. If your detections only fire if a malware-detection engine properly classifies the sample, reconsider how much that monitoring capability is trusted.

In this example, the attacker used the HTA file to download and execute a PowerShell script - another area RTA covers. There are a dozen or more ways that PowerShell can be abused during post-exploitation, including but not limited to input redirection, obfuscation of script contents, use of script-block encoding and some command line arguments. Simple things, like payload encoding, are very easy to express harmlessly (the example performs a ping command) in an RTA as seen in Figure 4.

Figure 4: Endgame’s “powershell_args.py” RTA command output

Although this attack example demonstrates how RTA can help evaluate defenses against a range of tactics, it is simply one use case. Endgame RTA covers over a quarter of the ATT&CK™ matrix currently, including the creation and removal of common and uncommon persistence mechanisms, a wide variety of commands that perform lateral movement, abusing native tools for encoding or obfuscation and many more!

Bringing RTA to the Community

Endgame’s RTA is an internal tool we created for use within Endgame Research and are now releasing as an open source technology that organizations can use today to help assess their ability to detect, triage, and respond to threats. This assessment process has become increasingly important to enterprises as new tactics are discovered and new breaches come to light.  

We plan to expand the RTA project in two ways.  First, we will continue to expand coverage of ATT&CK™, adding scripts for cells not yet covered and deepening coverage in cells we already include. We also plan to enhance the usability of the RTA framework to make it easier to run, providing the raw material for self-assessment.

In addition, Roberto Rodriguez of SpecterOps and I will be presenting a complete method for quantifying the assessment process - regardless of the technology you use for detection - at BSidesCharm in April. We’ll combine RTA with the exceptional work Roberto started in the blogposts How hot is your hunt team and Ready to hunt? First show me your data to offer more guidance on using ATT&CK™ and RTA to measure your detection coverage.

While Roberto’s examples may focus on hunting, as the example I provided in this post demonstrates, RTA has much broader applicability for a range of use cases. Endgame looks forward to working with the community to build out this capability, demonstrate a useful application of the ATT&CK™ matrix and, most importantly, help organizations and practitioners apply concrete methods and metrics for assessing their defensive coverage.