Our first blog post served as an introductory platform on Continuous Adversary Emulation, a cornerstone of the Continuous Purple Teaming methodology. We delved into how it stands apart from traditional penetration testing and began to explore the offensive spectrum of cyber exercises – simulating known adversaries within our own IT landscapes and aim to transition from theory to practice. This post will guide you through actionable strategies to implement adversary emulation effectively with some concrete examples.

We’ll discuss how to streamline these complex tasks, making them accessible and manageable for your purple and blue teams. Covering ways to simplify these tasks, making them more approachable and manageable for your purple and blue teams. It's important to recognize, as we will cover in this post, that this does not negate the need for dedicated offensive activities that demand specialized expertise. On the contrary, these expert-driven offensive measures are vital for any organization and can be harnessed within our Continuous Purple Teaming methodology to perform certain tasks in an automated manner, enriching the overall security strategy.

Capabilities required for Adversary emulation.

Adversary emulation tools are able to emulate certain tactics, techniques, and procedures (TTPs) of real-world attackers. There aim is to provide a realistic simulation of how an adversary might carry out an attack on an organization's network, which can be useful for identifying weaknesses and improving incident prevention, detection and response capabilities.

There are two primary forms of adversary emulation:

• Manual, Full-Stack Emulation: Unlike automated emulation, manual emulation is a more comprehensive approach where security professionals develop an adversary emulation plan and manually execute the attack paths. This process can be more labor-intensive and requires a deep understanding of the adversary's TTPs. Full-stack emulation considers the entire attack lifecycle, from initial reconnaissance to data exfiltration, and provides a thorough examination of an organization's defensive capabilities.
• Automated/Scripted Emulation: This approach uses automated tools and scripts to replicate specific adversary behaviors linked to the MITRE ATT&CK framework. The MITRE ATT&CK framework is a globally accessible knowledge base of adversary tactics and techniques based on real-world observations. Automated tools can be programmed to execute a sequence of actions that simulate the TTPs of known threat actors. This form of emulation is often used to test specific security controls or to continuously assess the effectiveness of security measures.

Which tool is right for you? There are many aspects to take into account when looking for an adversary emulation tool and a lot of these tools are out there. Before integration it’s important to consider the various capabilities that align with your organization's needs and the specific goals of your purple team exercises. Certain tools will have a better fit with your technology landscape and others will be easier to integrate within your purple team. Below is a list of  capabilities and considerations choosing your adversary emulation tool:

• Technique Coverage: Assess the range and sophistication of the techniques available within the tool, referencing the MITRE ATT&CK framework. High coverage means less need for developing additional techniques. For example, a tool that supports complex multi-stage attack scenarios, like Cobalt Strike, might offer more extensive assessments without the need for custom script development.
• Ease of Use and Accessibility: The tool should have an intuitive interface and comprehensive documentation, which is particularly beneficial for teams new to adversary emulation. A tool like Atomic Red Team, with its straightforward scripts and community-provided guidelines, can be advantageous for quick deployment and learning.
• Integration Capabilities: It's important for the tool to integrate well with your existing security stack and infrastructure automation platforms. For instance, if you use Terraform, a tool that can deploy agents via Terraform scripts would streamline the setup. An example might be SCYTHE, which can integrate with various CI/CD pipelines for automated deployment.
• Automation Features: Determine the automation level, from scripted attacks to intelligent adaptation during exercises. Some tools may offer features to identify misconfigurations or suggest improvements, such as the adaptability of Caldera to change tactics in response to network defenses.
• Scalability: Verify that the tool can scale with your organization. If your operations span multiple environments, including cloud, the tool should support this. For example, a scalable tool would allow the deployment of agents across different VPCs or cloud services without a hitch.
• Reporting and Analytics: Comprehensive reporting is critical for actionable insights post-exercise. For management, tools that can track and present improvement over time or highlight industry-specific threats are invaluable. An example is the detailed reporting and trend analysis provided by commercial solutions like Rapid7's InsightIDR.
• Realism: Choose a tool that offers realistic emulation to mirror actual adversary actions. This might include the ability to chain output from one technique as input to another, providing a more accurate simulation of advanced threats.
• Open-source vs Commercial Tooling: Balance the cost against the potential ROI. Open-source tools, such as Atomic Red team, may be cost-effective and community-driven, while commercial tools such as scythe often come with a additional capabilities and a curated database. This typically includes the latest threat intelligence, attack vectors, new campaigns and overall support.
• Community and Vendor Support: Robust community or vendor support can significantly improve the tool's effectiveness. Community-driven tools like Red Team Automation (RTA) benefit from shared resources and collective expertise, while commercial tools generally offer dedicated support and regular updates.

Adversary emulation Tooling

Various tools are at your disposal to simulate attacks based on real-world adversary behaviors. As highlighted previously, it's important to assess specific capabilities that align with your organization's requirements when choosing these tools. The diagram below categorizes a selection of these tools based on the type of adversary emulation they are best suited for, offering a visual representation to help you navigate the selection process effectively:

https://images.contentstack.io/v3/assets/blt36c2e63521272fdc/blt2d5fa58686687ce7/65b9b0a6c600058f92d5b5b4/ContinuousPurpleTeaming1.png

It's essential to recognize that while some tools may lean towards automated or manual emulation, many offer functionalities that can be adapted to both approaches depending on the level of customization and user input. Selecting the right tool will not only be based on the type of emulation you intend to perform but also on the sophistication of the scenarios you need.  

• CALDERA: Developed by MITRE, CALDERA uses the ATT&CK framework to automate complex attack patterns. It's designed to be adaptable, allowing users to configure and execute a wide range of adversary behaviors without extensive manual oversight. More about CALDERA can be found on its official webpage. - https://github.com/mitre/caldera
• Atomic Red Team: This tool provides a collection of small, modular tests for executing specific ATT&CK techniques. Its design caters to straightforward and targeted testing, making it ideal for quickly validating defenses against known threats. More information is available on the Atomic Red Team GitHub page. - https://github.com/redcanaryco/atomic-red-team
• Red Team Automation (RTA): RTA is focused on the rapid deployment of ATT&CK techniques in an automated manner. It is beneficial for simulating known attack patterns quickly and consistently across various environments. You can learn more about RTA on its GitHub repository. - https://github.com/endgameinc/RTA
• Cobalt Strike: A tool known for its advanced stealth and post-exploitation capabilities, Cobalt Strike is popular among red teams for simulating sophisticated cyber attacks. It provides a comprehensive set of tools for surveillance and maintaining a foothold in a simulated breach scenario. Detailed information is available on the Cobalt Strike website. - https://www.cobaltstrike.com/
• Metasploit Framework: This open-source framework is widely used for developing and executing exploit code against remote target machines. It allows for the custom development of exploits and payloads, facilitating tailored attack simulations. More about Metasploit can be found on its official website. - https://www.metasploit.com/
• SCYTHE: SCYTHE offers a platform for creating and executing customizable and realistic attack campaigns. It allows teams to simulate entire attack lifecycles, from initial entry to actions on objectives. Find more about SCYTHE at their website. - https://www.scythe.io/
• AttackIQ: AttackIQ provides a continuous security validation platform to challenge your security controls against the ever-evolving threat landscape. It aligns with MITRE ATT&CK framework and helps in identifying gaps in defenses and improving the effectiveness of existing security tools. More information is available on the AttackIQ website. - https://attackiq.com/

Cyber Ranges for Adversary Emulation

Another concept is typcally referred to as cyber ranges, can be a SaaS solution or a collection of infrastructure as code, configuration management or user data scripts that deploys a dedicated architecture in cloud environments. In case you want to start with Adversary emulation or having a dedicated environment to test your detection capabilities for specific adversaries. You can simulate these in a cyber range and see how certain systems are detecting the techniques or even validate and review certain configuration changes in a testing environment.

Linked to adversary emulation many of these cyber ranges are out there and very easy to deploy, Jason Ostrom also co-author for SEC598 and SANS Instructor has released some nice projects such as Purple Cloud and AutomatedEmulation.

• Purple Cloud: Developed by Jason Ostrom, this project offers a cloud-based cyber range designed for purple team exercises. It facilitates practical training and experimentation with different attack and defense strategies.
• AutomatedEMulation: Another project by Ostrom, focusing on automating the process of adversary emulation in a controlled environment. It pre-installs tools such as MITRE Caldera, VECTR and many more to simulate your emulation plans on a dedicated environment.

You can explore these projects on Jason Ostrom’s GitHub page for practical insights and tools to implement cyber ranges for effective adversary emulation: https://github.com/iknowjason

Unfortunately, it's not feasible to cover all the cyber ranges available, so please note that this is a selective list, particularly focusing on cloud-based ranges. These ranges, developed by Jason Ostrom, are highlighted for their relevance and detailed coverage in the course SEC598. Our emphasis here is on cloud environments, showcasing how these cyber ranges are particularly adept at providing flexible, scalable, and realistic testing grounds for adversary emulation and cybersecurity training in a cloud context.

Adversary emulation with Caldera

Let's dive into the practical application of adversary emulation. In our SEC598 course, we extensively use CALDERA in a lab scenario to illustrate these concepts. Therefore, we've selected CALDERA as the main tool for the next sections of this blog post, ofc these concepts can be applied to other toolings as well.

This decision is driven by our commitment to providing a hands-on, detailed approach to both understanding and implementing adversary emulation strategies. For those eager to try these techniques themselves, the GitHub project 'Automated Emulation' serves as an excellent resource to begin experimenting with these concepts in a practical setting."

Introduction to Caldara

CALDERA, developed by MITRE, is an advanced adversary emulation framework designed to automate the assessment of security networks using the ATT&CK framework. As of the latest update, CALDERA is at version 4.0.0 Alpha, which introduces significant improvements in user interface, API, and agent communication channels, making it more stable and versatile.

CALDERA operates on an agent/server model. The server component orchestrates the activities of multiple agents deployed across the network. These agents execute tasks and report back to the server, simulating an adversary moving within a network environment. The framework is designed to be flexible and adaptable, allowing for customization of adversary profiles, tactics, and techniques.

For deployment, CALDERA requires a server environment where the central application runs. The agents, which can be deployed on various hosts across the network, communicate with this server. This architecture allows CALDERA to simulate complex attack scenarios in a controlled and safe manner.

https://images.contentstack.io/v3/assets/blt36c2e63521272fdc/blt6c23a785a7bc2abd/65b9b1d1c600055a59d5b5be/ContinuousPurpleTeaming3.png

"Konstantinos Pantazis of NVISO has crafted an excellent blog post detailing the initial steps with CALDERA. This post skillfully explains the essential concepts and configurations necessary for embarking on your first emulation campaign. In our blog post, we're going to delve deeper, building upon these fundamental ideas. We'll provide a practical example to show how you can effectively emulate specific adversaries. For those interested in exploring these basic components of CALDERA in more detail, you can find Pantazis's blog post here: A Beginner's Guide to Adversary Emulation with CALDERA - https://blog.nviso.eu/2023/08/25/a-beginners-guide-to-adversary-emulation-with-caldera/

To set up your initial adversary emulation campaign with CALDERA, we won't cover every detail but will provide a concise overview of the key components that are required in the next sections:

• Agents: In CALDERA, agents act similarly to Remote Access Trojans (RATs). They are programs, written in any language, that execute commands on compromised systems. These agents regularly communicate with the CALDERA server via internet protocols (like HTTP, UDP, or DNS) and check in ('beacon') for new instructions. If an agent misses a check-in, it might indicate detection or compromise.
• Abilities: Abilities in CALDERA are specific sets of instructions executed by an agent. These are typically carried out immediately after the agent's initial beacon to the server.
• Adversaries: In the CALDERA context, adversaries are profiles comprising various abilities. These profiles represent the tactics, techniques, and procedures (TTPs) of known real-world Advanced Persistent Threat (APT) groups. When you run an operation, the chosen adversary profile determines which abilities are executed.
• Operations: Operations are simulated attack scenarios using TTPs from pre-configured adversary profiles. Operations can be automated, running without operator intervention, or manual, where the operator approves each command. Manual operations also allow for the addition of extra TTPs. To initiate an operation, at least one active agent is required.
• Plugins: These are extensions that enhance CALDERA’s functionality. Plugins can provide a range of additional features and capabilities to the core framework.

Step 1: Red Team Insights into Continuous Purple Teaming with MITRE CALDERA

We have multiple strategies at our disposal to utilize an automation framework like CALDERA for continuous assessments of our infrastructure. While researching and identifying techniques used by known adversaries is a common approach, another effective method is to leverage insights from dedicated red or purple team assessments. This involves dissecting the techniques employed during these assessments.

It's crucial to break down the red team scenarios into smaller segments, focusing specifically on the techniques utilized. This granular approach allows for a more precise emulation of real-world attacks and a better understanding of how these techniques impact your infrastructure.

In scenarios involving customized malware or unique code, the feasibility of using these elements in CALDERA for continuous purple teaming will depend on your specific agreements and ethical considerations.

The accompanying diagram illustrates on a high-level how you can convert specific activities from a dedicated red or purple team exercise into a consistent, ongoing purple teaming process. Here's a breakdown of the steps:

1. Execution of Red Team Assessment: Initially, the red team conducts their assessment based on a pre-defined scenario.
2. Post-Assessment Review: At the end of the red team assessment, a debriefing workshop is typically held, or detailed reports are provided. These outline the techniques used and which attack chains were successful.
3. Translating Findings to Continuous Purple Teaming: This information is extremely valuable for transforming a one-time red team assessment into a continuous purple teaming effort. By mapping identified TTPs (Tactics, Techniques, and Procedures) to specific 'abilities' in your adversary emulation tool, you can reconstruct the entire attack chain used by the red team.
4. Regular Simulation Runs: With these details accurately defined in your emulation tool, such as CALDERA, you can schedule regular simulations of these attack scenarios.

https://images.contentstack.io/v3/assets/blt36c2e63521272fdc/blt56ee9a70de1848d7/65b9b1d193cdf14c867cb53c/ContinuousPurpleTeaming4.png

By adhering to this structured approach, you effectively utilize the insights gained from red team assessments in a consistent and actionable manner. This strategy ensures that your security measures are not merely reactive but are proactively evolving. Rather than relying solely on a static report detailing findings and recommendations, this method allows you to continuously validate your organization’s resilience against specific scenarios. By doing so, you move towards a more dynamic and recurrent assessment model, ensuring ongoing vigilance and adaptability in your cybersecurity defenses.

Initiating your continuous purple teaming activities begins with translating the identified attack chains into CALDERA techniques, known as 'abilities'. Let’s proceed to explore how we can define these abilities effectively within CALDERA

Step 2: Analyze & Leverage Known abilities in CALDERA.

Once your CALDERA server is operational, you will have access to a web interface. As detailed in the blog post 'A Beginner's Guide to Adversary Emulation with CALDERA', the interface provides distinct roles: a red user and a blue user. In our context, since we are focusing on configuring offensive activities, we will be operating primarily in the capacity of a red user, which is typically categorized under offensive operations.

When you log into the CALDERA server (typically accessible via HTTP on port 8888), as a red team user, you will be presented with a specific interface.

Before initiating any operation in CALDERA, it's crucial to install at least one agent on your target system. The agent type necessary for your operation depends on the specific abilities you intend to execute. Different abilities often necessitate varied agents, each designed to perform particular tasks or to function effectively on specific target platforms.

For example, the accompanying image illustrates a scenario where the default Sandcat agent is successfully installed on a Windows machine via Powershell, with established communication to the CALDERA server. Typically, the Sandcat agent communicates via HTTP or HTTPS. However, you have the flexibility to customize the agent's configuration to suit the specific needs and security protocols of your environment:

https://images.contentstack.io/v3/assets/blt36c2e63521272fdc/blt0a79331e2243c75f/65b9b1d19274063f95eb73d3/ContinuousPurpleTeaming5.png

Once you have installed at least one agent and the agents is having the status ALOVE, you can begin executing certain abilities. When you navigate to the 'Abilities' section, you will find a comprehensive interface designed for managing and deploying various attack techniques.

Let's consider a sample red team report, which typically includes a section titled 'Attack Scenario Summary' or 'Storyline'. This section summarizes all the executed steps. Within this storyline, there is typically a reference to the techniques used and findings discovered during the assessment.

https://images.contentstack.io/v3/assets/blt36c2e63521272fdc/bltee3697a77f7b9651/65b9b1d1417875380d6760d6/ContinuousPurpleTeaming6.png

Some of these techniques used in red team assessments are easily identifiable and can be adapted accordingly. However, replicating other techniques might be more challenging, depending on the level of customization and specific components used in your assessment. At times, employing a more general technique with the default test can also be a valid approach, especially when the objective is to evaluate the exploitability of your target systems.

Based on the outcomes of our red team assessments, let's delve into the techniques available in Caldera and examine how we can consistently utilize them through our adversary emulation tool. Upon selecting the 'Abilities' page in CALDERA, you will be presented with a general overview of all the abilities installed by default. Additionally, you can create filters based on keywords, tactics, techniques, or even the plugins you have enabled.

https://images.contentstack.io/v3/assets/blt36c2e63521272fdc/blt4ee90193836fd598/65b9b1d1fd23e55e6c7da5ad/ContinuousPurpleTeaming7.png

Let's explore the tactics and techniques in Caldera to better understand and correlate the findings from our red team assessment, aiming to translate these into more automated emulation tasks. When we apply a filter to a specific technique, such as 'T1053.005', we discover that multiple techniques are available:

https://images.contentstack.io/v3/assets/blt36c2e63521272fdc/blt51071f7f43806afd/65b9b1d1a0c8782990592aa7/ContinuousPurpleTeaming8.png

Selecting the ability “PowerShell CMDlet Scheduled Task” has the most similarities which was executed during our red team assessment. Each ability has a detailed overview on the target platform, executor being used, the payloads that are being delivered and most importantly the commands that are being executed on the target:

https://images.contentstack.io/v3/assets/blt36c2e63521272fdc/blt9bd9c3299801a8e0/65b9b1d108561636f35e16cb/ContinuousPurpleTeaming9.png

In the details window of this ability, it is noted that a new scheduled task will be registered to execute at logon and launch the Windows Calculator (calc.exe). While calc.exe is not commonly used in red team assessments and may not add significant value to subsequent stages of your emulation, this action essentially allows you to verify whether your target systems permit the scheduling of registered tasks which can be used for malicious purposes.

Typically, for each technique listed in your report, it's crucial to examine the specific details and the context of their use. Often, you'll find various abilities associated with a particular technique.

It's important to understand that the presence of abilities related to a specific technique in CALDERA doesn't necessarily mean they can be directly applied as-is. These abilities often execute different commands and yield varying outcomes. Therefore, careful consideration is needed to align them with your needs. Ultimately, the goal is to achieve a comprehensive mapping: correlating the techniques used in your red team assessment with those available by default in CALDERA:

https://images.contentstack.io/v3/assets/blt36c2e63521272fdc/blt360afa85ddda0100/65b9b1d141787577086760d2/ContinuousPurpleTeaming10.png

Our mapping of the red team assessment reveals that most techniques can be aligned with a default ability, as illustrated earlier with the 'PowerShell Cmdlet Scheduled Task'. However, for certain techniques, customization or the creation of a new ability is required, which we will demonstrate in the next step.

Step 3: Creating new abilities in CALDERA to cover additional techniques.

Once the abilities have been identified, we can create new abilities or customize the default ones. If you wish to start with an existing ability, it is recommended to duplicate it first and then incorporate your custom code/ specific configuration into the duplicated ability.

In our example, we observed the use of T1078.004, Valid Accounts: Cloud Accounts, for gaining initial access to our environment. During the closing meeting or replay workshop with the red team, it was revealed that valid credentials were discovered in an AWS S3 bucket. These credentials were then utilized to gain local access to one of our systems.

From this explanation, it becomes apparent that this scenario is quite unique to our environment. Consequently, we might want to investigate whether these S3 buckets contain credentials and if they can potentially serve as an input source for other techniques.

Upon examining the details of our red team assessments, we identified the relevant commands, resources, and files. This information should be sufficient to easily create our own ability, one that produces a similar output and can be effectively incorporated into our adversary emulation plan.

https://images.contentstack.io/v3/assets/blt36c2e63521272fdc/blt1c899c5c1c9dbb36/65b9b1d108561675b75e16c7/ContinuousPurpleTeaming11.png

We are using the same technique IDs that were listed in the report, for clarification purposes this ability can also be linked to the technique T1552.002 – Credentials in files, but from our storyline the account was used and allocated to the initial foothold stage.

The red team report reveals that PowerShell was used to download and parse the sensitive file and retrieve usernames listed within that file:

$url="http://users.targetorganization.s3.amazonaws.com/-account-backup.txt";

$wc=New-Object System.Net.WebClient;$output="C:\Users\Public\account-backup.txt";

$wc.DownloadFile($url,$output); $file_content = cat C:\Users\Public\\account-backup.txt | sls ":";

$user = $file_content.Line.Split(":")[0];

Echo $user;

The output for our ability is stored in a variable called $user, this output can be transformed as a fact within Caldera so other techniques can also access this dynamic value, which will be configured in the parsers section of our ability.

As an example, you can create a new ability that validates if the user that was found is present on the local system, this can be done through PowerShell: Get-LocalUser -Name {host.user.name}

To illustrate the concept your ability has the following command:

https://images.contentstack.io/v3/assets/blt36c2e63521272fdc/blt975af5dc1ae1b9a5/65b9b1d1a0c878c5d2592aab/ContinuousPurpleTeaming12.png

The basics of our ability our created, ideally, we already want to run this ability on one of our agents to validate the working. You can easily to that by saving your ability and go to the access plugin to run a specific ability on a specific agent:

https://images.contentstack.io/v3/assets/blt36c2e63521272fdc/blt8bab9059dd76f7cf/65b9b1d155a88a448fda75fd/ContinuousPurpleTeaming13.png

Let’s validate our ability by running this on our agent, you can select any ability based on name, tactics, and technique:

https://images.contentstack.io/v3/assets/blt36c2e63521272fdc/blt71b5cef778750576/65b9b1d1461c1323f1b8f74d/ContinuousPurpleTeaming14.png

Once the ability is being executed you will see an additional entry on your access page that shows the progress of your ability, and you can also validate the output for that specific ability by clicking on the output button:

https://images.contentstack.io/v3/assets/blt36c2e63521272fdc/blt4e32641fc948f040/65b9b1d1292a0ec8ab87dd53/ContinuousPurpleTeaming15.png

In our example the username was successfully parsed, and you can see this as the following standard output where the username “Kelly.w” is being found in that specific file:

https://images.contentstack.io/v3/assets/blt36c2e63521272fdc/bltcbaba207d26865fb/65b9b1d155a88a3f76da75f5/ContinuousPurpleTeaming16.png

Now that we have confirmed our newly created ability works on our agents, let's explore how we can capture the standard output as a 'fact' within Caldera.

A "fact" is a piece of information that an agent knows or discovers during an operation. Facts are critical elements within Caldera's framework, they typically include details like usernames, passwords, file paths, system configurations, or any other relevant pieces of information. That information is used to drive decision-making processes, and to share information between different actions and abilities during your simulations.

Let’s edit our ability and add a parser to it so we can save the output as a fact within CALDERA:

https://images.contentstack.io/v3/assets/blt36c2e63521272fdc/blt43791081f12007a7/65b9b1d1461c134c5bb8f751/ContinuousPurpleTeaming17.png

In this scenario, we utilize the basic parser from the Stockpile plugin, which enables us to assign our output to a fact named 'host.user.name'. This approach simplifies the process of using the output as a fact, allowing us to easily reference 'host.user.name' in subsequent abilities.

Once your parser is configured and you rerun the ability through the access plugin, you should observe a more structured output as which displays our fact:

https://images.contentstack.io/v3/assets/blt36c2e63521272fdc/bltf48c3a7db38758bb/65b9b1d1461c1314efb8f749/ContinuousPurpleTeaming18.png

This fact can be utilized in other abilities. Within the command value, you reference a fact using the syntax #{}, so in our case, it would be #{host.user.name}. For instance, an ability can check locally if that user account exists on our target system:

As we have now learned to work with Caldera's default abilities, create our own customized abilities, and set facts for sharing across different abilities, let's proceed to chain the abilities mentioned above together. This will allow us to create our first basic emulation plan. Our next blog post, where we will cover a full emulation plan in detail, build upon this foundational understanding and how to start with adversary emulation.

Step 4: Create an adversary profile and execute these abilities through an operation in CALDERA

An adversary profile in Caldera represents a conceptual model of a threat actor, a specific type of attack, or the storyline derived from our Red Team assessments. Essentially, it is a collection of abilities that emulate the behaviors and tactics you have defined within the adversary profile.

The abilities that were covered in step 1 and step 2 can now be chained together by creating a new profile. Within the main menu you can find the adversaries section and click on the New profile button to create your own customized campaign. In our example we are going to use a clear naming that is linked to our red team assessment:

https://images.contentstack.io/v3/assets/blt36c2e63521272fdc/blt5a02eeb4f65ac2b3/65b9b1d1292a0e7e3b87dd4f/ContinuousPurpleTeaming19.png

Begin by selecting the abilities you wish to include and add them to the adversary profile. In certain cases, it's possible to add an entire adversary profile, complete with all its associated abilities. Simply identify the abilities you need and arrange them in chronological order within your adversary profile. The process of adding abilities can be intricate, as it requires knowledge of which tactics and techniques each ability corresponds to. Many relevant techniques are categorized under the tactic 'multiple' because they fit into various tactics within an emulation plan.

https://images.contentstack.io/v3/assets/blt36c2e63521272fdc/blta8550401d0ed9b61/65b9b1d168334a8591c61fba/ContinuousPurpleTeaming20.png

Let's proceed by integrating our custom ability that extracts usernames from an S3 bucket and checks if these users exist locally on the system. While this is an additional ability, it effectively illustrates how facts can be applied within an adversary profile. For this blogpost let’s limit the number of abilities to the ones that were discussed before.

https://images.contentstack.io/v3/assets/blt36c2e63521272fdc/blt88d9c9bac93c25bf/65b9b1d11f10e83c1b4ba404/ContinuousPurpleTeaming21.png

Now that you have created an adversary profile you can start by configuring an operation. There are many settings that can be configured within an operation such as:

• Target Groups in Caldera refer to the specific sets of machines or network entities that are the focus of an operation. You can define a group based on certain criteria like network segment, operating system, or any other distinguishing factor. When you launch an operation, it will be directed at the machines in this group.
• Planners are responsible for deciding the sequence of actions (or abilities) during an operation. They determine the strategy of an operation, including which abilities to execute and in what order. Different planners can be used for various operational goals, such as stealth, speed, or thoroughness.
• Obfuscators in Caldera are used to modify how commands are executed to evade detection. They can change the syntax or method of command execution without altering its functionality. This is crucial for simulating advanced adversaries who use obfuscation techniques to bypass security defenses.
• Parsers are used to process the output of abilities. When an ability is executed by an agent, the parser interprets the results and extracts useful information (facts) from them. These facts are then used to inform subsequent steps in the operation, allowing for dynamic and responsive attack simulations.

The emulation of our three abilities requires no customized configuration at this moment, as we plan to execute them immediately. When creating a new operation, it's important to give your operation a meaningful name and select the appropriate adversary profile.

https://images.contentstack.io/v3/assets/blt36c2e63521272fdc/blt897893b5ad90890e/65b9b1d155a88ace70da75f9/ContinuousPurpleTeaming22.png

Once the operation is started you can monitor the progress of your operation run, abilities being successful or failed, outputs linked to these abilities:

https://images.contentstack.io/v3/assets/blt36c2e63521272fdc/bltb69316a5e7f80337/65b9b1d160a2757bb67fd1ae/ContinuousPurpleTeaming23.png

You can also see that our facts are being parsed and used in our next ability by viewing the output of our first ability and validating the command of the next one:

https://images.contentstack.io/v3/assets/blt36c2e63521272fdc/blt8bc44c20c3fdbf61/65b9b1d1450fa4a4f4017360/ContinuousPurpleTeaming24.png

In our next blog post, we will delve into a comprehensive emulation plan, encompassing not only the execution of a full-scale operation but also the integration of customized payloads tailored for specific targets. This deep dive will equip you with advanced insights and practical know-how, further supporting your ongoing purple teaming efforts. By exploring these complex aspects of Caldera, we aim to enhance your ability to conduct thorough and effective cybersecurity simulations.

Conclusions:

Our journey through the realm of Continuous Purple Teaming has equipped us with insights and strategies for implementing adversary emulation. We've highlighted the distinction from traditional penetration testing and provided a roadmap for simulating real-world adversaries, or how certain complex assessments are translated to a more continuous testing approach.

Key takeaways include:

• As purple teaming matures as a security activity, it should integrate in Security Operations and become a continuous, highly automated, activity.
• There will always be a place for high-value, periodic, red team assessments to assess realistic defense level against threats.
• The importance of leveraging both manual and automated adversary emulation techniques.
• When properly implemented, purple teaming is a strong enabler for continuous improvement in TTP prevention and detection.

We've seen how CALDERA, with its compatibility with the MITRE ATT&CK framework, offers an adaptable platform for translating attack patterns into common adversary emulation plans. Its agent/server model, along with the capability to create and execute customizable adversary profiles, makes it a good tool for both learning and applying adversary emulation techniques.

In our upcoming blog post, we will present a comprehensive emulation plan using CALDERA. This will include the execution of full-scale operations and the integration of customized payloads, providing deeper insights into effective cybersecurity simulations. Stay tuned as we continue to navigate the sophisticated landscape of Continuous Purple Teaming focused with CALDERA.

Related course - SEC598: Security Automation for Offense, Defense, and Cloud