The Rust language ecosystem is growing each day, its popularity increasing, and with good reason. It’s the only mainstream language that provides memory and concurrency safety at compile time, with a powerful and rich build system (cargo), and a growing number of packages (crates).

My daily driver is still C++, as most of my work is about low-level system and kernel programming, where the Windows C and COM APIs are easy to consume. Rust is a system programming language, however, which means it plays, or at least can play, in the same playground as C/C++. The main snag is the verbosity required when converting C types to Rust. This “verbosity” can be alleviated with appropriate wrappers and macros. I decided to try writing a simple WDM driver that is not useless – it’s a Rust version of the “Booster” driver I demonstrate in my book (Windows Kernel Programming), that allows changing the priority of any thread to any value.

Getting Started

To prepare for building drivers, consult Windows Drivers-rs, but basically you should have a WDK installation (either normal or the EWDK). Also, the docs require installing LLVM, to gain access to the Clang compiler. I am going to assume you have these installed if you’d like to try the following yourself.

We can start by creating a new Rust library project (as a driver is a technically a DLL loaded into kernel space):

cargo new --lib booster

We can open the booster folder in VS Code, and begin are coding. First, there are some preparations to do in order for actual code to compile and link successfully. We need a build.rs file to tell cargo to link statically to the CRT. Add a build.rs file to the root booster folder, with the following code:

fn main() -> Result<(), wdk_build::ConfigError> {
    std::env::set_var("CARGO_CFG_TARGET_FEATURE", "crt-static");
    wdk_build::configure_wdk_binary_build()
}

(Syntax highlighting is imperfect because the WordPress editor I use does not support syntax highlighting for Rust)

Next, we need to edit cargo.toml and add all kinds of dependencies. The following is the minimum I could get away with:

[package]
name = "booster"
version = "0.1.0"
edition = "2021"

[package.metadata.wdk.driver-model]
driver-type = "WDM"

[lib]
crate-type = ["cdylib"]
test = false

[build-dependencies]
wdk-build = "0.3.0"

[dependencies]
wdk = "0.3.0"       
wdk-macros = "0.3.0"
wdk-alloc = "0.3.0" 
wdk-panic = "0.3.0" 
wdk-sys = "0.3.0"   

[features]
default = []
nightly = ["wdk/nightly", "wdk-sys/nightly"]

[profile.dev]
panic = "abort"
lto = true

[profile.release]
panic = "abort"
lto = true

The important parts are the WDK crates dependencies. It’s time to get to the actual code in lib.rs.

The Code

We start by removing the standard library, as it does not exist in the kernel:

#![no_std]

Next, we’ll add a few use statements to make the code less verbose:

use core::ffi::c_void;
use core::ptr::null_mut;
use alloc::vec::Vec;
use alloc::{slice, string::String};
use wdk::*;
use wdk_alloc::WdkAllocator;
use wdk_sys::ntddk::*;
use wdk_sys::*;

The wdk_sys crate provides the low level interop kernel functions. the wdk crate provides higher-level wrappers. alloc::vec::Vec is an interesting one. Since we can’t use the standard library, you would think the types like std::vec::Vec<> are not available, and technically that’s correct. However, Vec is actually defined in a lower level module named alloc::vec, that can be used outside the standard library. This works because the only requirement for Vec is to have a way to allocate and deallocate memory. Rust exposes this aspect through a global allocator object, that anyone can provide. Since we have no standard library, there is no global allocator, so one must be provided. Then, Vec (and String) can work normally:

#[global_allocator]
static GLOBAL_ALLOCATOR: WdkAllocator = WdkAllocator;

This is the global allocator provided by the WDK crates, that use ExAllocatePool2 and ExFreePool to manage allocations, just like would do manually.

Next, we add two extern crates to get the support for the allocator and a panic handler – another thing that must be provided since the standard library is not included. Cargo.toml has a setting to abort the driver (crash the system) if any code panics:

extern crate wdk_panic;
extern crate alloc;

Now it’s time to write the actual code. We start with DriverEntry, the entry point to any Windows kernel driver:

#[export_name = "DriverEntry"]
pub unsafe extern "system" fn driver_entry(
    driver: &mut DRIVER_OBJECT,
    registry_path: PUNICODE_STRING,
) -> NTSTATUS {

Those familiar with kernel drivers will recognize the function signature (kind of). The function name is driver_entry to conform to the snake_case Rust naming convention for functions, but since the linker looks for DriverEntry, we decorate the function with the export_name attribute. You could use DriverEntry and just ignore or disable the compiler’s warning, if you prefer.

We can use the familiar println! macro, that was reimplemented by calling DbgPrint, as you would if you were using C/C++. You can still call DbgPrint, mind you, but println! is just easier:

println!("DriverEntry from Rust! {:p}", &driver);
let registry_path = unicode_to_string(registry_path);
println!("Registry Path: {}", registry_path);

Unfortunately, it seems println! does not yet support a UNICODE_STRING, so we can write a function named unicode_to_string to convert a UNICODE_STRING to a normal Rust string:

fn unicode_to_string(str: PCUNICODE_STRING) -> String {
    String::from_utf16_lossy(unsafe {
        slice::from_raw_parts((*str).Buffer, (*str).Length as usize / 2)
    })
}

Back in DriverEntry, our next order of business is to create a device object with the name “\Device\Booster”:

let mut dev = null_mut();
let mut dev_name = UNICODE_STRING::default();
string_to_ustring("\\Device\\Booster", &mut dev_name);

let status = IoCreateDevice(
    driver,
    0,
    &mut dev_name,
    FILE_DEVICE_UNKNOWN,
    0,
    0u8,
    &mut dev,
);

The string_to_ustring function converts a Rust string to a UNICODE_STRING:

fn string_to_ustring(s: &str, uc: &mut UNICODE_STRING) -> Vec<u16> {
    let mut wstring: Vec<_> = s.encode_utf16().collect();
    uc.Length = wstring.len() as u16 * 2;
    uc.MaximumLength = wstring.len() as u16 * 2;
    uc.Buffer = wstring.as_mut_ptr();
    wstring
}

This may look more complex than we would like, but think of this as a function that is written once, and then just used all over the place. In fact, maybe there is such a function already, and just didn’t look hard enough. But it will do for this driver.

If device creation fails, we return a failure status:

if !nt_success(status) {
    println!("Error creating device 0x{:X}", status);
    return status;
}

nt_success is similar to the NT_SUCCESS macro provided by the WDK headers.

Next, we’ll create a symbolic link so that a standard CreateFile call could open a handle to our device:

let mut sym_name = UNICODE_STRING::default();
let _ = string_to_ustring("\\??\\Booster", &mut sym_name);
let status = IoCreateSymbolicLink(&mut sym_name, &mut dev_name);
if !nt_success(status) {
    println!("Error creating symbolic link 0x{:X}", status);
    IoDeleteDevice(dev);
    return status;
}

All that’s left to do is initialize the device object with support for Buffered I/O (we’ll use IRP_MJ_WRITE for simplicity), set the driver unload routine, and the major functions we intend to support:

    (*dev).Flags |= DO_BUFFERED_IO;

    driver.DriverUnload = Some(boost_unload);
    driver.MajorFunction[IRP_MJ_CREATE as usize] = Some(boost_create_close);
    driver.MajorFunction[IRP_MJ_CLOSE as usize] = Some(boost_create_close);
    driver.MajorFunction[IRP_MJ_WRITE as usize] = Some(boost_write);

    STATUS_SUCCESS
}

Note the use of the Rust Option<> type to indicate the presence of a callback.

The unload routine looks like this:

unsafe extern "C" fn boost_unload(driver: *mut DRIVER_OBJECT) {
    let mut sym_name = UNICODE_STRING::default();
    string_to_ustring("\\??\\Booster", &mut sym_name);
    let _ = IoDeleteSymbolicLink(&mut sym_name);
    IoDeleteDevice((*driver).DeviceObject);
}

We just call IoDeleteSymbolicLink and IoDeleteDevice, just like a normal kernel driver would.

Handling Requests

We have three request types to handle – IRP_MJ_CREATE, IRP_MJ_CLOSE, and IRP_MJ_WRITE. Create and close are trivial – just complete the IRP successfully:

unsafe extern "C" fn boost_create_close(_device: *mut DEVICE_OBJECT, irp: *mut IRP) -> NTSTATUS {
    (*irp).IoStatus.__bindgen_anon_1.Status = STATUS_SUCCESS;
    (*irp).IoStatus.Information = 0;
    IofCompleteRequest(irp, 0);
    STATUS_SUCCESS
}

The IoStatus is an IO_STATUS_BLOCK but it’s defined with a union containing Status and Pointer. This seems to be incorrect, as Information should be in a union with Pointer (not Status). Anyway, the code accesses the Status member through the “auto generated” union, and it looks ugly. Definitely something to look into further. But it works.

The real interesting function is the IRP_MJ_WRITE handler, that does the actual thread priority change. First, we’ll declare a structure to represent the request to the driver:

#[repr(C)]
struct ThreadData {
    pub thread_id: u32,
    pub priority: i32,
}

The use of repr(C) is important, to make sure the fields are laid out in memory just as they would with C/C++. This allows non-Rust clients to talk to the driver. In fact, I’ll test the driver with a C++ client I have that used the C++ version of the driver. The driver accepts the thread ID to change and the priority to use. Now we can start with boost_write:

unsafe extern "C" fn boost_write(_device: *mut DEVICE_OBJECT, irp: *mut IRP) -> NTSTATUS {
    let data = (*irp).AssociatedIrp.SystemBuffer as *const ThreadData;

First, we grab the data pointer from the SystemBuffer in the IRP, as we asked for Buffered I/O support. This is a kernel copy of the client’s buffer. Next, we’ll do some checks for errors:

let status;
loop {
    if data == null_mut() {
        status = STATUS_INVALID_PARAMETER;
        break;
    }
    if (*data).priority < 1 || (*data).priority > 31 {
        status = STATUS_INVALID_PARAMETER;
        break;
    }

The loop statement creates an infinite block that can be exited with a break. Once we verified the priority is in range, it’s time to locate the thread object:

let mut thread = null_mut();
status = PsLookupThreadByThreadId(((*data).thread_id) as *mut c_void, &mut thread);
if !nt_success(status) {
    break;
}

PsLookupThreadByThreadId is the one to use. If it fails, it means the thread ID probably does not exist, and we break. All that’s left to do is set the priority and complete the request with whatever status we have:

        KeSetPriorityThread(thread, (*data).priority);
        ObfDereferenceObject(thread as *mut c_void);
        break;
    }
    (*irp).IoStatus.__bindgen_anon_1.Status = status;
    (*irp).IoStatus.Information = 0;
    IofCompleteRequest(irp, 0);
    status
}

That’s it!

The only remaining thing is to sign the driver. It seems that the crates support signing the driver if an INF or INX files are present, but this driver is not using an INF. So we need to sign it manually before deployment. The following can be used from the root folder of the project:

signtool sign /n wdk /fd sha256 target\debug\booster.dll

The /n wdk uses a WDK test certificate typically created automatically by Visual Studio when building drivers. I just grab the first one in the store that starts with “wdk” and use it.

The silly part is the file extension – it’s a DLL and there currently is no way to change it automatically as part of cargo build. If using an INF/INX, the file extension does change to SYS. In any case, file extensions don’t really mean that much – we can rename it manually, or just leave it as DLL.

Installing the Driver

The resulting file can be installed in the “normal” way for a software driver, such as using the sc.exe tool (from an elevated command window), on a machine with test signing on. Then sc start can be used to load the driver into the system:

sc.exe sc create booster type= kernel binPath= c:\path_to_driver_file
sc.exe start booster

Testing the Driver

I used an existing C++ application that talks to the driver and expects to pass the correct structure. It looks like this:

#include <Windows.h>
#include <stdio.h>

struct ThreadData {
	int ThreadId;
	int Priority;
};

int main(int argc, const char* argv[]) {
	if (argc < 3) {
		printf("Usage: boost <tid> <priority>\n");
		return 0;
	}

	int tid = atoi(argv[1]);
	int priority = atoi(argv[2]);

	HANDLE hDevice = CreateFile(L"\\\\.\\Booster",
		GENERIC_WRITE, 0, nullptr, OPEN_EXISTING, 0,
		nullptr);

	if (hDevice == INVALID_HANDLE_VALUE) {
		printf("Failed in CreateFile: %u\n", GetLastError());
		return 1;
	}

	ThreadData data;
	data.ThreadId = tid;
	data.Priority = priority;
	DWORD ret;
	if (WriteFile(hDevice, &data, sizeof(data),
		&ret, nullptr))
		printf("Success!!\n");
	else
		printf("Error (%u)\n", GetLastError());

	CloseHandle(hDevice);

	return 0;
}

Here is the result when changing a thread’s priority to 26 (ID 9408):

Conclusion

Writing kernel drivers in Rust is possible, and I’m sure the support for this will improve quickly. The WDK crates are at version 0.3, which means there is still a way to go. To get the most out of Rust in this space, safe wrappers should be created so that the code is less verbose, does not have unsafe blocks, and enjoys the benefits Rust can provide. Note, that I may have missed some wrappers in this simple implementation.

You can find a couple of more samples for KMDF Rust drivers here.

The code for this post can be found at https://github.com/zodiacon/Booster.

Structured Storage is a Windows technology that abstracts the notions of files and directories behind COM interfaces – mainly IStorage and IStream. Its primary intent is to provide a way to have a file system hierarchy within a single physical file.

Structured Storage has been around for many years, where its most famous usage was in Microsoft Office files (*.doc, *.ppt, *.xls, etc.) – before Office moved to the extended file formats (*.docx, *.pptx, etc.). Of course, the old formats are still very much supported.

The Structured Storage interfaces (IStorage representing a directory, and IStream representing a file) are just that – interfaces. To actually use them, some implementation must be available. Windows provided an implementation of Structured Storage called Compound Files. These terms are sometime used interchangeably, but the distinction is important: Compound Files is just one implementation of Structured Storage – there could be others. Compound Files does not implement everything that could be implemented based on the defined Structured Storage interfaces, but it implements a lot, definitely enough to make it useful.

You can download an old tool (but still works well) called SSView, which can be used to graphically view the contents of physical files that were created by using the Compound File implementation. Here is a screenshot of SSView, looking at some DOC file:

Here is a more interesting example – information persisted using Sysinternals Autoruns tool (discussed later):

A more interesting hierarchy is clearly visible – although it’s all in a single file!

The Main Interfaces

The IStorage interface represents a “directory”, that can contain other directories and “files”, represented as IStream interface implementations. To get started a physical file can be created with StgCreateStorageEx or an existing file opened with StgOpenStorageEx. Both return an IStorage pointer on success. From there, methods on IStorage can be called to create or open other directories (storages) and/or files (streams).

The most useful methods on IStorage are CreateStorage, CreateStream, OpenStorage and OpenStream. Enumeration of storages/streams is possible with EnumElements. Here is an example for opening a compound file for read access (filename is from a command line argument):

CComPtr<IStorage> spStg;
auto hr = ::StgOpenStorageEx(argv[1], STGM_READ | STGM_SHARE_EXCLUSIVE,
	STGFMT_STORAGE, 0, nullptr, nullptr, __uuidof(IStorage), reinterpret_cast<void**>(&spStg));
if (FAILED(hr)) {
	printf("Failed to open file (0x%X)\n", hr);
	return hr;
}

The following demonstrates enumerating the hierarchy of a given storage, recursively:

void EnumItems(IStorage* stg, int indent = 0) {
	CComPtr<IEnumSTATSTG> spEnum;
	stg->EnumElements(0, nullptr, 0, &spEnum);
	if (spEnum == nullptr)
		return;

	STATSTG stat;
	while (S_OK == spEnum->Next(1, &stat, nullptr)) {
		if (indent)
			printf(std::string(indent, ' ').c_str());
		printf("%ws", stat.pwcsName);
		if (stat.type == STGTY_STORAGE) {
			printf(" [DIR]\n");
			CComPtr<IStorage> spSubStg;
			stg->OpenStorage(stat.pwcsName, nullptr, 
                STGM_READ | STGM_SHARE_EXCLUSIVE, 0, 0, &spSubStg);
			if (spSubStg)
				EnumItems(spSubStg, indent + 1);
		}
		else
			printf(" (%u bytes)\n", stat.cbSize.LowPart);
		::CoTaskMemFree(stat.pwcsName);
	}
}

Each item has a name, but streams (“files”) can have data. The cbSize member of STATSTG returns that size. A stream is just an abstraction over a bunch of bytes. To actually read/write from/to a stream, it needs to be opened with IStorage::OpenStream before accessing the data with IStream::Read, IStream::Write and similar methods.

More on Streams

The IStream interface is used in various places within the Windows API, not just part of Structured Storage. It represents an abstraction over a buffer, that in theory could be anywhere – that’s the nice thing about an abstraction. Given an IStream pointer, you can read, wrote, seek, copy to another stream, clone, and even commit/revert a transaction, if supported by the implementation. Compound Files, by the way, doesn’t support transactions on streams.

Outside of Structured Storage, streams can be obtained in several ways.

The CreateStreamOnHGlobal API creates a memory buffer over an optional HGLOBAL (can be NULL to allocate a new one) and returns an IStream pointer to that memory buffer. This is useful when dealing with the clipboard for example, as it requires an HGLOBAL, which may not be convenient to work with. By getting an IStream pointer, the code can work with it (maybe reading it from another stream, or manually populating it with data), and then calling GetHGlobalFromStream to get the underlying HGLOBAL before passing it to the clipboard (e.g. SetClipboardData).

A stream can also be obtained for a file directly by calling SHCreateStreamOnFile, providing a convenient access to file data, abstracted as IStream.

Another case where IStream makes an appearance is in ActiveX controls persistence.

Yet another example of using IStream is as a way to “package” information for COM object’s state that would allow creating a proxy to that object from a different apartment by calling CoMarshalInterThreadInterfaceInStream (probably the longest-name COM API), that captures the state required (as an IStream) to pass to another apartment, where the corresponding CoGetInterfaceAndReleaseStream can be called to generate a proxy to the original object, if needed.

Case Study: Autoruns

Back in 2021 when I was working for the Sysinternals team, one of my tasks was to modernize Autoruns from a GUI perspective. I thought I would take this opportunity to do a significant rewrite, so that it would be easier to maintain the tool and improve it as needed. Fortunately, Mark Russinovich was onboard with that, although my time estimate for this project was way off But I digress.

One of the features of Autoruns is the ability to save the information provided by the tool so it can be loaded later, possibly on a different machine. This is non-trivial, as some of the information is not easy to persist, such as icons. I don’t recall if the old Autoruns persisted them, but I definitely wanted to do so.

The old Autoruns format was sequential in nature, storing structures of data linearly in the file. Any new properties that needed to be added would require offset changes, which forced changing the format “version” and make the correct decisions when reading a file in an various “old” formats.

I wanted to make persistence more flexible, so I decided to change the format completely to be a compound file. With this scheme, adding new properties would not cause any issues – a new stream may be added, but other streams are not disturbed. The code could just ignore properties (could be storages and/or streams) it wasn’t aware of. This made the format extensible by definition, immune to any offset changes, and very easy to view using tools, like SSView. The above screenshot is from an Autoruns-persisted file.

Persisting icons, by the way, becomes pretty easy, because ImageList objects, used by Autoruns to hold collection of icons can be persisted to a stream with a single function call: ImageList_Write; very convenient!

Conclusion

The Structured Storage idea is a powerful one, and the Compound File implementation provided by Windows is pretty good and flexible. One of the reasons Microsoft moved Office to a new format was the need to make files smaller, so the new extended formats are ZIP compressed. Their internal format changed as well, and is not using Compound Files for the most part. A Structured Storage file could be compressed, saving disk space, while still maintaining convenient access using storages and streams.

Today I’m happy to announce a new 5-day live training to take place in December, “C++ Programming Masterclass”. This is remote training presented live, where the sessions are recorded for later viewing if you miss any.

The course is scheduled for the beginning of December with 10 half-days spanning two and a half weeks.

The full syllabus, who should attend, with the exact dates and times can be found here: C++ Programming Masterclass.

Here is a quick summary of the course modules:

• Module 1: Introduction to Modern C++
• Module 2: C++ Fundamentals
• Module 3: Standard Library Basics
• Module 4: Object Oriented Design and Programming
• Module 5: Modern Language Features and Libraries (Part 1)
• Module 6: Templates
• Module 7: Modern Language Features and Libraries (Part 2)
• Module 8: The C++ Standard Library
• Module 9: Concurrency in Modern C++

The course is currently heavily discounted, with the a price increase after November 1st. Register here: C++ Programming Masterclass

If you have any questions, feel free to get in touch via email ([email protected]), Linkedin, or X (@zodiacon)

In part 3 we implemented the bulk of what makes a DataStack – push, pop and clear operations. We noted a few remaining deficiencies that need to be taken care of. Let’s begin.

Object Destruction

A DataStack object is deallocated when the last reference to it removed (typically all handles are closed). Any other cleanup must be done explicitly. The DeleteProcedure member of the OBJECT_TYPE_INITIALIZER is an optional callback we can set to be called just before the structure is freed:

init.DeleteProcedure = OnDataStackDelete;

The callback is simple – it’s called with the object about to be destroyed. We can use the cleanup support from part 3 to free the dynamic state of the stacked items:

void OnDataStackDelete(_In_ PVOID Object) {
	auto ds = (DataStack*)Object;
	DsClearDataStack(ds);
}

Querying Information

The native API provides many functions starting with NtQueryInformation* with an object type like process, thread, file, etc. We’ll add a similar function for querying information about DataStack objects. A few declarations are in order, mimicking similar declarations used by other query APIs:

typedef struct _DATA_STACK_CONFIGURATION {
	ULONG MaxItemSize;
	ULONG MaxItemCount;
	ULONG_PTR MaxSize;
} DATA_STACK_CONFIGURATION;

typedef enum _DataStackInformationClass {
	DataStackItemCount,
	DataStackTotalSize,
	DataStackConfiguration,
} DataStackInformationClass;

The query API itself mimics all the other Query APIs in the native API:

NTSTATUS NTAPI NtQueryInformationDataStack(
	_In_ HANDLE DataStackHandle,
	_In_ DataStackInformationClass InformationClass,
	_Out_ PVOID Buffer,
	_In_ ULONG BufferSize,
	_Out_opt_ PULONG ReturnLength);

The implementation (in kernel mode) is not complicated, just verbose. As with other APIs, we’ll start by getting the object itself from the handle, asking for DATA_STACK_QUERY access mask:

NTSTATUS NTAPI NtQueryInformationDataStack(_In_ HANDLE DataStackHandle, 
    _In_ DataStackInformationClass InformationClass, 
    _Out_ PVOID Buffer, _In_ ULONG BufferSize, 
    _Out_opt_ PULONG ReturnLength) {
	DataStack* ds;
	auto status = ObReferenceObjectByHandleWithTag(DataStackHandle,
        DATA_STACK_QUERY, g_DataStackType,
		ExGetPreviousMode(), DataStackTag, (PVOID*)&ds, nullptr);
	if (!NT_SUCCESS(status))
		return status;

Next, we check parameters:

// if no buffer provided then ReturnLength must be 
// non-NULL and buffer size must be zero
//
if (!ARGUMENT_PRESENT(Buffer) && (!ARGUMENT_PRESENT(ReturnLength) || BufferSize != 0))
	return STATUS_INVALID_PARAMETER;
//
// if buffer provided, then size must be non-zero
//
if (ARGUMENT_PRESENT(Buffer) && BufferSize == 0)
	return STATUS_INVALID_PARAMETER;

The rest is pretty standard. Let’s look at one information class:

ULONG len = 0;
switch (InformationClass) {
	case DataStackItemCount: 
        len = sizeof(ULONG); break;
	case DataStackTotalSize: 
        len = sizeof(ULONG_PTR); break;
	case DataStackConfiguration: 
        len = sizeof(DATA_STACK_CONFIGURATION); break;
	default: 
        return STATUS_INVALID_INFO_CLASS;
}

if (BufferSize < len) {
	status = STATUS_BUFFER_TOO_SMALL;
}
else {
	if (ExGetPreviousMode() != KernelMode) {
		__try {
			if (ARGUMENT_PRESENT(Buffer))
				ProbeForWrite(Buffer, BufferSize, 1);
			if (ARGUMENT_PRESENT(ReturnLength))
				ProbeForWrite(ReturnLength, sizeof(ULONG), 1);
		}
		__except (EXCEPTION_EXECUTE_HANDLER) {
			return GetExceptionCode();
		}
	}

	switch (InformationClass) {
		case DataStackItemCount:
		{
			ExAcquireFastMutex(&ds->Lock);
			auto count = ds->Count;
			ExReleaseFastMutex(&ds->Lock);

			if (ExGetPreviousMode() != KernelMode) {
				__try {
					*(ULONG*)Buffer = count;
				}
				__except (EXCEPTION_EXECUTE_HANDLER) {
					return GetExceptionCode();
				}
			}
			else {
				*(ULONG*)Buffer = count;
			}
			break;
		}
//...
//
// set returned bytes if requested
//
if (ARGUMENT_PRESENT(ReturnLength)) {
	if (ExGetPreviousMode() != KernelMode) {
		__try {
			*ReturnLength = len;
		}
		__except (EXCEPTION_EXECUTE_HANDLER) {
			return GetExceptionCode();
		}
	}
	else {
		*ReturnLength = len;
	}
}

ObDereferenceObjectWithTag(ds, DataStackTag);
return status;

You can find the other information classes implemented in the source code in a similar fashion.

To round it up, we’ll add Win32-like APIs that call the native APIs. The Native APIs call the driver in a similar way as the other native API user-mode implementations.

BOOL WINAPI GetDataStackSize(HANDLE hDataStack, ULONG_PTR* pSize) {
	auto status = NtQueryInformationDataStack(hDataStack, 
        DataStackTotalSize, pSize, sizeof(ULONG_PTR), nullptr);
	if (!NT_SUCCESS(status))
		SetLastError(RtlNtStatusToDosError(status));
	return NT_SUCCESS(status);
}

BOOL WINAPI GetDataStackItemCount(HANDLE hDataStack, ULONG* pCount) {
	auto status = NtQueryInformationDataStack(hDataStack,
        DataStackItemCount, pCount, sizeof(ULONG), nullptr);
	if (!NT_SUCCESS(status))
		SetLastError(RtlNtStatusToDosError(status));
	return NT_SUCCESS(status);
}

BOOL WINAPI GetDataStackConfig(HANDLE hDataStack, DATA_STACK_CONFIG* pConfig) {
	auto status = NtQueryInformationDataStack(hDataStack,
        DataStackConfiguration, pConfig, 
        sizeof(DATA_STACK_CONFIG), nullptr);
	if (!NT_SUCCESS(status))
		SetLastError(RtlNtStatusToDosError(status));
	return NT_SUCCESS(status);
}

Waitable Objects

Waitable objects, also called Dispatcher objects, maintain a state called Signaled or Non-Signaled, where the meaning of “signaled” depends on the object type. For example, process objects are signaled when terminated. Same for thread objects. Job objects are signaled when all processes in the job terminate. And so on.

Waitable objects can be waited on with WaitForSingleObject / WaitForMultipleObjects and friends in the Windows API, which call native APIs like NtWaitForSingleObject / NtWaitForMultipleObjects, which eventually get to the kernel and call ObWaitForSingleObject / ObWaitForMultipleObjects which finally invoke KeWaitForSingleObject / KeWaitForMultipleObjects (both documented in the WDK).

It would be nice if DataStack objects would be dispatcher objects, where “signaled” would mean the data stack is not empty, and vice-versa. The first thing to do is make sure that the SYNCHRONIZE access mask is valid for the object type. This is the default, so nothing special to do here. GENERIC_READ also adds SYNCHRONIZE for convenience.

In order to be a dispatcher object, the structure managing the object must start with a DISPATCHER_HEADER structure (which is provided by the WDK headers). For example, KPROCESS and KTHREAD start with DISPATCHER_HEADER. Same for all other dispatcher objects – well, almost. If we look at an EJOB (using symbols), we’ll see the following:

kd> dt nt!_EJOB
   +0x000 Event            : _KEVENT
   +0x018 JobLinks         : _LIST_ENTRY
   +0x028 ProcessListHead  : _LIST_ENTRY
   +0x038 JobLock          : _ERESOURCE
...

The DISPATCHER_HEADER is in the KEVENT. In fact, a KEVENT is just a glorified DISPATCHER_HEADER:

typedef struct _KEVENT {
    DISPATCHER_HEADER Header;
} KEVENT, *PKEVENT, *PRKEVENT;

The advantage of using a KEVENT is that the event API is available – this is taken advantage of by the Job implementation. For processes and threads, the work of signaling is done internally by the Process and Thread APIs.

For the DataStack implementation, we’ll take the Job approach, as the scheduler APIs are internal and undocumented. The DataStack now looks like this:

struct DataStack {
	KEVENT Event;
	LIST_ENTRY Head;
	FAST_MUTEX Lock;
	ULONG Count;
	ULONG MaxItemCount;
	ULONG_PTR Size;
	ULONG MaxItemSize;
	ULONG_PTR MaxSize;
};

In addition, we have to initialize the event as well as the other members:

void DsInitializeDataStack(DataStack* DataStack, ...) {
//...
	KeInitializeEvent(&DataStack->Event, NotificationEvent, FALSE);
//...
}

The event is initialized as a Notification Event (Manual Reset in user mode terminology). Why? This is just a choice. We could extend the DataStack creation API to allow choosing Notification (manual reset) vs. Synchronization (auto reset) – I’ll leave that for interested coder.

Next, we need to set or reset the event when appropriate. It starts in the non-signaled state (the FALSE in KeInitializeEvent), since the data stack starts empty. In the implementation of DsPushDataStack we signal the event if the count is incremented from zero to 1:

NTSTATUS DsPushDataStack(DataStack* ds, PVOID Item, ULONG ItemSize) {
//...
	if (NT_SUCCESS(status)) {
		InsertTailList(&ds->Head, &buffer->Link);
		ds->Count++;
		ds->Size += ItemSize;
		if(ds->Count == 1)
			KeSetEvent(&ds->Event, EVENT_INCREMENT, FALSE);
	}
//...

In the pop implementation, we clear (reset) the event if the item count drops to zero:

NTSTATUS DsPopDataStack(DataStack* ds, PVOID buffer, ULONG inputSize, ULONG* itemSize) {
//...
memcpy(buffer, item->Data, item->Size);
ds->Count--;
ds->Size -= item->Size;
ExFreePool(item);
if (ds->Count == 0)
	KeClearEvent(&ds->Event);
return STATUS_SUCCESS;
//...

These operations are performed under the protection of the fast mutex, of course.

Testing

Here is one way to amend the test application to use WaitForSingleObject:

// wait 5 seconds at most for data to appear
while (WaitForSingleObject(h, 5000) == WAIT_OBJECT_0) {
	DWORD size = sizeof(buffer);
	if (!PopDataStack(h, buffer, &size) && GetLastError() != ERROR_NO_DATA) {
		printf("Error in PopDataStack (%u)\n", GetLastError());
		break;
	}
//...
	DWORD count;
	DWORD_PTR total;
	if (GetDataStackItemCount(h, &count) && GetDataStackSize(h, &total))
		printf("Data stack Item count: %u Size: %zu\n", count, total);
}

Refer to the project source code for the full sample.

Summary

This four-part series demonstrated creating a new kernel object type and using only exported functions to implement it. I hope this sheds more light on certain mechanisms used by the Windows kernel.

In the first part we looked at creating a new kernel object type. In the second part we implemented creation of new DataStack objects and opening existing objects by name. In this part, we’ll implement the main functionality of a DataStack, that makes a DataStack what it is.

Before we get on with that, there is one correction I must make. A good comment from Luke (@lukethezoid) on X said that although the code returns a 32-bit HANDLE to 32-bit callers, it’s not nearly enough because all pointers passed using NtDeviceIoControlFile/DeviceIoControl would be wrong – 32-bit pointers would be passed to the 64-bit kernel and that would cause a crash unless we do some work. For example, a UNICODE_STRING provided by a 32-bit client has a different binary layout than a 64-bit one. Wow64 processes (32-bit x86 running on x64) have two versions of NtDll.Dll in their address space. One is the “native” 64-bit, and the other is a special 32-bit variant. I say “special” because it’s not the same NtDll.Dll you would find on a true 32-bit system. This is because this special DLL knows it’s on a 64-bit system, and provides conversions for parameters (pointer and structures) before invoking the “real” 64-bit DLL API. Here is a snapshot from Process Explorer showing a 32-bit process with two NtDll.Dll files – the native 64-bit loaded into a high address, while the other loaded into a low address (within the first 4GB of address space):

The changes required to support Wow64 processes are not difficult to make, but not too interesting, either, so I won’t be implementing them. Instead, 32-bit clients will be blocked from using DataStacks. We should block on the kernel side for sure, and also in user mode to fail faster. In the kernel, we could use something like this in IRP_MJ_CREATE or IRP_MJ_DEVICE_CONTROL handles:

if (IoIs32bitProcess(Irp)) {
	status = STATUS_NOT_IMPLEMENTED;
    // complete the request...
}

In user mode, we can prevent DataStack.Dll from loading in the first place in Wow64 processes:

BOOL APIENTRY DllMain(HMODULE hModule, DWORD reason, LPVOID) {
	switch (reason) {
		case DLL_PROCESS_ATTACH:
            // C++ 17
			if (BOOL wow; IsWow64Process(GetCurrentProcess(), &wow) && wow)
				return FALSE;
//...

Note, however, that on true 32-bit systems everything should work just fine, as user mode and kernel mode are bitness-aligned.

Implementing the Actual Data Stack

Now we’re ready to focus on implementing the stack functionality – push, pop, and clear.

We’ll start from user mode, and gradually move to kernel mode. First, we want nice APIs for clients to use that have “Win32” conventions like so:

BOOL WINAPI PushDataStack(_In_ HANDLE hDataStack, _In_ const PVOID buffer, _In_ DWORD size);
BOOL WINAPI PopDataStack(_In_ HANDLE hDataStack, _Out_ PVOID buffer, _Inout_ DWORD* size);
BOOL WINAPI ClearDataStack(_In_ HANDLE hDataStack);

The APIs return BOOL to indicate success/failure, and GetLastError could be used to get additional information in case of an error. Let’s start with push:

BOOL WINAPI PushDataStack(HANDLE hDataStack, const PVOID buffer, DWORD size) {
	auto status = NtPushDataStack(hDataStack, buffer, size);
	if (!NT_SUCCESS(status))
		SetLastError(RtlNtStatusToDosError(status));

	return NT_SUCCESS(status);
}

Nothing to it. Call NtPushDataStack and update the last error is needed. NtPushDataStack just packs the arguments in a helper structure and sends to the driver, just like we did with CreateDataStack and OpenDataStack:

NTSTATUS NTAPI NtPushDataStack(_In_ HANDLE DataStackHandle, 
    _In_ const PVOID Item, _In_ ULONG ItemSize) {
	DataStackPush data;
	data.DataStackHandle = DataStackHandle;
	data.Buffer = Item;
	data.Size = ItemSize;

	IO_STATUS_BLOCK ioStatus;
	return NtDeviceIoControlFile(g_hDevice, nullptr, nullptr, nullptr, &ioStatus,
		IOCTL_DATASTACK_PUSH, &data, sizeof(data), nullptr, 0);
}

Now we switch to the kernel side. The DeviceIoControl handler just forwards the arguments to the “real” NtPushDataStack:

case IOCTL_DATASTACK_PUSH:
{
	auto data = (DataStackPush*)Irp->AssociatedIrp.SystemBuffer;
	if (dic.InputBufferLength < sizeof(*data)) {
		status = STATUS_BUFFER_TOO_SMALL;
		break;
	}
	status = NtPushDataStack(data->DataStackHandle, data->Buffer, data->Size);
	break;
}

Now we’re getting to the interesting parts. First, we need to check if the arguments make sense:

NTSTATUS NTAPI NtPushDataStack(HANDLE DataStackHandle, 
    const PVOID Item, ULONG ItemSize) {
	if (ItemSize == 0)
		return STATUS_INVALID_PARAMETER_3;

	if (!ARGUMENT_PRESENT(Item))
		return STATUS_INVALID_PARAMETER_2;

If the pushed data size is zero or the pointer to the data is NULL, then it’s an error. The ARGUMENT_PRESENT macro returns true if the given pointer is not NULL. Notice that we can return specific “invalid parameter” error based on the parameter index. Unfortunately, user mode callers always get a generic ERROR_INVALID_PARAMETER regardless. But at least kernel callers can benefit from the extra detail.

Next, we need to get to the DataStack object corresponding to the given handle; in fact, the handle could be bad, or not point to a DataStack object at all. This is a job for ObReferenceObjectByHandle or one of its variants:

DataStack* ds;
auto status = ObReferenceObjectByHandleWithTag(DataStackHandle,
    DATA_STACK_PUSH, g_DataStackType, ExGetPreviousMode(), 
    DataStackTag, (PVOID*)&ds, nullptr);
if (!NT_SUCCESS(status))
	return status;

ObReferenceObjectByHandleWithTag attempts to retrieve the object pointer given a handle, a type object, and access. The added tag provides a simple way to “track” the caller taking the reference. I’ve defined DataStackTag to be used for dynamic memory allocations as well (we’ll do very soon):

const ULONG DataStackTag = 'ktsD';

It’s the same kind of tag typically provided to allocation functions. You can read the tag from right to left (that’s how it would be displayed in tools as we’ll see later) – “Dstk”, kind of short for “Data Stack”. The function also checks if the handle has the required access mask (DATA_STACK_PUSH in this case), and will fail if the handle is not powerful enough. ExGetPreviousMode is provided to the function to indicate who is the original caller – for kernel callers, any access will be granted (the access mask is not really used).

With the object in hand, we can do the work, delegated to a separate function, and then not to forget to dereference the object or it will leak:

status = DsPushDataStack(ds, Item, ItemSize);
ObDereferenceObjectWithTag(ds, DataStackTag);

return status;

Technically, we don’t have to use a separate function, but it mimics how the kernel works for complex objects: there is another layer of implementation that works with the object directly (no more handles involved).

The DataStack structure mentioned in the previous parts holds the data for the implementation, and will be treated as a classic C structure – no member functions just to mimic how the Windows kernel is implemented – almost everything in C, not C++:

struct DataStack {
	LIST_ENTRY Head;
	FAST_MUTEX Lock;
	ULONG Count;
	ULONG MaxItemCount;
	ULONG_PTR Size;
	ULONG MaxItemSize;
	ULONG_PTR MaxSize;
};

struct DataBlock {
	LIST_ENTRY Link;
	ULONG Size;
	UCHAR Data[1];
};

The “stack” will be managed by a linked list (the classic LIST_ENTRY and friends) implementation provided by the kernel. We’ll push by adding to the tail and pop by removing from the tail. (Clearly, it would be just as easy to implement a queue, rather than, or in addition to, a stack.) We need a lock to prevent corruption of the list, and a fast mutex will do the job. Count stores the number of items currently on the data stack, and Size has the total size in bytes. MaxSize and MaxItemCount are initialized when the DataStack is created and provide some control over the limits of the data stack. Values of zero indicate no special limit.

The second structure, DataBlock is the one that holds the actual data, along with its size, and of course a link to the list. Data[1] is just a placeholder, where the data is going to be copied to, assuming we allocate the correct size.

We’ll start the push implementation in an optimistic manner, and allocate the DataBlock structure with the required size based on the data provided:

NTSTATUS DsPushDataStack(DataStack* ds, PVOID Item, ULONG ItemSize) {
	auto buffer = (DataBlock*)ExAllocatePool2(POOL_FLAG_PAGED | POOL_FLAG_UNINITIALIZED, 
		ItemSize + sizeof(DataBlock), DataStackTag);
	if (buffer == nullptr)
		return STATUS_INSUFFICIENT_RESOURCES;

We use the (relatively) new ExAllocatePool2 API to allocate the memory block (ExAllocatePoolWithTag is deprecated from Windows version 2004, but you can use it with an old-enough WDK or if you turn off the deprecation warning). We allocate the buffer uninitialized, as we’ll copy the data to it very soon, so no need for zeroed buffer. Technically, we allocate one extra byte beyond what we need, but that’s not a big deal. Now we can copy the data from the client’s provided buffer, being careful to probe user-mode buffers under exception protection:

auto status = STATUS_SUCCESS;
if (ExGetPreviousMode() != KernelMode) {
	__try {
		ProbeForRead(Item, ItemSize, 1);
		memcpy(buffer->Data, Item, ItemSize);
	}
	__except (EXCEPTION_EXECUTE_HANDLER) {
		ExFreePool(buffer);
		return GetExceptionCode();
	}
}
else {
	memcpy(buffer->Data, Item, ItemSize);
}
buffer->Size = ItemSize;

If an exception occurs because of a bad user-mode buffer, we can return the exception code and that’s it. Note that ProbeForRead does not work for kernel addresses, and cannot prevent crashes – this is intentional. Only user-mode buffers are out of control from the kernel’s side.

Now we can add the item to the stack if the limits are not violated, while updating the DataStack’s stats:

ExAcquireFastMutex(&ds->Lock);
do {
	if (ds->MaxItemCount == ds->Count) {
		status = STATUS_NO_MORE_ENTRIES;
		break;
	}

	if (ds->MaxItemSize && ItemSize > ds->MaxItemSize) {
		status = STATUS_NOT_CAPABLE;
		break;
	}

	if (ds->MaxSize && ds->Size + ItemSize > ds->MaxSize) {
		status = STATUS_NOT_CAPABLE;
		break;
	}
} while (false);

if (NT_SUCCESS(status)) {
	InsertTailList(&ds->Head, &buffer->Link);
	ds->Count++;
	ds->Size += ItemSize;
}
ExReleaseFastMutex(&ds->Lock);

if (!NT_SUCCESS(status))
	ExFreePool(buffer);

return status;

First, we acquire the fast mutex to prevent data races. I opted not to use any C++ RAII type here to make things as clear as possible – we have to be careful not to return before releasing the fast mutex. Next, the do/while non-loop is used to check if any setting is violated, in which case the status is set to some failure. The status values I chose may not look perfect – the right thing to do is create new NTSTATUS values that would be specific for DataStacks, but I was too lazy. The interested reader/coder is welcome to do it right.

Inserting the item involves calling InsertTailList, and then just updating the item count and total byte size. If anything fails, we are careful to free the buffer to prevent a memory leak. This is for push.

Popping Items

The pop operation works along similar lines. In this case, the client asks to pop an item but needs to provide a large-enough buffer to store the data. We’ll use an additional size pointer argument, that on input indicates the buffer’s size, and on output indicates the actual item size. First, the “Win32” API:

BOOL WINAPI PopDataStack(HANDLE hDataStack, PVOID buffer, DWORD* size) {
	auto status = NtPopDataStack(hDataStack, buffer, size);
	if (!NT_SUCCESS(status))
		SetLastError(RtlNtStatusToDosError(status));

	return NT_SUCCESS(status);
}

Just delegating the work to the native API, which forwards to the kernel with a helper structure:

NTSTATUS NTAPI NtPopDataStack(_In_ HANDLE DataStackHandle, _In_ PVOID Buffer, _Inout_ PULONG ItemSize) {
	DataStackPop data;
	data.DataStackHandle = DataStackHandle;
	data.Buffer = Buffer;
	data.Size = ItemSize;

	IO_STATUS_BLOCK ioStatus;
	return NtDeviceIoControlFile(g_hDevice, nullptr, nullptr, nullptr, &ioStatus,
		IOCTL_DATASTACK_POP, &data, sizeof(data), nullptr, 0);
}

This should be expected by now. On the kernel side, things are more interesting. First, get the object based on the handle, then send it to the lower-layer function if successful:

NTSTATUS NTAPI NtPopDataStack(HANDLE DataStackHandle, 
    PVOID Buffer, PULONG BufferSize) {
	if (!ARGUMENT_PRESENT(BufferSize))
		return STATUS_INVALID_PARAMETER_3;

	ULONG size;
	if (ExGetPreviousMode() != KernelMode) {
		__try {
			ProbeForRead(BufferSize, sizeof(ULONG), 1);
			size = *BufferSize;
		}
		__except (EXCEPTION_EXECUTE_HANDLER) {
			return GetExceptionCode();
		}
	}
	else {
		size = *BufferSize;
	}

	if (!ARGUMENT_PRESENT(Buffer) && size != 0)
		return STATUS_INVALID_PARAMETER_2;

	DataStack* ds;
	auto status = ObReferenceObjectByHandleWithTag(DataStackHandle,
        DATA_STACK_POP, g_DataStackType, ExGetPreviousMode(), 
        DataStackTag, (PVOID*)&ds, nullptr);
	if (!NT_SUCCESS(status))
		return status;

	status = DsPopDataStack(ds, Buffer, size, BufferSize);
	ObDereferenceObjectWithTag(ds, DataStackTag);
	return status;
}

The input buffer size is extracted, being careful to probe the user mode pointer. The real work is done in DsPopDataStack. First, take the lock. Second, see if the data stack is empty – if so, no pop operation possible. If the input size is zero, return the size of the top element:

NTSTATUS DsPopDataStack(DataStack* ds, PVOID buffer, 
    ULONG inputSize, ULONG* itemSize) {
	ExAcquireFastMutex(&ds->Lock);
	__try {
		if (inputSize == 0) {
			//
			// return size of next item
			//			
			__try {
				if (ds->Count == 0) {
					//
					// stack empty
					//
					*itemSize = 0;
				}
				else {
					auto top = CONTAINING_RECORD(ds->Head.Blink, DataBlock, Link);
					*itemSize = top->Size;
				}
				return STATUS_SUCCESS;
			}
			__except (EXCEPTION_EXECUTE_HANDLER) {
				return GetExceptionCode();
			}
		}

The locking here works differently than the push implementation by using a __finally block, which is the one releasing the fast mutex. This ensures that no matter how we leave the __try block, the lock will be released for sure.

The CONTAINING_RECORD macro is used correctly to get to the item from the link (LIST_ENTRY). Technically, in this case we could just make a simple cast, as the LIST_ENTRY member is the first in a DataBlock. Notice how we get to the top item: Head.Blink, which points to the tail (last) item.

If the data stack is empty, we place zero in the item size pointer and return an error (abusing yet another existing error):

if (ds->Count == 0) {
	__try {
		*itemSize = 0;
	}
	__except (EXCEPTION_EXECUTE_HANDLER) {
		return GetExceptionCode();
	}
	return STATUS_PIPE_EMPTY;
}

If manage to get beyond this point, then there is an item, and we need to remove it, copy the data to the client’s buffer (if it’s big enough), and free the kernel’s copy of the buffer:

	auto link = RemoveTailList(&ds->Head);
	NT_ASSERT(link != &ds->Head);
	
	auto item = CONTAINING_RECORD(link, DataBlock, Link);
	__try {
		*itemSize = item->Size;
		if (inputSize < item->Size) {
			//
			// buffer too small
			// reinsert item
			//
			InsertTailList(&ds->Head, link);
			return STATUS_BUFFER_TOO_SMALL;
		}
		else {
			memcpy(buffer, item->Data, item->Size);
			ds->Count--;
			ds->Size -= item->Size;
			ExFreePool(item);
			return STATUS_SUCCESS;
		}
	}
	__except (EXCEPTION_EXECUTE_HANDLER) {
		return GetExceptionCode();
	}
}
__finally {
	ExReleaseFastMutex(&ds->Lock);
}

The call to RemoveTailList removes the top item from the list. The next assert verifies the list wasn’t empty before the removal (it can’t be as we dealt with that case in the previous code section). Remember, that if a list is empty calling RemoveTailList or RemoveHeadList returns the head’s pointer.

If the client’s buffer is too small, we reinsert the item back and bail. Otherwise, we copy the data, update the data stack’s stats and free our copy of the item.

Cleanup

The stack clear operation is relatively straightforward of them all. Here is the kernel part that matters:

NTSTATUS DsClearDataStack(DataStack* ds) {
	ExAcquireFastMutex(&ds->Lock);
	LIST_ENTRY* link;

	while ((link = RemoveHeadList(&ds->Head)) != &ds->Head) {
		auto item = CONTAINING_RECORD(link, DataBlock, Link);
		ExFreePool(item);
	}
	ds->Count = 0;
	ds->Size = 0;
	ExReleaseFastMutex(&ds->Lock);

	return STATUS_SUCCESS;
}

We take the lock, and then go over the list, removing and freeing each item. Finally, we update the stats to zero items and zero bytes.

Testing

Here is one way to test – having an executable run twice, the first instance pushes some items, and the second one popping items. main creates a data stack with a name. If it’s a new object, it assumes the role of “pusher”. Otherwise, it assumes the role of “popper”:

int main() {
	HANDLE hDataStack = CreateDataStack(nullptr, 0, 100, 
        10 << 20, L"MyDataStack");
	if (!hDataStack) {
		printf("Failed to create data stack (%u)\n", GetLastError());
		return 1;
	}

	printf("Handle created: 0x%p\n", hDataStack);

	if (GetLastError() == ERROR_ALREADY_EXISTS) {
		printf("Opened an existing object... will pop elements\n");
		PopItems(hDataStack);
	}
	else {
		Sleep(5000);

		PushString(hDataStack, "Hello, data stack!");
		PushString(hDataStack, "Pushing another string...");
		for (int i = 1; i <= 10; i++) {
			Sleep(100);
			PushDataStack(hDataStack, &i, sizeof(i));
		}
	}

	CloseHandle(hDataStack);
	return 0;
}

When creating a named object, if GetLastError returns ERROR_ALREADY_EXISTS, it means a handle is returned to an existing object. In our current implementation, this actually won’t work. We have to fix the CreateDataStack implementation like so:

HANDLE hDataStack;
auto status = NtCreateDataStack(&hDataStack, &attr, maxItemSize, maxItemCount, maxSize);
if (NT_SUCCESS(status)) {
	const NTSTATUS STATUS_OBJECT_NAME_EXISTS = 0x40000000;

	if (status == STATUS_OBJECT_NAME_EXISTS) {
		SetLastError(ERROR_ALREADY_EXISTS);
	}
	else {
		SetLastError(0);
	}
	return hDataStack;
}

After calling NtCreateDataStack we fix the returned “error” if the kernel returns STATUS_OBJECT_NAME_EXISTS. Now the previous will work correctly.

PushString is a little helper to push strings:

bool PushString(HANDLE h, std::string const& text) {
	auto ok = PushDataStack(h, (PVOID)text.c_str(), (ULONG)text.length() + 1);
	if (!ok)
		printf("Error in PushString: %u\n", GetLastError());
	return ok;
}

Finally, PopItems does some popping:

void PopItems(HANDLE h) {
	BYTE buffer[256];

	auto tick = GetTickCount64();
	while (GetTickCount64() - tick < 10000) {
		DWORD size = sizeof(buffer);
		if (!PopDataStack(h, buffer, &size) && GetLastError() != ERROR_NO_DATA) {
			printf("Error in PopDataStack (%u)\n", GetLastError());
			break;
		}
		if (size) {
			printf("Popped %u bytes: ", size);
			if (size > sizeof(int))
				printf("%s\n", (PCSTR)buffer);
			else
				printf("%d\n", *(int*)buffer);
		}
		Sleep(300);
	}
}

Not very exciting, but is good enough for this simple test. Here is some output, first from the “pusher” and then the “popper”:

E:\Test>DSTest.exe
Handle created: 0x00000000000000F8

E:\Test>DSTest.exe
Handle created: 0x0000000000000104
Opened an existing object... will popup elements
Popped 4 bytes: 2
Popped 4 bytes: 5
Popped 4 bytes: 8
Popped 4 bytes: 10
Popped 4 bytes: 9
Popped 4 bytes: 7
Popped 4 bytes: 6
Popped 4 bytes: 4
Popped 4 bytes: 3
Popped 4 bytes: 1
Popped 26 bytes: Pushing another string...
Popped 19 bytes: Hello, data stack!

What’s Next?

Are we done? Not quite. Astute readers may have noticed a little problem. What happens if a DataStack object is destroyed (e.g., the last handle to it is closed), but the stack is not empty? That memory will leak, as we have no “desctructor”. Running the “pusher” a few times without a second process that pops items results in a leak. Here is my PoolMonX tool showing the leak:

Notice the “Dstk” tag and the number of allocations being higher that deallocations.

Another feature we are missing is the ability to wait on a DataStack until data is available, if the stack is empty, maybe by calling the WaitForSingleObject API. It would be nice to have that.

Yet another missing element is the ability to query DataStack objects – how much memory is being used, how many items, etc.

We’ll deal with these aspects in the next part.

In Part 1 we’ve seen how to create a new kernel object type. The natural next step is to implement some functionality associated with the new object type. Before we dive into that, let’s take a broader view of what we’re trying to do. For comparison purposes, we can take an existing kernel object type, such as a Semaphore or a Section, or any other object type, look at how it’s “invoked” to get an idea of what we need to do.

A word of warning: this is a code-heavy post, and assumes the reader is fairly familiar with Win32 and native API conventions, and has basic understanding of device driver writing.

The following diagram shows the call flow when creating a semaphore from user mode starting with the CreateSemaphore(Ex) API:

A process calls the officially documented CreateSemaphore, implemented in kernel32.dll. This calls the native (undocumented) API NtCreateSemaphore, converting arguments as needed from Win32 conventions to native conventions. NtCreateSemaphore has no “real” implementation in user mode, as the kernel is the only one which can create a semaphore (or any other kernel object for that matter). NtDll has code to transition the CPU to kernel mode by using the syscall machine instruction on x64. Before issuing a syscall, the code places a number into the EAX CPU register. This number – system service index, indicates what operation is being requested.

On the kernel side of things, the System Service Dispatcher uses the value in EAX as an index into the System Service Descriptor Table (SSDT) to locate the actual function to call, pointing to the real NtCreateSemaphore implementation. Semaphores are relatively simple objects, so creation is a matter of allocating memory for a KSEMAPHORE structure (and a header), done with OnCreateObject, initializing the structure, and then inserting the object into the system (ObInsertObject).

More complex objects are created similarly, although the actual creation code in the kernel may be more elaborate. Here is a similar diagram for creating a Section object:

As can be seen in the diagram, creating a section involves a private function (MiCreateSection), but the overall process is the same.

We’ll try to mimic creating a DataStack object in a similar way. However, extending NtDll for our purposes is not an option. Even using syscall to make the transition to the kernel is problematic for the following reasons:

• There is no entry in the SSDT for something like NtCreateDataStack, and we can’t just add an entry because PatchGuard does not like when the SSDT changes.
• Even if we could add an entry to the SSDT safely, the entry itself is tricky. On x64, it’s not a 64-bit address. Instead, it’s a 28-bit offset from the beginning of the SSDT (the lower 4 bits store the number of parameters passed on the stack), which means the function cannot be too far from the SSDT’s address. Our driver can be loaded to any address, so the offset to anything mapped may be too large to be stored in an SSDT entry.
• We could fix that problem perhaps by adding code in spare bytes at the end of the kernel mapped PE image, and add a JMP trampoline call to our real function…

Not easy, and we still have the PatchGuard issue. Instead, we’ll go about it in a simpler way – use DeviceIoControl (or the native NtDeviceIoControlFile) to pass the parameters to our driver. The following diagram illustrates this:

We’ll keep the “Win32 API” functions and “Native APIs” implemented in the same DLL for convenience. Let’s from the top, moving from user space to kernel space. Implementing CreateDataStack involves converting Win32 style arguments to native-style arguments before calling NtCreateDataStack. Here is the beginning:

HANDLE CreateDataStack(_In_opt_ SECURITY_ATTRIBUTES* sa, 
    _In_ ULONG maxItemSize, _In_ ULONG maxItemCount, 
    _In_ ULONG_PTR maxSize, _In_opt_ PCWSTR name) {

Notice the similarity to functions like CreateSemaphore, CreateMutex, CreateFileMapping, etc. An optional name is accepted, as DataStack objects can be named.

Native APIs work with UNICODE_STRINGs and OBJECT_ATTRIBUTES, so we need to do some work to be able to call the native API:

NTSTATUS NTAPI NtCreateDataStack(_Out_ PHANDLE DataStackHandle, 
    _In_opt_ POBJECT_ATTRIBUTES DataStackAttributes, 
    _In_ ULONG MaxItemSize, _In_ ULONG MaxItemCount, ULONG_PTR MaxSize);

We start by building an OBJECT_ATTRIBUTES:

UNICODE_STRING uname{};
if (name && *name) {
	RtlInitUnicodeString(&uname, name);
}
OBJECT_ATTRIBUTES attr;
InitializeObjectAttributes(&attr, 
	uname.Length ? &uname : nullptr, 
	OBJ_CASE_INSENSITIVE | (sa && sa->bInheritHandle ? OBJ_INHERIT : 0) | (uname.Length ? OBJ_OPENIF : 0),
	uname.Length ? GetUserDirectoryRoot() : nullptr, 
	sa ? sa->lpSecurityDescriptor : nullptr);

If a name exists, we wrap it in a UNICODE_STRING. The security attributes are used, if provided. The most interesting part is the actual name (if provided). When calling a function like the following:

CreateSemaphore(nullptr, 100, 100, L"MySemaphore");

The object name is not going to be just “MySemaphore”. Instead, it’s going to be something like “\Sessions\1\BaseNamedObjects\MySemaphore”. This is because the Windows API uses “local” session-relative names by default. Our DataStack API should provide the same semantics, which means the base directory in the Object Manager’s namespace for the current session must be used. This is the job of GetUserDirectoryRoot. Here is one way to implement it:

HANDLE GetUserDirectoryRoot() {
	static HANDLE hDir;
	if (hDir)
		return hDir;

	DWORD session = 0;
	ProcessIdToSessionId(GetCurrentProcessId(), &session);

	UNICODE_STRING name;
	WCHAR path[256];
	if (session == 0)
		RtlInitUnicodeString(&name, L"\\BaseNamedObjects");
	else {
		wsprintfW(path, L"\\Sessions\\%u\\BaseNamedObjects", session);
		RtlInitUnicodeString(&name, path);
	}
	OBJECT_ATTRIBUTES dirAttr;
	InitializeObjectAttributes(&dirAttr, &name, OBJ_CASE_INSENSITIVE, nullptr, nullptr);
	NtOpenDirectoryObject(&hDir, DIRECTORY_QUERY, &dirAttr);
	return hDir;
}

We just need to do that once, since the resulting directory handle can be stored in a global/static variable for the lifetime of the process; we won’t even bother closing the handle. The native NtOpenDirectoryObject is used to open a handle to the correct directory and return it. Notice that for session 0, there is a special rule: its directory is simply “\BaseNamedObjects”.

There is a snag in the above handling, as it’s incomplete. UWP processes have their own object directory based on their AppContainer SID, which looks like “\Sessions\1\AppContainerNamedObjects\{AppContainerSid}”, which the code above is not dealing with. I’ll leave that as an exercise for the interested coder.

Back in CreateDataStack – the session-relative directory handle is stored in the OBJECT_ATTRIBUTES RootDirectory member. Now we can call the native API:

HANDLE hDataStack;
auto status = NtCreateDataStack(&hDataStack, &attr, maxItemSize, maxItemCount, maxSize);
if (NT_SUCCESS(status))
	return hDataStack;

SetLastError(RtlNtStatusToDosError(status));
return nullptr;

If we get a failed status, we convert it to a Win32 error with RtlNtStatusToDosError and call SetLastError to make it available to the caller via the usual GetLastError. Here is the full CreateDataStack function for easier reference:

HANDLE CreateDataStack(_In_opt_ SECURITY_ATTRIBUTES* sa, 
    _In_ ULONG maxItemSize, _In_ ULONG maxItemCount, 
    _In_ ULONG_PTR maxSize, _In_opt_ PCWSTR name) {
	UNICODE_STRING uname{};
	if (name && *name) {
		RtlInitUnicodeString(&uname, name);
	}
	OBJECT_ATTRIBUTES attr;
	InitializeObjectAttributes(&attr, 
		uname.Length ? &uname : nullptr, 
		OBJ_CASE_INSENSITIVE | (sa && sa->bInheritHandle ? OBJ_INHERIT : 0) | (uname.Length ? OBJ_OPENIF : 0),
		uname.Length ? GetUserDirectoryRoot() : nullptr, 
		sa ? sa->lpSecurityDescriptor : nullptr);
	
	HANDLE hDataStack;
	auto status = NtCreateDataStack(&hDataStack, &attr, maxItemSize, maxItemCount, maxSize);
	if (NT_SUCCESS(status))
		return hDataStack;

	SetLastError(RtlNtStatusToDosError(status));
	return nullptr;
}

Next, we need to handle the native implementation. Since we just call our driver, we package the arguments in a helper structure and send it to the driver via NtDeviceIoControlFile:

NTSTATUS NTAPI NtCreateDataStack(_Out_ PHANDLE DataStackHandle,
    _In_opt_ POBJECT_ATTRIBUTES DataStackAttributes, 
    _In_ ULONG MaxItemSize, _In_ ULONG MaxItemCount, ULONG_PTR MaxSize) {
	DataStackCreate data;
	data.MaxItemCount = MaxItemCount;
	data.MaxItemSize = MaxItemSize;
	data.ObjectAttributes = DataStackAttributes;
	data.MaxSize = MaxSize;

	IO_STATUS_BLOCK ioStatus;
	return NtDeviceIoControlFile(g_hDevice, nullptr, nullptr,
        nullptr, &ioStatus, IOCTL_DATASTACK_CREATE, 
        &data, sizeof(data), DataStackHandle, sizeof(HANDLE));
}

Where is g_Device coming from? When our DataStack.Dll is loaded into a process, we can open a handle to the device exposed by the driver (which we have yet to implement). In fact, if we can’t obtain a handle, the DLL should fail to load:

HANDLE g_hDevice = INVALID_HANDLE_VALUE;

bool OpenDevice() {
	UNICODE_STRING devName;
	RtlInitUnicodeString(&devName, L"\\Device\\KDataStack");
	OBJECT_ATTRIBUTES devAttr;
	InitializeObjectAttributes(&devAttr, &devName, 0, nullptr, nullptr);
	IO_STATUS_BLOCK ioStatus;
	return NT_SUCCESS(NtOpenFile(&g_hDevice, GENERIC_READ | GENERIC_WRITE, &devAttr, &ioStatus, 0, 0));
}

void CloseDevice() {
	if (g_hDevice != INVALID_HANDLE_VALUE) {
		CloseHandle(g_hDevice);
		g_hDevice = INVALID_HANDLE_VALUE;
	}
}

BOOL APIENTRY DllMain(HMODULE hModule, DWORD reason, LPVOID) {
	switch (reason) {
		case DLL_PROCESS_ATTACH:
			DisableThreadLibraryCalls(hModule);
			return OpenDevice();

		case DLL_THREAD_ATTACH:
		case DLL_THREAD_DETACH:
		case DLL_PROCESS_DETACH:
			CloseDevice();
			break;
	}
	return TRUE;
}

OpenDevice uses the native NtOpenFile to open a handle, as the driver does not provide a symbolic link to make it slightly harder to reach it directly from user mode. If OpenDevice returns false, the DLL will unload.

Kernel Space

Now we move to the kernel side of things. Our driver must create a device object and expose IOCTLs for calls made from user mode. The additions to DriverEntry are pretty standard:

extern "C" NTSTATUS
DriverEntry(PDRIVER_OBJECT DriverObject, PUNICODE_STRING RegistryPath) {
	UNREFERENCED_PARAMETER(RegistryPath);

	auto status = DsCreateDataStackObjectType();
	if (!NT_SUCCESS(status)) {
		return status;
	}

	UNICODE_STRING devName = RTL_CONSTANT_STRING(L"\\Device\\KDataStack");
	PDEVICE_OBJECT devObj;
	status = IoCreateDevice(DriverObject, 0, &devName, FILE_DEVICE_UNKNOWN, 0, FALSE, &devObj);
	if (!NT_SUCCESS(status))
		return status;

	DriverObject->DriverUnload = OnUnload;
	DriverObject->MajorFunction[IRP_MJ_CREATE] = 
    DriverObject->MajorFunction[IRP_MJ_CLOSE] =
		[](PDEVICE_OBJECT, PIRP Irp) -> NTSTATUS {
		Irp->IoStatus.Status = STATUS_SUCCESS;
		IoCompleteRequest(Irp, IO_NO_INCREMENT);
		return STATUS_SUCCESS;
		};

	DriverObject->MajorFunction[IRP_MJ_DEVICE_CONTROL] = OnDeviceControl;

	return STATUS_SUCCESS;
}

The driver creates a single device object with the name “\Device\DataStack” that was used in DllMain to open a handle to that device. IRP_MJ_CREATE and IRP_MJ_CLOSE are supported to make the driver usable. Finally, IRP_MJ_DEVICE_CONTROL handling is set up (OnDeviceControl).

The job of OnDeviceControl is to propagate the data provided by helper structures to the real implementation of the native APIs. Here is the code that covers IOCTL_DATASTACK_CREATE:

NTSTATUS OnDeviceControl(PDEVICE_OBJECT, PIRP Irp) {
	auto stack = IoGetCurrentIrpStackLocation(Irp);
	auto& dic = stack->Parameters.DeviceIoControl;
	auto len = 0U;
	auto status = STATUS_INVALID_DEVICE_REQUEST;

	switch (dic.IoControlCode) {
		case IOCTL_DATASTACK_CREATE:
		{
			auto data = (DataStackCreate*)Irp->AssociatedIrp.SystemBuffer;
			if (dic.InputBufferLength < sizeof(*data)) {
				status = STATUS_BUFFER_TOO_SMALL;
				break;
			}
			HANDLE hDataStack;
			status = NtCreateDataStack(&hDataStack, 
                data->ObjectAttributes, 
                data->MaxItemSize, 
                data->MaxItemCount, 
                data->MaxSize);
			if (NT_SUCCESS(status)) {
				len = IoIs32bitProcess(Irp) ? sizeof(ULONG) : sizeof(HANDLE);
				memcpy(data, &hDataStack, len);
			}
			break;
		}
	}

	Irp->IoStatus.Status = status;
	Irp->IoStatus.Information = len;
	IoCompleteRequest(Irp, IO_NO_INCREMENT);
	return status;
}

NtCreateDataStack is called with the unpacked arguments. The only trick here is the use of IoIs32bitProcess to check if the calling process is 32-bit. If so, 4 bytes should be copied back as the handle instead of 8 bytes.

The real work of creating a DataStack object (finally), falls on NtCreateDataStack. First, we need to have a structure that manages DataStack objects. Here it is:

struct DataStack {
	LIST_ENTRY Head;
	FAST_MUTEX Lock;
	ULONG Count;
	ULONG MaxItemCount;
	ULONG_PTR Size;
	ULONG MaxItemSize;
	ULONG_PTR MaxSize;
};

The details are not important now, since we’re dealing with object creation only. But we should initialize the structure properly when the object is created. The first major step is telling the kernel to create a new object of DataStack type:

NTSTATUS NTAPI NtCreateDataStack(_Out_ PHANDLE DataStackHandle,
    _In_opt_ POBJECT_ATTRIBUTES DataStackAttributes, 
    _In_ ULONG MaxItemSize, _In_ ULONG MaxItemCount, ULONG_PTR MaxSize) {
	auto mode = ExGetPreviousMode();
	extern POBJECT_TYPE g_DataStackType;
	//
	// sanity check
	//
	if (g_DataStackType == nullptr)
		return STATUS_NOT_FOUND;

	DataStack* ds;
	auto status = ObCreateObject(mode, g_DataStackType, DataStackAttributes, mode, 
		nullptr, sizeof(DataStack), 0, 0, (PVOID*)&ds);
	if (!NT_SUCCESS(status)) {
		KdPrint(("Error in ObCreateObject (0x%X)\n", status));
		return status;
	}

ObCreateObject looks like this:

NTSTATUS NTAPI ObCreateObject(
	_In_ KPROCESSOR_MODE ProbeMode,
	_In_ POBJECT_TYPE ObjectType,
	_In_opt_ POBJECT_ATTRIBUTES ObjectAttributes,
	_In_ KPROCESSOR_MODE OwnershipMode,
	_Inout_opt_ PVOID ParseContext,
	_In_ ULONG ObjectBodySize,
	_In_ ULONG PagedPoolCharge,
	_In_ ULONG NonPagedPoolCharge,
	_Deref_out_ PVOID* Object);

ExGetPreviousMode returns the caller’s mode (UserMode or KernelMode enum values), and based off of that we ask ObCreateObject to make the relevant probing and security checks. ObjectType is our DataStack type object, ObjectBodySize is sizeof(DataStack), our data structure. The last parameter is where the object pointer is returned.

If this succeeds, we need to initialize the structure appropriately, and then add the object to the system “officially”, where the object header would be built as well:

DsInitializeDataStack(ds, MaxItemSize, MaxItemCount, MaxSize);
HANDLE hDataStack;
status = ObInsertObject(ds, nullptr, DATA_STACK_ALL_ACCESS, 0, nullptr, &hDataStack);
if (NT_SUCCESS(status)) {
	*DataStackHandle = hDataStack;
}
else {
	KdPrint(("Error in ObInsertObject (0x%X)\n", status));
}
return status;

DsInitializeDataStack is a helper function to initialize an empty DataStack:

void DsInitializeDataStack(DataStack* DataStack, ULONG MaxItemSize, ULONG MaxItemCount, ULONG_PTR MaxSize) {
	InitializeListHead(&DataStack->Head);
	ExInitializeFastMutex(&DataStack->Lock);
	DataStack->Count = 0;
	DataStack->MaxItemCount = MaxItemCount;
	DataStack->Size = 0;
	DataStack->MaxItemSize = MaxItemSize;
	DataStack->MaxSize = MaxSize;
}

This is it for CreateDataStack and its chain of called functions. Handling OpenDataStack is similar, and simpler, as the heavy lifting is done by the kernel.

Opening an Existing DataStack Object

OpenDataStack attempts to open a handle to an existing DataStack object by name:

HANDLE OpenDataStack(_In_ ACCESS_MASK desiredAccess, _In_ BOOL inheritHandle, _In_ PCWSTR name) {
	if (name == nullptr || *name == 0) {
		SetLastError(ERROR_INVALID_NAME);
		return nullptr;
	}

	UNICODE_STRING uname;
	RtlInitUnicodeString(&uname, name);
	OBJECT_ATTRIBUTES attr;
	InitializeObjectAttributes(&attr,
		&uname,
		OBJ_CASE_INSENSITIVE | (inheritHandle ? OBJ_INHERIT : 0),
		GetUserDirectoryRoot(),
		nullptr);
	HANDLE hDataStack;
	auto status = NtOpenDataStack(&hDataStack, desiredAccess, &attr);
	if (NT_SUCCESS(status))
		return hDataStack;

	SetLastError(RtlNtStatusToDosError(status));
	return nullptr;
}

Again, from a high-level perspective it looks similar to APIs like OpenSemaphore or OpenEvent. NtOpenDataStack will make a call to the driver via NtDeviceIoControlFile, packing the arguments:

NTSTATUS NTAPI NtOpenDataStack(_Out_ PHANDLE DataStackHandle, 
    _In_ ACCESS_MASK DesiredAccess, 
    _In_ POBJECT_ATTRIBUTES DataStackAttributes) {
	DataStackOpen data;
	data.DesiredAccess = DesiredAccess;
	data.ObjectAttributes = DataStackAttributes;

	IO_STATUS_BLOCK ioStatus;
	return NtDeviceIoControlFile(g_hDevice, nullptr, nullptr, nullptr, &ioStatus,
		IOCTL_DATASTACK_OPEN, &data, sizeof(data), DataStackHandle, sizeof(HANDLE));
}

Finally, the implementation of NtOpenDataStack in the kernel is surprisingly simple:

NTSTATUS NTAPI NtOpenDataStack(_Out_ PHANDLE DataStackHandle, 
    _In_ ACCESS_MASK DesiredAccess, 
    _In_ POBJECT_ATTRIBUTES DataStackAttributes) {
	return ObOpenObjectByName(DataStackAttributes, g_DataStackType, ExGetPreviousMode(),
		nullptr, DesiredAccess, nullptr, DataStackHandle);
}

The simplicity is thanks to the generic ObOpenObjectByName kernel API, which is not documented, but is exported, that attempts to open a handle to any named object:

NTSTATUS ObOpenObjectByName(
	_In_ POBJECT_ATTRIBUTES ObjectAttributes,
	_In_ POBJECT_TYPE ObjectType,
	_In_ KPROCESSOR_MODE AccessMode,
	_Inout_opt_ PACCESS_STATE AccessState,
	_In_opt_ ACCESS_MASK DesiredAccess,
	_Inout_opt_ PVOID ParseContext,
	_Out_ PHANDLE Handle);

That’s it for creating and opening a DataStack object. Let’s test it!

Testing

After deploying the driver to a test machine, we can write simple code to create a DataStack object (named or unnamed), and see if it works. Then, we’ll close the handle:

#include <Windows.h>
#include <stdio.h>
#include "..\DataStack\DataStackAPI.h"

int main() {
	HANDLE hDataStack = CreateDataStack(nullptr, 0, 100, 10 << 20, L"MyDataStack");
	if (!hDataStack) {
		printf("Failed to create data stack (%u)\n", GetLastError());
		return 1;
	}

	printf("Handle created: 0x%p\n", hDataStack);

	auto hOpen = OpenDataStack(GENERIC_READ, FALSE, L"MyDataStack");
	if (!hOpen) {
		printf("Failed to open data stack (%u)\n", GetLastError());
		return 1;
	}

	CloseHandle(hDataStack);
	CloseHandle(hOpen);
	return 0;
}

Here is what Process Explorer shows when the handle is open, but not yet closed:

Let’s check the kernel debugger:

kd> !object \Sessions\2\BaseNamedObjects\MyDataStack
Object: ffffc785bb6e8430  Type: (ffffc785ba4fd830) DataStack
    ObjectHeader: ffffc785bb6e8400 (new version)
    HandleCount: 1  PointerCount: 32769
    Directory Object: ffff92013982fe70  Name: MyDataStack
lkd> dt nt!_OBJECT_TYPE ffffc785ba4fd830
   +0x000 TypeList         : _LIST_ENTRY [ 0xffffc785`bb6e83e0 - 0xffffc785`bb6e83e0 ]
   +0x010 Name             : _UNICODE_STRING "DataStack"
   +0x020 DefaultObject    : (null) 
   +0x028 Index            : 0x4c 'L'
   +0x02c TotalNumberOfObjects : 1
   +0x030 TotalNumberOfHandles : 1
   +0x034 HighWaterNumberOfObjects : 1
   +0x038 HighWaterNumberOfHandles : 2
...

After opening the second handle (by name), the debugger reports two handles (different run):

lkd> !object ffffc585f68e25f0
Object: ffffc585f68e25f0  Type: (ffffc585ee55df10) DataStack
    ObjectHeader: ffffc585f68e25c0 (new version)
    HandleCount: 2  PointerCount: 3
    Directory Object: ffffaf8deb3c60a0  Name: MyDataStack

The source code can be found here.

In future parts, we’ll implement the actual DataStack functionality.

Windows provides much of its functionality via kernel objects. Common examples are processes, threads, mutexes, semaphores, sections, and many more. We can see the object types supported on a particular Windows system by using a tool such as Object Explorer, or in a more limited way – WinObj. Here is a view from Object Explorer:

Every object type has a name (e.g. “Process”), an index, the pool to use for allocating memory for these kind of objects (typically Non Paged pool or Paged pool), and a few more properties. The “dynamic” part in the above screenshot shows that an object type keeps track of the number of objects and handles currently used for that type. Object types are themselves objects, as is perhaps evident from the fact that there is an object type called “Type” (index 2, the first one), and its number of “objects” is the number of object types supported on this version of Windows.

User mode clients use kernel objects by invoking APIs. For example, CreateProcess creates a process object and a thread object, returning handles to both. OpenProcess, on the other hand, tries to obtain a handle to an existing process object, given its process ID and the access mask requested. Similar APIs exist for other object types. All this should be fairly familiar to reader of this blog.

A New Kernel Object Type

Can we create a new kernel object type, that provides some useful functionality to user-mode (and kernel-mode) clients? Perhaps we should first ask, why would we want to do that? As an alternative, we can create a kernel driver that exposes the desired functionality through I/O control codes, invoked with DeviceIoControl by clients. We can certainly create nice wrappers that provide nicer-looking functions so that clients do not need to see the DeviceIoControl calls.

I can see at two reasons to go the “new object type” approach:

• It’s a great learning opportunity, and could be fun
• We get lots of things for free, such as handle and objects management (handle count, ref count), sharing capabilities, just like any other kernel object, and some common APIs clients are already familiar with, like CloseHandle.

OK, let’s assume we want to go down this route. How do we create an object type, or a “generic” kernel object, for that matter. As it turns out, the kernel functions needed are exported, but they are not documented. We’ll use them anyway

We’ll create an Empty WDM Driver in Visual Studio, delete the INF file so we are left with an empty project with the correct settings for the compiler and linker. For more information on creating driver projects, consult the official documentation, or my book, “Windows Kernel Programming, 2nd edition“.

We’ll add a C++ file to the project, and write our DriverEntry routine. At first, its only job is to create a new type object:

extern "C" NTSTATUS
DriverEntry(PDRIVER_OBJECT DriverObject, PUNICODE_STRING) {
    DriverObject->DriverUnload = OnUnload;
    return DsCreateDataStackObjectType();
}

The kernel object type we’ll implement is called “DataStack”, and it’s supposed to provide a stack-like functionality. You may be wondering what’s so special about that? Every language/library under the sun has some stack data structure. Implemented as kernel objects, these data stacks offer some benefits that are normally unavailable:

• Thread Synchronization, so the data stack can be accessed freely by clients. Data race prevention is the burden of the implementation.
• These data stacks can be shared between processes, something not offered by any stack implementation you find in languages/libraries.

You could argue that it would be possible to implement such data stacks on top of Section (File Mapping) objects, which allow sharing of memory, with some API that does all the heavy lifting. This is true in essence, but not ideal. The section could be misused by accessing it directly without regard to the data stack implemented. And besides, it’s not the point. You could come up with another kind of object that would not lend itself to easy implementation in other ways.

Back to DriverEntry: The only call is to a helper function, whose purpose is to create the object type. Creating an object type is done with ObCreateObjectType, declared like so:

NTSTATUS NTAPI ObCreateObjectType(
	_In_ PUNICODE_STRING TypeName,
	_In_ POBJECT_TYPE_INITIALIZER ObjectTypeInitializer,
	_In_opt_ PSECURITY_DESCRIPTOR sd,
	_Deref_out_ POBJECT_TYPE* ObjectType);

TypeName is the type name for the new type, which must be unique. ObjectTypeInitializer provides various properties for the type object, some of which we’ll examine momentarily. sd is an optional security descriptor to assign to the new type object, where NULL means the kernel will provide an appropriate default. ObjectType is the returned object type pointer, if successful. The POBJECT_TYPE is not defined in all its glory in the WDK headers, so it’s treated as a PVOID, but that’s fine. We won’t need to look inside.

The simplest way to create the type object would be like so:

UNICODE_STRING typeName = RTL_CONSTANT_STRING(L"DataStack");
OBJECT_TYPE_INITIALIZER init{ sizeof(init) };
auto status = ObCreateObjectType(&typeName, &init, nullptr,
    &g_DataStackType);

The OBJECT_TYPE_INITIALIZER has a Length first member, which must be initialized to the size of the structure, as is common in many Windows APIs. The rest of the structure is zeroed out, which is good enough for our first attempt. The returned pointer lands in a global variable (g_DataStackType), that we can use if needed.

The Unload routine may try to remove the new object type like so:

void OnUnload(PDRIVER_OBJECT DriverObject) {
    UNREFERENCED_PARAMETER(DriverObject);
    ObDereferenceObject(g_DataStackType);
}

Let’s see the effect this could has when the driver is deployed and loaded. First, Here is Object Explorer on a Windows 11 VM where the driver is deployed:

Notice the new object type, with an index of 76 and the name “DataStack”. There are zero objects and zero handles for this kind of object right now (no big surprise there). Let’s see what the kernel debugger has to say:

lkd> !object \objecttypes\datastack
Object: ffff8385bd5ff570 Type: (ffff8385b26a8d00) Type
ObjectHeader: ffff8385bd5ff540 (new version)
HandleCount: 0 PointerCount: 2
Directory Object: ffffc9067828b730 Name: DataStack

Clearly there is such a object type, and it has 2 references, one of which we are holding in our kernel variable. We can examine the object in its more specific role as a object type:

lkd> dt nt!_OBJECT_TYPE ffff8385bd5ff570
   +0x000 TypeList         : _LIST_ENTRY [ 0xffff8385`bd5ff570 - 0xffff8385`bd5ff570 ]
   +0x010 Name             : _UNICODE_STRING "DataStack"
   +0x020 DefaultObject    : (null) 
   +0x028 Index            : 0x4c 'L'
   +0x02c TotalNumberOfObjects : 0
   +0x030 TotalNumberOfHandles : 0
   +0x034 HighWaterNumberOfObjects : 0
   +0x038 HighWaterNumberOfHandles : 0
   +0x040 TypeInfo         : _OBJECT_TYPE_INITIALIZER
   +0x0b8 TypeLock         : _EX_PUSH_LOCK
   +0x0c0 Key              : 0x61746144
   +0x0c8 CallbackList     : _LIST_ENTRY [ 0xffff8385`bd5ff638 - 0xffff8385`bd5ff638 ]
   +0x0d8 SeMandatoryLabelMask : 0
   +0x0dc SeTrustConstraintMask : 0

Note the Name and the fact that there are zero objects and handles.

Let’s see what happens when we unload the driver. Since we’re dereferencing our reference (g_DataStackType), the object type is still alive, as the kernel holds to another reference (this was generated in a different run of the system, so the addresses are not the same):

lkd> !object \objecttypes\datastack
Object: ffffc081d94fd330 Type: (ffffc081cd6af5e0) Type
ObjectHeader: ffffc081d94fd300 (new version)
HandleCount: 0 PointerCount: 1
Directory Object: ffff968b2c233a10 Name: DataStack

Why do we have another reference? The type object is permanent, as we can see if we examine its object header:

lkd> dt nt!_OBJECT_HEADER ffffc081d94fd300
+0x000 PointerCount : 0n1
+0x008 HandleCount : 0n0
+0x008 NextToFree : (null)
+0x010 Lock : _EX_PUSH_LOCK
+0x018 TypeIndex : 0xc5 ''
+0x019 TraceFlags : 0 ''
+0x019 DbgRefTrace : 0y0
+0x019 DbgTracePermanent : 0y0
+0x01a InfoMask : 0x3 ''
+0x01b Flags : 0x13 ''
+0x01b NewObject : 0y1
+0x01b KernelObject : 0y1
+0x01b KernelOnlyAccess : 0y0
+0x01b ExclusiveObject : 0y0
+0x01b PermanentObject : 0y1
…

We could remove the “permanent” flag from the type object by making it “temporary” (a.k.a. normal) in our Unload routine, like so:

HANDLE hType;
auto status = ObOpenObjectByPointer(g_DataStackType,
    OBJ_KERNEL_HANDLE, nullptr, 0, nullptr, KernelMode, &hType);
if (NT_SUCCESS(status)) {
    status = ZwMakeTemporaryObject(hType);
    ZwClose(hType);
}
ObDereferenceObject(g_DataStackType);

Calling ZwMakeTemporaryObject (a documented API) removes the permanent bit, so that ObDereferenceObject removes the last reference of the DataStack object type. Unfortunately, this works too well – it also causes the system to crash (BSOD), and that’s because the kernel does not expect type objects to be deleted. It makes sense, since objects of that type may still be alive. Even if the kernel could determine that no objects of that type are alive right now, and allow the deletion, what would that mean for future creations? Worse, it’s possible to create objects privately without a header (very common in the kernel), which means the kernel is unaware of these objects to begin with. The bottom line is, type objects cannot be destroyed safely. In our case, it means the driver should remain alive at all times, but regardless, it should not attempt to destroy the type object.

Object Type Customization

The code to create the DataStack object type did not do any customizations. Possible customizations are available via the OBJECT_TYPE_INITIALIZER structure:

typedef struct _OBJECT_TYPE_INITIALIZER {
	USHORT Length;
	union {
		USHORT Flags;
		struct {
			UCHAR CaseInsensitive : 1;
			UCHAR UnnamedObjectsOnly : 1;
			UCHAR UseDefaultObject : 1;
			UCHAR SecurityRequired : 1;
			UCHAR MaintainHandleCount : 1;
			UCHAR MaintainTypeList : 1;
			UCHAR SupportsObjectCallbacks : 1;
			UCHAR CacheAligned : 1;
			UCHAR UseExtendedParameters : 1;
			UCHAR _Reserved : 7;
		};
	};

	ULONG ObjectTypeCode;
	ULONG InvalidAttributes;
	GENERIC_MAPPING GenericMapping;
	ULONG ValidAccessMask;
	ULONG RetainAccess;
	POOL_TYPE PoolType;
	ULONG DefaultPagedPoolCharge;
	ULONG DefaultNonPagedPoolCharge;
	OB_DUMP_METHOD DumpProcedure;
	OB_OPEN_METHOD OpenProcedure;
	OB_CLOSE_METHOD CloseProcedure;
	OB_DELETE_METHOD DeleteProcedure;
	OB_PARSE_METHOD ParseProcedure;
	OB_SECURITY_METHOD SecurityProcedure;
	OB_QUERYNAME_METHOD QueryNameProcedure;
	OB_OKAYTOCLOSE_METHOD OkayToCloseProcedure;
	ULONG WaitObjectFlagMask;
	USHORT WaitObjectFlagOffset;
	USHORT WaitObjectPointerOffset;
} OBJECT_TYPE_INITIALIZER, * POBJECT_TYPE_INITIALIZER;

This is quite a structure. The various *Procedure members are callbacks. You can find their prototypes in the ReactOS source code, but for now you can just replace all of them with an opaque PVOID to make it easier to deal with the structure. At this point, we’ll customize our object type’s creation like so:

UNICODE_STRING typeName = RTL_CONSTANT_STRING(L"DataStack");
OBJECT_TYPE_INITIALIZER init{ sizeof(init) };
init.PoolType = NonPagedPoolNx;
init.DefaultNonPagedPoolCharge = sizeof(DataStack);
init.ValidAccessMask = DATA_STACK_ALL_ACCESS;
GENERIC_MAPPING mapping{
    STANDARD_RIGHTS_READ | DATA_STACK_QUERY,
    STANDARD_RIGHTS_WRITE | DATA_STACK_PUSH | DATA_STACK_POP | 
        DATA_STACK_CLEAR,
    STANDARD_RIGHTS_EXECUTE | SYNCHRONIZE, DATA_STACK_ALL_ACCESS
};
init.GenericMapping = mapping;

auto status = ObCreateObjectType(&typeName, &init, nullptr,
    &g_DataStackType);

The PoolType member indicates from which pool objects of this type should be allocated. I’ve selected the Non Paged pool with no execute allowed. DataStack is the structure we’ll use for the implementation of the type. We’ll see what that looks like in the next post. For now, we indicate to the kernel that the base memory consumption of objects of DataStack type is the size of that structure.

Next, we see some constants being used that I have defined in a file that is going to be shared between the driver and user mode clients that has some definitions, similar to other object kinds:

#define DATA_STACK_QUERY	0x1
#define DATA_STACK_PUSH		0x2
#define DATA_STACK_POP		0x4
#define DATA_STACK_CLEAR	0x8

#define DATA_STACK_ALL_ACCESS (STANDARD_RIGHTS_REQUIRED | SYNCHRONIZE | DATA_STACK_QUERY | DATA_STACK_PUSH | DATA_STACK_POP | DATA_STACK_CLEAR)

These #defines provide specific access mask bits for objects of the DataStack type. We’ll use these in the implementation so that only powerful-enough handles would allow the relevant access. In the object type creation we use these in the ValidAccessMask member to indicate what is valid to request by clients, and also to provide generic mapping. Generic mapping is a standard feature used by Windows to map generic rights (GENERIC_READ, GENERIC_WRITE, GENERIC_EXECUTE, and GENERIC_ALL) to specific rights appropriate for the object type. You can see these mappings in Object Explorer for all object types. For example, if a client asks for GENERIC_READ when opening a DataStack object, the access requested is going to be DATA_STACK_QUERY.

What’s Next?

We have an object type, that’s great! But we can’t create objects of this type, nor use it in any way. We’re missing the actual implementation. From a user-mode perspective, we’d like to expose an API, not much different in spirit than other object types:

NTSTATUS NTAPI NtCreateDataStack(
	_Out_ PHANDLE DataStackHandle, 
	_In_opt_ POBJECT_ATTRIBUTES DataStackAttributes, 
	_In_ ULONG MaxItemSize, 
	_In_ ULONG MaxItemCount, 
	ULONG_PTR MaxSize);
NTSTATUS NTAPI NtOpenDataStack(
	_Out_ PHANDLE DataStackHandle, 
	_In_ ACCESS_MASK DesiredAccess, 
	_In_opt_ POBJECT_ATTRIBUTES DataStackAttributes);
NTSTATUS NTAPI NtQueryDataStack(
	_In_ HANDLE DataStackHandle, 
	_In_ DataStackInformationClass InformationClass, 
	_Out_ PVOID Buffer, 
	_In_ ULONG BufferSize, 
	_Out_opt_ PULONG ReturnLength);
NTSTATUS NTAPI NtPushDataStack(
	_In_ HANDLE DataStackHandle, 
	_In_ PVOID Item, 
	_In_ ULONG ItemSize);
NTSTATUS NTAPI NtPopDataStack(
	_In_ HANDLE DataStackHandle, 
	_Out_ PVOID Buffer, 
	_Inout_ PULONG BufferSize);
NTSTATUS NTAPI NtClearDataStack(_In_ HANDLE DataStackHandle);

if you’re familiar with the Windows Native API, the “spirit” of these DataStack API is the same. The big questions are, how do we implement these APIs – in user mode and kernel mode? We’ll look into it in the next post.

I have not provided a prebuilt project for this part. Feel free to type things yourself, as there is not too much code at this point. In the next post, I’ll provide a Github repo that has all the code. See you then!

Rust has been reported as “most loved language” by some developer surveys, promising safety not available with other mainstream languages. What does that safety really mean? Let’s look deeper.

The idea of a “safe” language has been a hot topic for some time now, where Rust apparently reigns supreme. But what is that “safety”? The primary safety referred to is memory safety.

Languages such as C and C++ allow the programmer to access any memory location without any checks. For example, you can access an array element beyond the size of the array and the compiler wouldn’t know – it will happily compile the code, only to crash at runtime, if you’re lucky. If you are unlucky, some random memory would be accessed, which may cause incorrect behavior, which is worse. Rust guarantees that such random memory access will not compile, or if it does compile, it will panic (crash) at runtime rather than accessing memory outside the bounds of the array.

Is Rust the only language that provides this kind of memory safety? Not really. Other languages provide similar safety guarantees, such as Python, C#, and Java. One extra nicety you get with Rust is the unavailability of a null reference/pointer – there is no null in safe Rust.

“Safe” Rust? Rust has two personalities – one being Safe Rust – the “normal” Rust developers typically use, but there is a second, mostly hidden part of Rust – unsafe Rust. Unsafe Rust has all the capabilities (and dangers) of unsafe languages, just like C and C++. With unsafe Rust, you can (attempt) to access any memory location, and do things the Rust compiler will not normally allow. Ideally, Rust developers won’t use unsafe Rust, but nothing stops them from doing so. A canonical example is mutating a global variable – it must be done in an unsafe context:

static mut myglobal:i32 = 0;

fn do_work() {
    //myglobal += 1;  // does not compile

    unsafe {
        myglobal += 1;
    }
}

Unsafe is the power behind safe Rust. At the end of the day, hardware (like CPUs) is “unsafe” – it will do anything it’s told, attempt to access any memory location, etc. This is why safe Rust is provided – it wraps (in many cases) unsafe code, which is necessary to get things done. Safe Rust must trust Unsafe Rust implicitly – the compiler has no way of verifying the correctness of unsafe code. As long as developers use safe Rust, they get a guarantee from the Rust compiler and well-debugged unsafe Rust libraries, that their code would not misbehave as it relates to memory safety.

The Second Rust Safety

You may be wondering why, in the above example, is the access to a mutable global variable requires an unsafe context? This brings in the second part of Rust safety, which really sets it apart from other memory-safe languages – safety in a concurrent environment.

Typical code is multithreaded these days, at least for the purpose of utilizing today’s multicore systems. Writing correct multithreaded code, however, is far from trivial. Latent bugs may lurk in code for years, only manifesting at what appears to be random times, making these bugs difficult to fix as it’s difficult to reproduce. Bugs that cannot be reliably reproduced cannot be truly fixed.

For example, languages such as C# and Java will allow you to access shared data concurrently from multiple threads, where at least one of the threads is writing to the shared data. This is the definition of a data race, something to avoid at all costs. These languages provide facilities to deal with data races, but there is no enforcement in using them correctly (or at all). For example, C# has the lock keyword that can be used to execute a block of code by one thread at a time, avoiding a data race if that block accesses the shared data. What happens if another block of code accesses the same shared data without using the correct lock statement (using the same internal lock) or no lock at all? You get a data race again, and because of timing, it would occur at unpredictable times, making it difficult to identify, let alone fix.

This is where Rust offers a different approach. Its famous ownership model, which caters for memory safety is also the one providing concurrency safety. If you try to access shared data from multiple threads, the compiler will refuse to compile the code. This is exactly what happens in the global variable example. The compiler cannot prove that every piece of code accesses the global variable in a multithreaded-safe way. To use safe code, the data to be protected can be wrapped in a Mutex:

static myglobal2: Mutex<i32> = Mutex::new(0);

fn do_work() {
    let mut data = myglobal2.lock().unwrap();
    *data += 1;
}

Notice the Mutex is not mutable. With the above code, there is no way to access the shared data without going through the mutex. This is really powerful, and is not offered by other memory-safe languages. The code may look a bit weird, “dereferencing” the integer, but this is necessary in this case as the returned object from lock().unwrap() is not an integer reference, but a wrapper type where the deref operator is overloaded. With an i32, a Mutex is not really needed, as atomic<i32> would be more efficient and easier to use, but with more interesting objects, this is not too bad, such as the following thread-safe access to a Vec<i32>:

static mydata: Mutex<Vec<i32>> = Mutex::new(Vec::new());

fn do_work() {
    let mut v = mydata.lock().unwrap();
    v.push(42);
}

Is it all Rainbows and Unicorns?

Not quite. Rust is known for its steep learning curve. As any beginning Rust developer knows, it sometimes feels like a fight between the developer and the Rust compiler. With time, developers learn to appreciate the compiler, as it makes sure nasty stuff does not happen at runtime. But because of this strong guarantee, seemingly simple things turn out to be not so simple. For example, consider this simple function that returns the longer of two strings (technically string slices):

fn longest(a: &str, b: &str) -> &str {
    if a.len() > b.len() {
        a
    }
    else {
        b
    }
}

This function fails compilation with an error:

error[E0106]: missing lifetime specifier
  --> src/main.rs:26:33
   |
26 | fn longest(a: &str, b: &str) -> &str {
   |               ----     ----     ^ expected named lifetime parameter
   |
   = help: this function's return type contains a borrowed value, but the signature does not say whether it is borrowed from `a` or `b`
help: consider introducing a named lifetime parameter
   |
26 | fn longest<'a>(a: &'a str, b: &'a str) -> &'a str {
   |           ++++     ++          ++          ++

One of the powers of the Rust compiler is its explanations and suggestions, that help in resolving compiler errors in many cases, like this one. Lifetime specifiers introduce complexities when working with references (to whatever), because the compiler must be sure that the references don’t potentially outlive the objects they are referencing. If the compiler can’t prove it, it will fail compilation. The problem with the above code is that the compiler can’t tell which reference would be returned from calls to the longest function. Every call could return either a reference to a or to b. But why is that a problem? Here is an example:

let s1 = String::from("Hello");
let s3;

{
    let s2 = String::from("Goodbye");
    s3 = longest(&s1, &s2);
}

println!("Longest: {}", s3);

Here, s3 should reference either s1 or s2, but s2 only lives in the inner scope, which means that s3 potentially could reference a dead string; the Rust compiler cannot take that chance. Lifetime annotations tell the compiler what to expect:

fn longest<'a>(a: &'a str, b: &'a str) -> &'a str {
    if a.len() > b.len() {
        a
    }
    else {
        b
    }
}

The syntax looks weird before you get used to it (I won’t get into the details here), but the annotations specify that a, b and the returned reference all have the same lifetime. This makes the compiler happy in that it knows what is allowed to be passed in when calling longest. In particular, it will fail compilation of the call to longest above with the following error:

error[E0597]: `s2` does not live long enough
  --> src/main.rs:18:27
   |
17 |         let s2 = String::from("Goodbye");
   |             -- binding `s2` declared here
18 |         s3 = longest(&s1, &s2);
   |                           ^^^ borrowed value does not live long enough
19 |     }
   |     - `s2` dropped here while still borrowed
20 |
21 |     println!("Longest: {}", s3);
   |                             -- borrow later used here

This means that if s1 and s2 don’t have the same lifetimes, the code fails to compile.

Fortunately, lifetime annotations are not needed in the majority of cases (there are a few lifetime elision rules that help), and in many cases references are not returned or stored. Still, this is another new concept Rust developers need to learn which has no equivalent in other languages.

So, is Rust the Ultimate Language?

It’s not enough to have an elegant, performant, or whatever programming language to succeed. There are other factors, which are even more important. Just look at Python – it has terrible performance, but still extremely popular. Here are some factors I consider necessary for a language to thrive:

• The language itself
• Libraries availability
• Ecosystem and community
• Tooling

Libraries are critical – reinventing the wheel is not an option if software is to be delivered reliably and quickly enough. For Rust, the library ecosystem is large and growing rapidly. Just look at https://crates.io.

Tooling is another major one. Rust has its own package manager, cargo, which is extremely flexible and powerful, adding a lot to the ease of using external packages, and many other configurations. The other part of “tooling” is an IDE. This is where I thing Rust is still lacking. The most popular IDE used for Rust development is Visual Studio Code, with an extension called rust-analyzer. The extension is powerful and always improving, but VS Code itself is very generic, and is not geared towards Rust in particular.

Personally, myself being a user of Visual Studio mostly, I feel the difference. When working with C# or C++ in Visual Studio, the integration is tight and powerful. The debugger is more flexible and powerful than what VS Code can offer with its generality. Visual Studio “supports” Rust as well with the same rust-analyzer extension that is used in VS Code, but it’s not the same experience, and it feels like a “foreign” extension. Unfortunately, Microsoft doesn’t seem to have a plan for providing first-class support for Rust in Visual Studio even though they claim to use Rust a lot, even within the Windows OS.

One new tool worth mentioning is Jetbrain’s RustRover, which is dedicated to Rust. Personally, I feel some of the Jetbrains tools to be more complex to use than I would have liked, but it’s probably a matter of getting used to these tools.

Conclusion

Rust is going places, that’s for sure. It has lots of power and flexibility, and can be used for anything, from low-level stuff like UEFI application, to web applications, other server application, and the cloud. There are some parts of Rust syntax and philosophy that I don’t completely agree with, but any language has compromises.

There is no “perfect programming language”, of course, although I dreamed of creating one in my youth

Asynchronous Procedure Calls (APCs) in Windows are objects that can be attached to threads. Every thread has its own APC queue, where an APC stores a function and arguments to call. The idea is that we’d like the a function to be executed by a specific thread, rather than some arbitrary thread. This is because the process this thread is part of is important for some reason, so the APC (when executed) has full access to that process resources.

Technically, there are user-mode, kernel-mode, and Special kernel-mode APCs, In this post I’ll discuss user mode APCs, those directly supported by the Windows API. (There is also Special user-mode APCs, but these are not generally usable). Here is a conceptual representation of a thread and its APC queues:

A Thread and its APC queues

When a user mode APC is queued to a thread (more on that later), the APC just sits there in the queue, doing nothing. To actually run the APCs currently attached to a thread, that thread must go into an alertable wait (also called alertable state). When in that state, any and all APCs in the thread’s queue execute in sequence. But how does a thread go into an alertable wait?

There are a few functions that can do that. The simplest is SleepEx, the extended Sleep function:

DWORD SleepEx(DWORD msec, BOOL alertable);

If alertable is FALSE, the function is identical to Sleep. Otherwise, the thread sleeps for the designated time (which can be zero), and if any APCs exits in its queue (or appear while it’s sleeping), will execute now, and the sleep is over, in which case the return value from SleepEx is WAIT_IO_COMPLETION rather than zero. A typical call might be SleepEx(0, TRUE) to force all queued APCs to run (if there are any). You can think of this call as a “garbage collection” of APCs. If a thread does not ever go into an alertable wait, any attached APCs will never execute.

Other ways of entering an alertable wait involve using the extended versions of the various wait functions, such as WaitForSingleObjectEx, WaitForMultipleObjectsEx, where an additional Boolean argument is accepted just like SleepEx. MsgWaitForMultipleObjectsEx can do that as well, although the alertable state is specified with a flag (MWMO_ALERTABLE) rather than a Boolean.

Now that know how user mode APCs work, we can try to put them to good use.

Asynchronous I/O Completion

The “classic” use of user mode APCs is to get notified of asynchronous I/O operations. Windows has several mechanisms for this purpose, one of which involves APCs. Specifically, the ReadFileEx and WriteFileEx APIs receive an extra parameter (compared to their non-Ex variants) that is a callback to be invoked when the asynchronous I/O operation completes. The catch is, that the callback is wrapped in an APC queued to the requesting thread, which means it can only execute if/when that thread enters an alertable wait. Here is some conceptual code:

HANDLE hFile = ::CreateFile(..., FILE_FLAG_OVERLAPPED, nullptr);
OVERLAPPED ov{};
ov.Offset = ...;
::ReadFileEx(hFile, buffer, size, &ov, OnIoCompleted);
// other work...
// from time to time, execute APCs:
::SleepEx(0, TRUE);

The Completion routine has the following prototype:

void OnIoCompletion(DWORD dwErrorCode,
    DWORD dwNumberOfBytesTransfered,
    LPOVERLAPPED lpOverlapped);

In practice, this mechanism of being notified of asynchronous /O completion is not very popular, because it’s usually inconvenient to use the same thread for completion. In fact, the thread might exit before the I/O completed. Still, it’s an option that utilizes APCs.

Injecting a DLL into a Process

It’s sometimes useful to “force” somehow another process to load a DLL you provide. The classic way of achieving that is by using the CreateRemoteThread API, where the “thread function” is set to the address of the LoadLibrary APS, because a thread’s function and LoadLibrary have the same prototype from a binary perspective – both accept a pointer. LoadLibrary is passed the path of the DLL to load. You can find a video I made to show this classic technique here: https://youtu.be/0jX9UoXYLa4. A full source code example is here: https://github.com/zodiacon/youtubecode/tree/main/Injection.

The problem with this approach is that it’s pretty visible – anti-malware kernel drivers get notified when a thread is created, and if created by a thread in a different process, that may be suspicious from the driver’s perspective. By the way, the “legitimate” usage of CreateRemoteThread is for a debugger to break in forcefully to a target process in an initial attach, by forcing a new thread in the process to call DbgBreakPoint.

Using an APC, we may be able to “persuade” an existing thread to load our DLL. This is much stealthier, since it’s an existing thread loading a DLL – a very common occurrence. To achieve that, we can use the generic QueueUserAPC API:

DWORD QueueUserAPC(
  PAPCFUNC  pfnAPC,
  HANDLE    hThread,
  ULONG_PTR dwData
);

Fortunately, an APC function has the same binary layout as a thread function – again receiving some kind of pointer. The main issue with this technique is that the target thread may not get into an alertable wait ever. To increase the probability of success, we can queue the APC to all threads in the target process – we just need one to enter an alertable wait. This works well for processes like Explorer, which have so many threads it practically always works. Here is a link to a video I made to show this technique: https://youtu.be/RBCR9Gvp5BM

A Natural Queue

Lastly, since APCs are stored in a queue, we can create a “work queue” very naturally just by utilizing APCs. If you need a queue of functions to be invoked sequentially, you could manage them yourself with the help of (say) std::queue<> in C++. But that queue is not thread-safe, so you would have to properly protect it. If you’re using .NET, you may use ConcurrentQueue<> to help with synchronization, but you still would need to build some kind of loop to pop items, invoke them, etc. With APCs, this all becomes natural and pretty easy:

void WorkQueue(HANDLE hQuitEvent) {
    while(::WaitForSingleObjectEx(hQuitEvent, INFINITE, TRUE) != WAIT_OBJECT_0)
        ;
}

Simplicity itself. An event object can be used to quit this infinite loop (SetEvent called from somewhere). The thread waits for APCs to appear in its queue, and runs them when they do, returning to waiting. Clients of this queue call QueueUserAPC to enqueue work items (callbacks) to that thread. That’s it – simple and elegant.

Summary

APCs provide a way to allow callbacks to be invoked sequentially by a target thread. Maybe you can find other creative use of APCs.

Millions of machines around the world crashed a few days ago, showing the dreaded “Blue Screen of Death” (BSOD), affecting banks, airports, hospitals, and many other businesses, all using the Windows OS and CrowdStrike’s Endpoint Detection and Response (EDR) software. What was going on? How can such a calamity happen in one single swoop?

First, foul play was suspected – a cyber security attack perhaps. But it turned out to be a bad update of CrowdStrike’s “Falcon” software agent that caused all this mess. What is a BSOD anyway?

Code running on Windows can run in two primary modes – user mode and kernel mode. User mode is restricted in its access, which cannot harm the OS. This is the mode applications run with – such as Explorer, Word, Notepad, and any other application. Kernel mode, however, has (almost) unlimited power. But, as the American hero movies like to say, “With great power comes great responsibility” – and this is where things can go wrong.

Kernel code is trusted, by definition, because it can do anything. The Windows kernel is the main part of kernel space, but third-party components may be installed in the kernel, called device drivers. Classic device drivers are required to manage hardware devices – connecting them to the rest of the OS. The fact that you can move your cursor with the mouse, see something on the screen, hear game sounds, etc., means there are device drivers dealing with the hardware correctly, some of which are not written by Microsoft.

If a driver misbehaves, such as causing an exception to occur, the system crashes with the infamous BSOD. This is not some kind of punishment, but a protection mechanism. If a driver misbehaves (or any other kernel component), it is best to stop now, rather than letting the code continue execution which might cause more damage, like data corruption, and even prevent Windows from starting successfully.

Third party drivers are the only entities that Microsoft has no full control over. Drivers must be properly signed, but that only guarantees that they have not been tampered with, as it does not necessarily guarantee quality.

Most Windows systems have some Anti-virus or EDR software protecting them. By default, you get Windows Defender, but there are more powerful EDRs out there, CrowdStrike’s Falcon being one of the leaders in this space.

The “incident” involved a bad update that caused a BSOD when Windows restarted. Restarting again did not help, as a BSOD was showing immediately because of a bug in the driver when it’s loaded and initialized. The only recourse was to boot the system in Safe Mode, where only a minimal set of drivers is loaded, disable the problematic driver, and reboot again. Unfortunately, this has to be done manually on millions of machines.

The Windows kernel treats all kernel components in the same way, regardless of whether that component is from Microsoft or not. Any driver, no matter how insignificant, that causes an unhandled exception, will crash the system with a BSOD. Maybe it would be wise to somehow designate drivers as “important” or “not that important” so they may be treated differently in case of failure. But that is not how Windows works, and in any case, an anti-malware driver is likely to be tagged as “important”.

This entire incident certainly raises questions and concerns – a single point of failure has shown itself in full force. Perhaps a different approach to handling kernel components should be considered.

Personally, I was never comfortable with Windows uniform treatment of kernel components and drivers, but in practice it’s unclear what would be a good way to deal with such exceptions. One alternative is to write drivers in user mode, and many are written in this way, especially to handle relatively slow devices, like USB devices. This works well, but is not good enough for an EDR’s needs.

Perhaps specifically for anti-malware drivers, any unhandled exception should be treated differently by disabling the driver in some way. Not easy to do – the driver has typically registered with the kernel (e.g. PsSetCreateProcessNotifyRoutineEx and others), its callbacks may be running right now – how would the kernel “disable” it safely. I don’t think there is a safe way to do that without significant changes in driver protocols. The best one case do (in theory) is issue a BSOD, but disable the offending driver automatically, so that the next restart will work, and a warning can be issued to the user.

This is not ideal, but is certainly better than the alternative experienced in the recent crash. Why is it not ideal? First, the system will be unprotected, unless some default (like Defender) would automatically take over. Second, it’s difficult to determine with 100% certainty that the driver causing the crash is the ultimate culprit. For example, another (buggy) driver may write to anywhere in kernel space, including memory that may belong to our anti-malware driver, and that write operation does not cause an immediate crash since the memory is valid. Later, our driver will stumble upon the bad data and crash. Even so, some kind of mechanism to prevent the widespread crash must be set in place.

The Application Verifier tool that is part of the Windows SDK provide a way to analyze processes for various types of misbehavior. The GUI provided looks like the following:

Application Verifier application window

To add an application, you can browse your file system and select an executable. The Application Verifier settings are based around the executable name only – not a full path. This is because verifier settings are stored in a subkey under Image File Execution Options with the name of the executable. For the notepad example above, you’ll find the following in the Registry:

Key for notepad.exe under the IFEO subkey

This IFEO subkey is used for NT Global Flags settings, one of which is using the Application Verifier. The GlobalFlag value is shown to be 0x100, which is the bit used for the verifier. Another way to set it without any extra information is using the GFlags tool, part of the Debugging Tools for Windows package:

GFlags tool

The Application Verifier lists a bunch of DLLs under the VerifierDLLs value. Each one must be located in the system directory (e.g., c:\Windows\System32). Full paths are not supported; this is intentional, because the list of DLLs are going to be loaded to any process running the specified executable, and it would be risky to load DLLs from arbitrary locations in the file system. The system directory, as well as the IFEO key are normally write-accessible by administrators only.

The list of verifier DLLs is selected based on the set of tests selected by the user on the right hand side of the GUI. You’ll find subkeys that are used by the system-provided verifier DLLs with more settings related to the tests selected.

The nice thing about any verifier DLL specified, is that these DLLs are loaded early in the process lifetime, by verifier.dll (in itself loaded by NTDLL.dll), before any other DLLs are loaded into the process. Even attaching a debugger to the process while launching it would “miss” the loading of these DLLs.

This behavior makes this technique attractive for injecting a DLL into arbitrary processes. It’s even possible to enable Application Verifier globally and even dynamically (without the need to restart the system), so that these DLLs are injected into all processes (except protected processes).

Writing a Verifier DLL

Application Verifier tests descriptions is not the focus of this post. Rather, we’ll look into what it takes to create such a DLL that can be injected early and automatically into processes of our choice. As we’ll see, it’s not just about mere injection. The verifier infrastructure (part of verifier.dll) provides convenient facilities to hook functions.

If we create a standard DLL, set up the verifier entries while adding our DLL to the list of verifier DLLs (possibly removing the “standard” ones), and try to run our target executable (say, notepad), we get the following nasty message box:

The process shuts down, which means that if a verifier DLL fails to be properly processed, the process terminates rather than “skipping” the DLL.

Launching notepad with WinDbg spits the following output:

ModLoad: 00007ff7`6dfa0000 00007ff7`6dfd8000   notepad.exe
ModLoad: 00007ffd`978f0000 00007ffd`97ae8000   ntdll.dll
ModLoad: 00007ffd`1f650000 00007ffd`1f6c4000   C:\Windows\System32\verifier.dll
Page heap: pid 0x10CEC: page heap enabled with flags 0x3.
AVRF: notepad.exe: pid 0x10CEC: flags 0x81643027: application verifier enabled
ModLoad: 00007ffc`cabd0000 00007ffc`cad6f000   C:\Windows\SYSTEM32\MyVerify.dll
ModLoad: 00007ffd`97650000 00007ffd`9770d000   C:\Windows\System32\KERNEL32.dll
ModLoad: 00007ffd`951b0000 00007ffd`954a6000   C:\Windows\System32\KERNELBASE.dll
AVRF: provider MyVerify.dll did not initialize correctly

Clearly the DLL did not initialize correctly, which is what the NTSTATUS 0xc0000142 was trying to tell us in the message box.

DLLs are initialized with the DllMain function that typically looks like this:

BOOL WINAPI DllMain(HMODULE hModule, DWORD reason, PVOID lpReserved) {
	switch (reason) {
		case DLL_PROCESS_ATTACH:
		case DLL_THREAD_ATTACH:
		case DLL_THREAD_DETACH:
		case DLL_PROCESS_DETACH:
			break;
	}
	return TRUE;
}

The classic four values shown are used by the DLL to run code when it’s loaded into a process (DLL_PROCESS_ATTACH), unloaded from a process (DLL_PROCESS_DETACH), a thread is created in the process (DLL_THREAD_ATTACH), and thread is exiting in the process (DLL_THREAD_DETACH). It turns out that there is a fifth value, which must be used with verifiier DLLs:

#define DLL_PROCESS_VERIFIER 4

Returning TRUE from such a case is not nearly enough. Instead, a structure expected by the caller of DllMain must be initialized and its address provided in lpReserved. The following structures and callback type definitions are needed:

typedef struct _RTL_VERIFIER_THUNK_DESCRIPTOR {
	PCHAR ThunkName;
	PVOID ThunkOldAddress;
	PVOID ThunkNewAddress;
} RTL_VERIFIER_THUNK_DESCRIPTOR, *PRTL_VERIFIER_THUNK_DESCRIPTOR;

typedef struct _RTL_VERIFIER_DLL_DESCRIPTOR {
	PWCHAR DllName;
	ULONG DllFlags;
	PVOID DllAddress;
	PRTL_VERIFIER_THUNK_DESCRIPTOR DllThunks;
} RTL_VERIFIER_DLL_DESCRIPTOR, *PRTL_VERIFIER_DLL_DESCRIPTOR;

typedef void (NTAPI* RTL_VERIFIER_DLL_LOAD_CALLBACK) (
	PWSTR DllName,
	PVOID DllBase,
	SIZE_T DllSize,
	PVOID Reserved);
typedef void (NTAPI* RTL_VERIFIER_DLL_UNLOAD_CALLBACK) (
	PWSTR DllName,
	PVOID DllBase,
	SIZE_T DllSize,
	PVOID Reserved);
typedef void (NTAPI* RTL_VERIFIER_NTDLLHEAPFREE_CALLBACK) (
	PVOID AllocationBase,
	SIZE_T AllocationSize);

typedef struct _RTL_VERIFIER_PROVIDER_DESCRIPTOR {
	ULONG Length;
	PRTL_VERIFIER_DLL_DESCRIPTOR ProviderDlls;
	RTL_VERIFIER_DLL_LOAD_CALLBACK ProviderDllLoadCallback;
	RTL_VERIFIER_DLL_UNLOAD_CALLBACK ProviderDllUnloadCallback;

	PWSTR VerifierImage;
	ULONG VerifierFlags;
	ULONG VerifierDebug;

	PVOID RtlpGetStackTraceAddress;
	PVOID RtlpDebugPageHeapCreate;
	PVOID RtlpDebugPageHeapDestroy;

	RTL_VERIFIER_NTDLLHEAPFREE_CALLBACK ProviderNtdllHeapFreeCallback;
} RTL_VERIFIER_PROVIDER_DESCRIPTOR;

That’s quite a list. The main structure is RTL_VERIFIER_PROVIDER_DESCRIPTOR
that has a pointer to an array of RTL_VERIFIER_DLL_DESCRIPTOR
(the last element in the array must end with all zeros), which in itself points to an array of RTL_VERIFIER_THUNK_DESCRIPTOR
, used for specifying functions to hook. There are a few callbacks as well. At a minimum, we can define this descriptor like so (no hooking, no special code in callbacks):

RTL_VERIFIER_DLL_DESCRIPTOR noHooks{};

RTL_VERIFIER_PROVIDER_DESCRIPTOR desc = {
	sizeof(desc),
	&noHooks,
	[](auto, auto, auto, auto) {},
	[](auto, auto, auto, auto) {},
	nullptr, 0, 0,
	nullptr, nullptr, nullptr,
	[](auto, auto) {},
};

We can define these simply as global variables and return the address of desc in the handling of DLL_PROCESS_VERIFIER:

case DLL_PROCESS_VERIFIER:
	*(PVOID*)lpReserved = &desc;
	break;

With this code in place, we can try launching notepad again (after copying MyVerify.Dll to the System32 directory). Here is the output from WinDbg:

ModLoad: 00007ff7`6dfa0000 00007ff7`6dfd8000   notepad.exe
ModLoad: 00007ffd`978f0000 00007ffd`97ae8000   ntdll.dll
ModLoad: 00007ffd`1f650000 00007ffd`1f6c4000   C:\Windows\System32\verifier.dll
Page heap: pid 0xB30C: page heap enabled with flags 0x3.
AVRF: notepad.exe: pid 0xB30C: flags 0x81643027: application verifier enabled
ModLoad: 00007ffd`25b50000 00007ffd`25cf1000   C:\Windows\SYSTEM32\MyVerify.dll
ModLoad: 00007ffd`97650000 00007ffd`9770d000   C:\Windows\System32\KERNEL32.dll
ModLoad: 00007ffd`951b0000 00007ffd`954a6000   C:\Windows\System32\KERNELBASE.dll
ModLoad: 00007ffd`963e0000 00007ffd`9640b000   C:\Windows\System32\GDI32.dll
ModLoad: 00007ffd`95790000 00007ffd`957b2000   C:\Windows\System32\win32u.dll
ModLoad: 00007ffd`95090000 00007ffd`951a7000   C:\Windows\System32\gdi32full.dll
...

This time it works. MyVerify.dll loads right after verifier.dll (which is the one managing verify DLLs).

Hooking Functions

As mentioned before, we can use the verifier engine’s support for hooking functions in arbitrary DLLs. Let’s give this a try by hooking into a couple of functions, GetMessage and CreateFile. First, we need to set up the structures for the hooks on a per-DLL basis:

RTL_VERIFIER_THUNK_DESCRIPTOR user32Hooks[] = {
	{ (PCHAR)"GetMessageW", nullptr, HookGetMessage },
	{ nullptr, nullptr, nullptr },
};

RTL_VERIFIER_THUNK_DESCRIPTOR kernelbaseHooks[] = {
	{ (PCHAR)"CreateFileW", nullptr, HookCreateFile },
	{ nullptr, nullptr, nullptr },
};

The second NULL in each triplet is where the original address of the hooked function is stored by the verifier engine. Now we fill the structure with the list of DLLs, pointing to the hook arrays:

RTL_VERIFIER_DLL_DESCRIPTOR dlls[] = {
	{ (PWCHAR)L"user32.dll", 0, nullptr, user32Hooks },
	{ (PWCHAR)L"kernelbase.dll", 0, nullptr, kernelbaseHooks },
	{ nullptr, 0, nullptr, nullptr },
};

Finally, we update the main structure with the dlls array:

RTL_VERIFIER_PROVIDER_DESCRIPTOR desc = {
	sizeof(desc),
	dlls,
	[](auto, auto, auto, auto) {},
	[](auto, auto, auto, auto) {},
	nullptr, 0, 0,
	nullptr, nullptr, nullptr,
	[](auto, auto) {},
};

The last thing is to actually implement the hooks:

BOOL WINAPI HookGetMessage(PMSG msg, HWND hWnd, UINT filterMin, UINT filterMax) {
	// get original function
	static const auto orgGetMessage = (decltype(::GetMessageW)*)user32Hooks[0].ThunkOldAddress;
	auto result = orgGetMessage(msg, hWnd, filterMin, filterMax);
	char text[128];
	sprintf_s(text, "Received message 0x%X for hWnd 0x%p\n", msg->message, msg->hwnd);
	OutputDebugStringA(text);
	return result;
}

HANDLE WINAPI HookCreateFile(PCWSTR path, DWORD access, DWORD share, LPSECURITY_ATTRIBUTES sa, DWORD cd, DWORD flags, HANDLE hTemplate) {
	// get original function
	static const auto orgCreateFile = (decltype(::CreateFileW)*)kernelbaseHooks[0].ThunkOldAddress;
	auto hFile = orgCreateFile(path, access, share, sa, cd, flags, hTemplate);
	char text[512];
	if (hFile == INVALID_HANDLE_VALUE)
		sprintf_s(text, "Failed to open file %ws (%u)\n", path, ::GetLastError());
	else
		sprintf_s(text, "Opened file %ws successfuly (0x%p)\n", path, hFile);

	OutputDebugStringA(text);
	return hFile;
}

The hooks just send some output with OutputDebugString. Here is an excerpt output when running notepad under a debugger:

ModLoad: 00007ff7`6dfa0000 00007ff7`6dfd8000   notepad.exe
ModLoad: 00007ffd`978f0000 00007ffd`97ae8000   ntdll.dll
ModLoad: 00007ffd`1f650000 00007ffd`1f6c4000   C:\Windows\System32\verifier.dll
Page heap: pid 0xEF18: page heap enabled with flags 0x3.
AVRF: notepad.exe: pid 0xEF18: flags 0x81643027: application verifier enabled
ModLoad: 00007ffd`25b80000 00007ffd`25d24000   C:\Windows\SYSTEM32\MyVerify.dll
ModLoad: 00007ffd`97650000 00007ffd`9770d000   C:\Windows\System32\KERNEL32.dll
ModLoad: 00007ffd`951b0000 00007ffd`954a6000   C:\Windows\System32\KERNELBASE.dll
ModLoad: 00007ffd`963e0000 00007ffd`9640b000   C:\Windows\System32\GDI32.dll
ModLoad: 00007ffd`95790000 00007ffd`957b2000   C:\Windows\System32\win32u.dll
ModLoad: 00007ffd`95090000 00007ffd`951a7000   C:\Windows\System32\gdi32full.dll
...
ModLoad: 00007ffd`964f0000 00007ffd`965bd000   C:\Windows\System32\OLEAUT32.dll
ModLoad: 00007ffd`96d10000 00007ffd`96d65000   C:\Windows\System32\shlwapi.dll
ModLoad: 00007ffd`965d0000 00007ffd`966e4000   C:\Windows\System32\MSCTF.dll
Opened file C:\Windows\Fonts\staticcache.dat successfuly (0x0000000000000164)
ModLoad: 00007ffd`7eac0000 00007ffd`7eb6c000   C:\Windows\System32\TextShaping.dll
ModLoad: 00007ffc`ed750000 00007ffc`ed82e000   C:\Windows\System32\efswrt.dll
ModLoad: 00007ffd`90880000 00007ffd`909d7000   C:\Windows\SYSTEM32\wintypes.dll
ModLoad: 00007ffd`8bf90000 00007ffd`8bfad000   C:\Windows\System32\MPR.dll
ModLoad: 00007ffd`8cae0000 00007ffd`8cce3000   C:\Windows\System32\twinapi.appcore.dll
Opened file C:\Windows\Registration\R000000000025.clb successfuly (0x00000000000001C4)
ModLoad: 00007ffd`823b0000 00007ffd`82416000   C:\Windows\System32\oleacc.dll
...
Received message 0x31F for hWnd 0x00000000001F1776
Received message 0xC17C for hWnd 0x00000000001F1776
Received message 0xF for hWnd 0x00000000001F1776
Received message 0xF for hWnd 0x00000000003010C0
Received message 0xF for hWnd 0x0000000000182E7A
Received message 0x113 for hWnd 0x00000000003319A8
...
ModLoad: 00007ffd`80e20000 00007ffd`80fd4000   C:\Windows\System32\WindowsCodecs.dll
ModLoad: 00007ffd`94ee0000 00007ffd`94f04000   C:\Windows\System32\profapi.dll
Opened file C:\Users\Pavel\AppData\Local\IconCache.db successfuly (0x0000000000000724)
ModLoad: 00007ffd`3e190000 00007ffd`3e1f6000   C:\Windows\System32\thumbcache.dll
Opened file C:\Users\Pavel\AppData\Local\Microsoft\Windows\Explorer\iconcache_idx.db successfuly (0x0000000000000450)
Opened file C:\Users\Pavel\AppData\Local\Microsoft\Windows\Explorer\iconcache_16.db successfuly (0x000000000000065C)
ModLoad: 00007ffd`90280000 00007ffd`90321000   C:\Windows\SYSTEM32\policymanager.dll

This application verifier technique is an interesting one, and fairly easy to use. The full example can be found at https://github.com/zodiacon/VerifierDLL.

Happy verifying!

What if you needed to get a list of all the open browser tabs in some browser? In the (very) old days you might assume that each tab is its own window, so you could find a main browser window (using FindWindow, for example), and then enumerate child windows with EnumChildWindows to locate the tabs. Unfortunately, this approach is destined to fail. Here is a screenshot of WinSpy looking at a main window of Microsoft Edge:

MS Edge showing only two child windows

The title of the main window hints to the existence of 26 tabs, but there are only two child windows and they are not tabs. The inevitable conclusion is that the tabs are not windows at all. They are being “drawn” with some technology that the Win32 windowing infrastructure doesn’t know about nor cares.

How can we get information about those browsing tabs? Enter UI Automation.

UI Automation has been around for many years, starting with the older technology called “Active Accessibility“. This technology is geared towards accessibility while providing rich information that can be consumed by accessibility clients. Although Active Accessibility is still supported for compatibility reasons, a newer technology called UI Automation supersedes it.

UI Automation provides a tree of UI automation elements representing various aspects of a user interface. Some elements represent “true” Win32 windows (have HWND), some represent internal controls like buttons and edit boxes (created with whatever technology), and some elements are virtual (don’t have any graphical aspects), but instead provide “metadata” related to other items.

The UI Automation client API uses COM, where the root object implements the IUIAutomation interface (it has extended interfaces implemented as well). To get the automation object, the following C++ code can be used (we’ll see a C# example later):

CComPtr<IUIAutomation> spUI;
auto hr = spUI.CoCreateInstance(__uuidof(CUIAutomation));
if (FAILED(hr))
	return Error("Failed to create Automation root", hr);

The client automation interfaces are declared in <UIAutomationClient.h>. The code uses the ATL CComPtr<> smart pointers, but any COM smart or raw pointers will do.

With the UI Automation object pointer in hand, several options are available. One is to enumerate the full or part of the UI element tree. To get started, we can obtain a “walker” object by calling IUIAutomation::get_RawViewWalker. From there, we can start enumerating by calling IUIAutomationTreeWalker interface methods, like GetFirstChildElement and GetNextSiblingElement.

Each element, represented by a IUIAutomationElement interface provides a set of properties, some available directly on the interface (e.g. get_CurrentName, get_CurrentClassName, get_CurrentProcessId), while others hide behind a generic method, get_CurrentPropertyValue, where each property has an integer ID, and the result is a VARIANT, to allow for various types of values.

Using this method, the menu item View Automation Tree in WinSpy shows the full automation tree, and you can drill down to any level, while many of the selected element’s properties are shown on the right:

WinSpy automation tree view

If you dig deep enough, you’ll find that MS Edge tabs have a UI automation class name of “EdgeTab”. This is the key to locating browser tabs. (Other browsers may have a different class name). To find tabs, we can enumerate the full tree manually, but fortunately, there is a better way. IUIAutomationElement has a FindAll method that searches for elements based on a set of conditions. The conditions available are pretty flexible – based on some property or properties of elements, which can be combined with And, Or, etc. to get more complex conditions. In our case, we just need one condition – a class name called “EdgeTab”.

First, we’ll create the root object, and the condition (error handling omitted for brevity):

int main() {
	::CoInitialize(nullptr);

	CComPtr<IUIAutomation> spUI;
	auto hr = spUI.CoCreateInstance(__uuidof(CUIAutomation));

	CComPtr<IUIAutomationCondition> spCond;
	CComVariant edgeTab(L"EdgeTab");
	spUI->CreatePropertyCondition(UIA_ClassNamePropertyId, edgeTab, &spCond);

We have a single condition for the class name property, which has an ID defined in the automation headers. Next, we’ll fire off the search from the root element (desktop):

CComPtr<IUIAutomationElementArray> spTabs;
CComPtr<IUIAutomationElement> spRoot;
spUI->GetRootElement(&spRoot);
hr = spRoot->FindAll(TreeScope_Descendants, spCond, &spTabs);

All that’s left to do is harvest the results:

int count = 0;
spTabs->get_Length(&count);
for (int i = 0; i < count; i++) {
	CComPtr<IUIAutomationElement> spTab;
	spTabs->GetElement(i, &spTab);
	CComBSTR name;
	spTab->get_CurrentName(&name);
	int pid;
	spTab->get_CurrentProcessId(&pid);
	printf("%2d PID %6d: %ws\n", i + 1, pid, name.m_str);
}

Try it!

.NET Code

A convenient Nuget package called Interop.UIAutomationClient.Signed provides wrappers for the automation API for .NET clients. Here is the same search done in C# after adding the Nuget package reference:

static void Main(string[] args) {
    const int ClassPropertyId = 30012;
    var ui = new CUIAutomationClass();
    var cond = ui.CreatePropertyCondition(ClassPropertyId, "EdgeTab");
    var tabs = ui.GetRootElement().FindAll(TreeScope.TreeScope_Descendants, cond);
    for (int i = 0; i < tabs.Length; i++) {
        var tab = tabs.GetElement(i);
        Console.WriteLine($"{i + 1,2} PID {tab.CurrentProcessId,6}: {tab.CurrentName}");
    }
}

More Automation

There is a lot more to UI automation – the word “automation” implies some more control. One capability of the API is providing various notifications when certain aspects of elements change. Examples include the IUIAutomation methods AddAutomationEventHandler, AddFocusChangedEventHandler, AddPropertyChangedEventHandler, and AddStructureChangedEventHandler.

More specific information on elements (and some control) is also available with more specific interfaces related to controls, such as IUIAutomationTextPattern, IUIAutomationTextRange, and others.

Happy automation!

In the previous post, I’ve shown how to write a minimal, but functional, Projected File System provider using C++. I also semi-promised to write a version of that provider in Rust. I thought we should start small, by implementing a command line tool I wrote years ago called objdir. Its purpose is to be a “command line” version of a simplified WinObj from Sysinternals. It should be able to list objects (name and type) within a given object manager namespace directory. Here are a couple of examples:

D:\>objdir \
PendingRenameMutex (Mutant)
ObjectTypes (Directory)
storqosfltport (FilterConnectionPort)
MicrosoftMalwareProtectionRemoteIoPortWD (FilterConnectionPort)
Container_Microsoft.OutlookForWindows_1.2024.214.400_x64__8wekyb3d8bbwe-S-1-5-21-3968166439-3083973779-398838822-1001 (Job)
MicrosoftDataLossPreventionPort (FilterConnectionPort)
SystemRoot (SymbolicLink)
exFAT (Device)
Sessions (Directory)
MicrosoftMalwareProtectionVeryLowIoPortWD (FilterConnectionPort)
ArcName (Directory)
PrjFltPort (FilterConnectionPort)
WcifsPort (FilterConnectionPort)
...

D:\>objdir \kernelobjects
MemoryErrors (SymbolicLink)
LowNonPagedPoolCondition (Event)
Session1 (Session)
SuperfetchScenarioNotify (Event)
SuperfetchParametersChanged (Event)
PhysicalMemoryChange (SymbolicLink)
HighCommitCondition (SymbolicLink)
BcdSyncMutant (Mutant)
HighMemoryCondition (SymbolicLink)
HighNonPagedPoolCondition (Event)
MemoryPartition0 (Partition)
...

Since enumerating object manager directories is required for our ProjFS provider, once we implement objdir in Rust, we’ll have good starting point for implementing the full provider in Rust.

This post assumes you are familiar with the fundamentals of Rust. Even if you’re not, the code should still be fairly understandable, as we’re mostly going to use unsafe rust to do the real work.

Unsafe Rust

One of the main selling points of Rust is its safety – memory and concurrency safety guaranteed at compile time. However, there are cases where access is needed that cannot be checked by the Rust compiler, such as the need to call external C functions, such as OS APIs. Rust allows this by using unsafe blocks or functions. Within unsafe blocks, certain operations are allowed which are normally forbidden; it’s up to the developer to make sure the invariants assumed by Rust are not violated – essentially making sure nothing leaks, or otherwise misused.

The Rust standard library provides some support for calling C functions, mostly in the std::ffi module (FFI=Foreign Function Interface). This is pretty bare bones, providing a C-string class, for example. That’s not rich enough, unfortunately. First, strings in Windows are mostly UTF-16, which is not the same as a classic C string, and not the same as the Rust standard String type. More importantly, any C function that needs to be invoked must be properly exposed as an extern "C" function, using the correct Rust types that provide the same binary representation as the C types.

Doing all this manually is a lot of error-prone, non-trivial, work. It only makes sense for simple and limited sets of functions. In our case, we need to use native APIs, like NtOpenDirectoryObject and NtQueryDirectoryObject. To simplify matters, there are crates available in crates.io (the master Rust crates repository) that already provide such declarations.

Adding Dependencies

Assuming you have Rust installed, open a command window and create a new project named objdir:

cargo new objdir

This will create a subdirectory named objdir, hosting the binary crate created. Now we can open cargo.toml (the manifest) and add dependencies for the following crates:

[dependencies]
ntapi = "0.4"
winapi = { version = "0.3.9", features = [ "impl-default" ] }

winapi provides most of the Windows API declarations, but does not provide native APIs. ntapi provides those additional declarations, and in fact depends on winapi for some fundamental types (which we’ll need). The feature “impl-default” indicates we would like the implementations of the standard Rust Default trait provided – we’ll need that later.

The main Function

The main function is going to accept a command line argument to indicate the directory to enumerate. If no parameters are provided, we’ll assume the root directory is requested. Here is one way to get that directory:

let dir = std::env::args().skip(1).next().unwrap_or("\\".to_owned());

(Note that unfortunately the WordPress system I’m using to write this post has no syntax highlighting for Rust, the code might be uglier than expected; I’ve set it to C++).

The args method returns an iterator. We skip the first item (the executable itself), and grab the next one with next. It returns an Option<String>, so we grab the string if there is one, or use a fixed backslash as the string.

Next, we’ll call a helper function, enum_directory that does the heavy lifting and get back a Result where success is a vector of tuples, each containing the object’s name and type (Vec<(String, String)>). Based on the result, we can display the results or report an error:

let result = enum_directory(&dir);
match result {
    Ok(objects) => {
        for (name, typename) in &objects {
            println!("{name} ({typename})");
        }
        println!("{} objects.", objects.len());
    },
    Err(status) => println!("Error: 0x{status:X}")
};

That is it for the main function.

Enumerating Objects

Since we need to use APIs defined within the winapi and ntapi crates, let’s bring them into scope for easier access at the top of the file:

use winapi::shared::ntdef::*;
use ntapi::ntobapi::*;
use ntapi::ntrtl::*;

I’m using the “glob” operator (*) to make it easy to just use the function names directly without any prefix. Why these specific modules? Based on the APIs and types we’re going to need, these are where these are defined (check the documentation for these crates).

enum_directory is where the real is done. Here its declararion:

fn enum_directory(dir: &str) -> Result<Vec<(String, String)>, NTSTATUS> {

The function accepts a string slice and returns a Result type, where the Ok variant is a vector of tuples consisting of two standard Rust strings.

The following code follows the basic logic of the EnumDirectoryObjects function from the ProjFS example in the previous post, without the capability of search or filter. We’ll add that when we work on the actual ProjFS project in a future post.

The first thing to do is open the given directory object with NtOpenDirectoryObject. For that we need to prepare an OBJECT_ATTRIBUTES and a UNICODE_STRING. Here is what that looks like:

let mut items = vec![];

unsafe {
    let mut udir = UNICODE_STRING::default();
    let wdir = string_to_wstring(&dir);
    RtlInitUnicodeString(&mut udir, wdir.as_ptr());
    let mut dir_attr = OBJECT_ATTRIBUTES::default();
    InitializeObjectAttributes(&mut dir_attr, &mut udir, OBJ_CASE_INSENSITIVE, NULL, NULL);

We start by creating an empty vector to hold the results. We don’t need any type annotation because later in the code the compiler would have enough information to deduce it on its own. We then start an unsafe block because we’re calling C APIs.

Next, we create a default-initialized UNICODE_STRING and use a helper function to convert a Rust string slice to a UTF-16 string, usable by native APIs. We’ll see this string_to_wstring helper function once we’re done with this one. The returned value is in fact a Vec<u16> – an array of UTF-16 characters.

The next step is to call RtlInitUnicodeString, to initialize the UNICODE_STRING based on the UTF-16 string we just received. Methods such as as_ptr are necessary to make the Rust compiler happy. Finally, we create a default OBJECT_ATTRIBUTES and initialize it with the udir (the UTF-16 directory string). All the types and constants used are provided by the crates we’re using.

The next step is to actually open the directory, which could fail because of insufficient access or a directory that does not exist. In that case, we just return an error. Otherwise, we move to the next step:

let mut hdir: HANDLE = NULL;
match NtOpenDirectoryObject(&mut hdir, DIRECTORY_QUERY, &mut dir_attr) {
    0 => {
        // do real work...
    },
    err => Err(err),
}

The NULL here is just a type alias for the Rust provided C void pointer with a value of zero (*mut c_void). We examine the NTSTATUS returned using a match expression: If it’s not zero (STATUS_SUCCESS), it must be an error and we return an Err object with the status. if it’s zero, we’re good to go. Now comes the real work.

We need to allocate a buffer to receive the object information in this directory and be prepared for the case the information is too big for the allocated buffer, so we may need to loop around to get the next “chunk” of data. This is how the NtQueryDirectoryObject is expected to be used. Let’s allocate a buffer using the standard Vec<> type and prepare some locals:

const LEN: u32 = 1 << 16;
let mut first = 1;
let mut buffer: Vec<u8> = Vec::with_capacity(LEN as usize);
let mut index = 0u32;
let mut size: u32 = 0;

We’re allocating 64KB, but could have chosen any number. Now the loop:

loop {
    let start = index;
    if NtQueryDirectoryObject(hdir, buffer.as_mut_ptr().cast(), LEN, 0, first, &mut index, &mut size) < 0 {
        break;
    }
    first = 0;
    let mut obuffer = buffer.as_ptr() as *const OBJECT_DIRECTORY_INFORMATION;
    for _ in 0..index - start {
        let item = *obuffer;
        let name = String::from_utf16_lossy(std::slice::from_raw_parts(item.Name.Buffer, (item.Name.Length / 2) as usize));
        let typename = String::from_utf16_lossy(std::slice::from_raw_parts(item.TypeName.Buffer, (item.TypeName.Length / 2) as usize));
        items.push((name, typename));
        obuffer = obuffer.add(1);
    }
}
Ok(items)

There are quite a few things going on here. if NtQueryDirectoryObject fails, we break out of the loop. This happens when there are is no more information to give. If there is data, buffer is cast to a OBJECT_DIRECTORY_INFORMATION pointer, and we can loop around on the items that were returned. start is used to keep track of the previous number of items delivered. first is 1 (true) the first time through the loop to force the NtQueryDirectoryObject to start from the beginning.

Once we have an item (item), its two members are extracted. item is of type OBJECT_DIRECTORY_INFORMATION and has two members: Name and TypeName (both UNICODE_STRING). Since we want to return standard Rust strings (which, by the way, are UTF-8 encoded), we must convert the UNICODE_STRINGs to Rust strings. String::from_utf16_lossy performs such a conversion, but we must specify the number of characters, because a UNICODE_STRING does not have to be NULL-terminated. The trick here is std::slice::from_raw_parts that can have a length, which is half of the number of bytes (Length member in UNICODE_STRING).

Finally, Vec<>.push is called to add the tuple (name, typename) to the vector. This is what allows the compiler to infer the vector type. Once we exit the loop, the Ok variant of Result<> is returned with the vector.

The last function used is the helper to convert a Rust string slice to a UTF-16 null-terminated string:

fn string_to_wstring(s: &str) -> Vec<u16> {
    let mut wstring: Vec<_> = s.encode_utf16().collect();
    wstring.push(0);    // null terminator
    wstring
}

And that is it. The Rust version of objdir is functional.

The full source is at zodiacon/objdir-rs: Rust version of the objdir tool (github.com)

If you want to know more about Rust, consider signing up for my upcoming Rust masterclass programming.

A little-known feature in modern Windows is the ability to expose hierarchical data using the file system. This is called Windows Projected File System (ProjFS), available since Windows 10 version 1809. There is even a sample that exposes the Registry hierarchy using this technology. Using the file system as a “projection” mechanism provides a couple of advantages over a custom mechanism:

• Any file viewing tool can present the information such as Explorer, or commands in a terminal.
• “Standard” file APIs are used, which are well-known, and available in any programming language or library.

Let’s see how to build a Projected File System provider from scratch. We’ll expose object manager directories as file system directories, and other types of objects as “files”. Normally, we can see the object manager’s namespace with dedicated tools, such as WinObj from Sysinternals, or my own Object Explorer:

WinObj showing parts of the object manager namespace

Here is an example of what we are aiming for (viewed with Explorer):

Explorer showing the root of the object manager namespace

First, support for ProjFS must be enabled to be usable. You can enable it with the Windows Features dialog or PowerShell:

Enable-WindowsOptionalFeature -Online -FeatureName Client-ProjFS -NoRestart

We’ll start by creating a C++ console application named ObjMgrProjFS; I’ve used the Windows Desktop Wizard project with a precompiled header (pch.h):

#pragma once

#include <Windows.h>
#include <projectedfslib.h>

#include <string>
#include <vector>
#include <memory>
#include <map>
#include <ranges>
#include <algorithm>
#include <format>
#include <optional>
#include <functional>

projectedfslib.h is where the ProjFS declarations reside. projectedfslib.lib is the import library to link against. In this post, I’ll focus on the main coding aspects, rather than going through every little piece of code. The full code can be found at https://github.com/zodiacon/objmgrprojfs. It’s of course possible to use other languages to implement a ProjFS provider. I’m going to attempt one in Rust in a future post

The projected file system must be rooted in a folder in the file system. It doesn’t have to be empty, but it makes sense to use such a directory for this purpose only. The main function will take the requested root folder as input and pass it to the ObjectManagerProjection class that is used to manage everything:

int wmain(int argc, const wchar_t* argv[]) {
	if (argc < 2) {
		printf("Usage: ObjMgrProjFS <root_dir>\n");
		return 0;
	}

	ObjectManagerProjection omp;
	if (auto hr = omp.Init(argv[1]); hr != S_OK)
		return Error(hr);

	if (auto hr = omp.Start(); hr != S_OK)
		return Error(hr);

	printf("Virtualizing at %ws. Press ENTER to stop virtualizing...\n", argv[1]);
	char buffer[3];
	gets_s(buffer);

	omp.Term();

	return 0;
}

Let start with the initialization. We want to create the requested directory (if it doesn’t already exist). If it does exist, we’ll use it. In fact, it could exist because of a previous run of the provider, so we can keep track of the instance ID (a GUID) so that the file system itself can use its caching capabilities. We’ll “hide” the GUID in a hidden file within the directory. First, create the directory:

HRESULT ObjectManagerProjection::Init(PCWSTR root) {
	GUID instanceId = GUID_NULL;
	std::wstring instanceFile(root);
	instanceFile += L"\\_obgmgrproj.guid";

	if (!::CreateDirectory(root, nullptr)) {
		//
		// failed, does it exist?
		//
		if (::GetLastError() != ERROR_ALREADY_EXISTS)
			return HRESULT_FROM_WIN32(::GetLastError());

If creation fails not because it exists, bail out with an error. Otherwise, get the instance ID that may be there and use that GUID if present:

	auto hFile = ::CreateFile(instanceFile.c_str(), GENERIC_READ, 
		FILE_SHARE_READ, nullptr, OPEN_EXISTING, 0, nullptr);
	if (hFile != INVALID_HANDLE_VALUE && ::GetFileSize(hFile, nullptr) == sizeof(GUID)) {
		DWORD ret;
		::ReadFile(hFile, &instanceId, sizeof(instanceId), &ret, nullptr);
		::CloseHandle(hFile);
	}
}

If we need to generate a new GUID, we’ll do that with CoCreateGuid and write it to the hidden file:

if (instanceId == GUID_NULL) {
	::CoCreateGuid(&instanceId);
	//
	// write instance ID
	//
	auto hFile = ::CreateFile(instanceFile.c_str(), GENERIC_WRITE, 0, nullptr, CREATE_NEW, FILE_ATTRIBUTE_HIDDEN, nullptr);
	if (hFile != INVALID_HANDLE_VALUE) {
		DWORD ret;
		::WriteFile(hFile, &instanceId, sizeof(instanceId), &ret, nullptr);
		::CloseHandle(hFile);
	}
}

Finally, we must register the root with ProjFS:

auto hr = ::PrjMarkDirectoryAsPlaceholder(root, nullptr, nullptr, &instanceId);
if (FAILED(hr))
	return hr;

m_RootDir = root;
return hr;

Once Init succeeds, we need to start the actual virtualization. To that end, a structure of callbacks must be filled so that ProjFS knows what functions to call to get the information requested by the file system. This is the job of the Start method:

HRESULT ObjectManagerProjection::Start() {
	PRJ_CALLBACKS cb{};
	cb.StartDirectoryEnumerationCallback = StartDirectoryEnumerationCallback;
	cb.EndDirectoryEnumerationCallback = EndDirectoryEnumerationCallback;
	cb.GetDirectoryEnumerationCallback = GetDirectoryEnumerationCallback;
	cb.GetPlaceholderInfoCallback = GetPlaceholderInformationCallback;
	cb.GetFileDataCallback = GetFileDataCallback;

	auto hr = ::PrjStartVirtualizing(m_RootDir.c_str(), &cb, this, nullptr, &m_VirtContext);
	return hr;
}

The callbacks specified above are the absolute minimum required for a valid provider. PrjStartVirtualizing returns a virtualization context that identifies our provider, which we need to use (at least) when stopping virtualization. It’s a blocking call, which is convenient in a console app, but for other cases, it’s best put in a separate thread. The this value passed in is a user-defined context. We’ll use that to delegate these static callback functions to member functions. Here is the code for StartDirectoryEnumerationCallback:

HRESULT ObjectManagerProjection::StartDirectoryEnumerationCallback(const PRJ_CALLBACK_DATA* callbackData, const GUID* enumerationId) {
	return ((ObjectManagerProjection*)callbackData->InstanceContext)->DoStartDirectoryEnumerationCallback(callbackData, enumerationId);
}

The same trick is used for the other callbacks, so that we can implement the functionality within our class. The class ObjectManagerProjection itself holds on to the following data members of interest:

struct GUIDComparer {
	bool operator()(const GUID& lhs, const GUID& rhs) const {
		return memcmp(&lhs, &rhs, sizeof(rhs)) < 0;
	}
};

struct EnumInfo {
	std::vector<ObjectNameAndType> Objects;
	int Index{ -1 };
};
std::wstring m_RootDir;
PRJ_NAMESPACE_VIRTUALIZATION_CONTEXT m_VirtContext;
std::map<GUID, EnumInfo, GUIDComparer> m_Enumerations;

EnumInfo is a structure used to keep an object directory’s contents and the current index requested by the file system. A map is used to keep track of all current enumerations. Remember, it’s the file system – multiple directory listings may be happening at the same time. As it happens, each one is identified by a GUID, which is why it’s used as a key to the map. m_VirtContext is the returned value from PrjStartVirtualizing.

ObjectNameAndType is a little structure that stores the details of an object: its name and type:

struct ObjectNameAndType {
	std::wstring Name;
	std::wstring TypeName;
};

The Callbacks

Obviously, the bulk work for the provider is centered in the callbacks. Let’s start with StartDirectoryEnumerationCallback. Its purpose is to let the provider know that a new directory enumeration of some sort is beginning. The provider can make any necessary preparations. In our case, it’s about adding a new enumeration structure to manage based on the provided enumeration GUID:

HRESULT ObjectManagerProjection::DoStartDirectoryEnumerationCallback(const PRJ_CALLBACK_DATA* callbackData, const GUID* enumerationId) {
	EnumInfo info;
	m_Enumerations.insert({ *enumerationId, std::move(info) });
	return S_OK;
}

We just add a new entry to our map, since we must be able to distinguish between multiple enumerations that may be happening concurrently. The complementary callback ends an enumeration which is where we delete the item from the map:

HRESULT ObjectManagerProjection::DoEndDirectoryEnumerationCallback(const PRJ_CALLBACK_DATA* callbackData, const GUID* enumerationId) {
	m_Enumerations.erase(*enumerationId);
	return S_OK;
}

So far, so good. The real work is centered around the GetDirectoryEnumerationCallback callback where actual enumeration must take place. The callback receives the enumeration ID and a search expression – the client may try to search using functions such as FindFirstFile / FindNextFile or similar APIs. The provided PRJ_CALLBACK_DATA contains the basic details of the request such as the relative directory itself (which could be a subdirectory). First, we reject any unknown enumeration IDs:

HRESULT ObjectManagerProjection::DoGetDirectoryEnumerationCallback(
	const PRJ_CALLBACK_DATA* callbackData, const GUID* enumerationId, 
	PCWSTR searchExpression, PRJ_DIR_ENTRY_BUFFER_HANDLE dirEntryBufferHandle) {

	auto it = m_Enumerations.find(*enumerationId); 
	if(it == m_Enumerations.end())
		return E_INVALIDARG;
    auto& info = it->second;

Next, we need to enumerate the objects in the provided directory, taking into consideration the search expression (that may require returning a subset of the items):

	if (info.Index < 0 || (callbackData->Flags & PRJ_CB_DATA_FLAG_ENUM_RESTART_SCAN)) {
		auto compare = [&](auto name) {
			return ::PrjFileNameMatch(name, searchExpression);
			};
		info.Objects = ObjectManager::EnumDirectoryObjects(callbackData->FilePathName, nullptr, compare);
		std::ranges::sort(info.Objects, [](auto const& item1, auto const& item2) { 
			return ::PrjFileNameCompare(item1.Name.c_str(), item2.Name.c_str()) < 0; 
			});
		info.Index = 0;
	}

There are quite a few things happening here. ObjectManager::EnumDirectoryObjects is a helper function that does the actual enumeration of objects in the object manager’s namespace given the root directory (callbackData->FilePathName), which is always relative to the virtualization root, which is convenient – we don’t need to care where the actual root is. The compare lambda is passed to EnumDirectoryObjects to provide a filter based on the search expression. ProjFS provides the PrjFileNameMatch function we can use to test if a specific name should be returned or not. It has the logic that caters for wildcards like * and ?.

Once the results return in a vector (info.Objects), we must sort it. The file system expects returned files/directories to be sorted in a case insensitive way, but we don’t actually need to know that. PrjFileNameCompare is provided as a function to use for sorting purposes. We call sort on the returned vector passing this function PrjFileNameCompare as the compare function.

The enumeration must happen if the PRJ_CB_DATA_FLAG_ENUM_RESTART_SCAN is specified. I also enumerate if it’s the first call for this enumeration ID.

Now that we have results (or an empty vector), we can proceed by telling ProjFS about the results. If we have no results, just return success (an empty directory):

if (info.Objects.empty())
	return S_OK;

Otherwise, we must call PrjFillDirEntryBuffer for each entry in the results. However, ProjFS provides a limited buffer to accept data, which means we need to keep track of where we left off because we may be called again (without the PRJ_CB_DATA_FLAG_ENUM_RESTART_SCAN flag) to continue filling in data. This is why we keep track of the index we need to use.

The first step in the loop is to fill in details of the item: is it a subdirectory or a “file”? We can also specify the size of its data and common times like creation time, modify time, etc.:

while (info.Index < info.Objects.size()) {
	PRJ_FILE_BASIC_INFO itemInfo{};
	auto& item = info.Objects[info.Index];
	itemInfo.IsDirectory = item.TypeName == L"Directory";
	itemInfo.FileSize = itemInfo.IsDirectory ? 0 : 
		GetObjectSize((callbackData->FilePathName + std::wstring(L"\\") + item.Name).c_str(), item);

We fill in two details: a directory or not, based on the kernel object type being “Directory”, and a file size (in case of another type object). What is the meaning of a “file size”? It can mean whatever we want it to mean, including just specifying a size of zero. However, I decided that the “data” being held in an object would be text that provides the object’s name, type, and target (if it’s a symbolic link). Here are a few example when running the provider and using a command window:

C:\objectmanager>dir p*
 Volume in drive C is OS
 Volume Serial Number is 18CF-552E

 Directory of C:\objectmanager

02/20/2024  11:09 AM                60 PdcPort.ALPC Port
02/20/2024  11:09 AM                76 PendingRenameMutex.Mutant
02/20/2024  11:09 AM                78 PowerMonitorPort.ALPC Port
02/20/2024  11:09 AM                64 PowerPort.ALPC Port
02/20/2024  11:09 AM                88 PrjFltPort.FilterConnectionPort
               5 File(s)            366 bytes
               0 Dir(s)  518,890,110,976 bytes free

C:\objectmanager>type PendingRenameMutex.Mutant
Name: PendingRenameMutex
Type: Mutant

C:\objectmanager>type powerport
Name: PowerPort
Type: ALPC Port

Here is PRJ_FILE_BASIC_INFO:

typedef struct PRJ_FILE_BASIC_INFO {
    BOOLEAN IsDirectory;
    INT64 FileSize;
    LARGE_INTEGER CreationTime;
    LARGE_INTEGER LastAccessTime;
    LARGE_INTEGER LastWriteTime;
    LARGE_INTEGER ChangeTime;
    UINT32 FileAttributes;
} PRJ_FILE_BASIC_INFO;

What is the meaning of the various times and file attributes? It can mean whatever you want – it might make sense for some types of data. If left at zero, the current time is used.

GetObjectSize is a helper function that calculates the number of bytes needed to keep the object’s text, which is what is reported to the file system.

Now we can pass the information for the item to ProjFS by calling PrjFillDirEntryBuffer:

	if (FAILED(::PrjFillDirEntryBuffer(
		(itemInfo.IsDirectory ? item.Name : (item.Name + L"." + item.TypeName)).c_str(), 
		&itemInfo, dirEntryBufferHandle)))
		break;
	info.Index++;
}

The “name” of the item is comprised of the kernel object’s name, and the “file extension” is the object’s type name. This is just a matter of choice – I could have passed the object’s name only so that it would appear as a file with no extension. If the call to PrjFillDirEntryBuffer fails, it means the buffer is full, so we break out, but the index is not incremented, so we can provide the next object in the next callback that does not requires a rescan.

We have two callbacks remaining. One is GetPlaceholderInformationCallback, whose purpose is to provide “placeholder” information about an item, without providing its data. This is used by the file system for caching purposes. The implementation is like so:

HRESULT ObjectManagerProjection::DoGetPlaceholderInformationCallback(const PRJ_CALLBACK_DATA* callbackData) {
	auto path = callbackData->FilePathName;
	auto dir = ObjectManager::DirectoryExists(path);
	std::optional<ObjectNameAndType> object;
	if (!dir)
		object = ObjectManager::ObjectExists(path);
	if(!dir && !object)
		return HRESULT_FROM_WIN32(ERROR_FILE_NOT_FOUND);

	PRJ_PLACEHOLDER_INFO info{};
	info.FileBasicInfo.IsDirectory = dir;
	info.FileBasicInfo.FileSize = dir ? 0 : GetObjectSize(path, object.value());
	return PrjWritePlaceholderInfo(m_VirtContext, callbackData->FilePathName, &info, sizeof(info));
}

The item could be a file or a directory. We use the file path name provided to figure out if it’s a directory kernel object or something else by utilizing some helpers in the ObjectManager class (we’ll examine those later). Then the structure PRJ_PLACEHOLDER_INFO is filled with the details and provided to PrjWritePlaceholderInfo.

The final required callback is the one that provides the data for files – objects in our case:

HRESULT ObjectManagerProjection::DoGetFileDataCallback(const PRJ_CALLBACK_DATA* callbackData, UINT64 byteOffset, UINT32 length) {
	auto object = ObjectManager::ObjectExists(callbackData->FilePathName);
	if (!object)
		return HRESULT_FROM_WIN32(ERROR_FILE_NOT_FOUND);

	auto buffer = ::PrjAllocateAlignedBuffer(m_VirtContext, length);
	if (!buffer)
		return E_OUTOFMEMORY;

	auto data = GetObjectData(callbackData->FilePathName, object.value());
	memcpy(buffer, (PBYTE)data.c_str() + byteOffset, length);
	auto hr = ::PrjWriteFileData(m_VirtContext, &callbackData->DataStreamId, buffer, byteOffset, length);
	::PrjFreeAlignedBuffer(buffer);

	return hr;
}

First we check if the object’s path is valid. Next, we need to allocate buffer for the data. There are some ProjFS alignment requirements, so we call PrjAllocateAlignedBuffer to allocate a properly-aligned buffer. Then we get the object data (a string, by calling our helper GetObjectData), and copy it into the allocated buffer. Finally, we pass the buffer to PrjWriteFileData and free the buffer. The byte offset provided is usually zero, but could theoretically be larger if the client reads from a non-zero position, so we must be prepared for it. In our case, the data is small, but in general it could be arbitrarily large.

GetObjectData itself looks like this:

std::wstring ObjectManagerProjection::GetObjectData(PCWSTR fullname, ObjectNameAndType const& info) {
	std::wstring target;
	if (info.TypeName == L"SymbolicLink") {
		target = ObjectManager::GetSymbolicLinkTarget(fullname);
	}
	auto result = std::format(L"Name: {}\nType: {}\n", info.Name, info.TypeName);
	if (!target.empty())
		result = std::format(L"{}Target: {}\n", result, target);
	return result;
}

It calls a helper function, ObjectManager::GetSymbolicLinkTarget in case of a symbolic link, and builds the final string by using format (C++ 20) before returning it to the caller.

That’s all for the provider, except when terminating:

void ObjectManagerProjection::Term() {
	::PrjStopVirtualizing(m_VirtContext);
}

The Object Manager

Looking into the ObjectManager helper class is somewhat out of the focus of this post, since it has nothing to do with ProjFS. It uses native APIs to enumerate objects in the object manager’s namespace and get details of a symbolic link’s target. For more information about the native APIs, check out my book “Windows Native API Programming” or search online. First, it includes <Winternl.h> to get some basic native functions like RtlInitUnicodeString, and also adds the APIs for directory objects:

typedef struct _OBJECT_DIRECTORY_INFORMATION {
	UNICODE_STRING Name;
	UNICODE_STRING TypeName;
} OBJECT_DIRECTORY_INFORMATION, * POBJECT_DIRECTORY_INFORMATION;

#define DIRECTORY_QUERY  0x0001

extern "C" {
	NTSTATUS NTAPI NtOpenDirectoryObject(
		_Out_ PHANDLE hDirectory,
		_In_ ACCESS_MASK AccessMask,
		_In_ POBJECT_ATTRIBUTES ObjectAttributes);

	NTSTATUS NTAPI NtQuerySymbolicLinkObject(
		_In_ HANDLE LinkHandle,
		_Inout_ PUNICODE_STRING LinkTarget,
		_Out_opt_ PULONG ReturnedLength);

	NTSTATUS NTAPI NtQueryDirectoryObject(
		_In_  HANDLE hDirectory,
		_Out_ POBJECT_DIRECTORY_INFORMATION DirectoryEntryBuffer,
		_In_  ULONG DirectoryEntryBufferSize,
		_In_  BOOLEAN  bOnlyFirstEntry,
		_In_  BOOLEAN bFirstEntry,
		_In_  PULONG  EntryIndex,
		_Out_ PULONG  BytesReturned);
	NTSTATUS NTAPI NtOpenSymbolicLinkObject(
		_Out_  PHANDLE LinkHandle,
		_In_   ACCESS_MASK DesiredAccess,
		_In_   POBJECT_ATTRIBUTES ObjectAttributes);
}

Here is the main code that enumerates directory objects (some details omitted for clarity, see the full source code in the Github repo):

std::vector<ObjectNameAndType> ObjectManager::EnumDirectoryObjects(PCWSTR path, 
	PCWSTR objectName, std::function<bool(PCWSTR)> compare) {
	std::vector<ObjectNameAndType> objects;
	HANDLE hDirectory;
	OBJECT_ATTRIBUTES attr;
	UNICODE_STRING name;
	std::wstring spath(path);
	if (spath[0] != L'\\')
		spath = L'\\' + spath;

	std::wstring object(objectName ? objectName : L"");

	RtlInitUnicodeString(&name, spath.c_str());
	InitializeObjectAttributes(&attr, &name, 0, nullptr, nullptr);
	if (!NT_SUCCESS(NtOpenDirectoryObject(&hDirectory, DIRECTORY_QUERY, &attr)))
		return objects;

	objects.reserve(128);
	BYTE buffer[1 << 12];
	auto info = reinterpret_cast<OBJECT_DIRECTORY_INFORMATION*>(buffer);
	bool first = true;
	ULONG size, index = 0;
	for (;;) {
		auto start = index;
		if (!NT_SUCCESS(NtQueryDirectoryObject(hDirectory, info, sizeof(buffer), FALSE, first, &index, &size)))
			break;
		first = false;
		for (ULONG i = 0; i < index - start; i++) {
			ObjectNameAndType data;
			auto& p = info[i];
			data.Name = std::wstring(p.Name.Buffer, p.Name.Length / sizeof(WCHAR));
			if(compare && !compare(data.Name.c_str()))
				continue;
			data.TypeName = std::wstring(p.TypeName.Buffer, p.TypeName.Length / sizeof(WCHAR));
			if(!objectName)
				objects.push_back(std::move(data));
			if (objectName && _wcsicmp(object.c_str(), data.Name.c_str()) == 0 || 
				_wcsicmp(object.c_str(), (data.Name + L"." + data.TypeName).c_str()) == 0) {
				objects.push_back(std::move(data));
				break;
			}
		}
	}
	::CloseHandle(hDirectory);
	return objects;
}

NtQueryDirectoryObject is called in a loop with increasing indices until it fails. The returned details for each entry is the object’s name and type name.

Here is how to get a symbolic link’s target:

std::wstring ObjectManager::GetSymbolicLinkTarget(PCWSTR path) {
	std::wstring spath(path);
	if (spath[0] != L'\\')
		spath = L"\\" + spath;

	HANDLE hLink;
	OBJECT_ATTRIBUTES attr;
	std::wstring target;
	UNICODE_STRING name;
	RtlInitUnicodeString(&name, spath.c_str());
	InitializeObjectAttributes(&attr, &name, 0, nullptr, nullptr);
	if (NT_SUCCESS(NtOpenSymbolicLinkObject(&hLink, GENERIC_READ, &attr))) {
		WCHAR buffer[1 << 10];
		UNICODE_STRING result;
		result.Buffer = buffer;
		result.MaximumLength = sizeof(buffer);
		if (NT_SUCCESS(NtQuerySymbolicLinkObject(hLink, &result, nullptr)))
			target.assign(result.Buffer, result.Length / sizeof(WCHAR));
		::CloseHandle(hLink);
	}
	return target;
}

See the full source code at https://github.com/zodiacon/ObjMgrProjFS.

Conclusion

The example provided is the bare minimum needed to write a ProjFS provider. This could be interesting for various types of data that is convenient to access with I/O APIs. Feel free to extend the example and resolve any bugs.

Unless you’ve been living under a rock for the past several years (and you are a software developer), the Rust programming language is hard to ignore – in fact, it’s been voted as the “most loved” language for several years (whatever that means). Rust provides the power and performance of C++ with full memory and concurrency safety. It’s a system programming languages, but has high-level features like functional programming style and modularity. That said, Rust has a relatively steep learning curve compared to other mainstream languages.

I’m happy to announce a new training class – Rust Programming Masterclass. This is a brand new, 4 day class, split into 8 half-days, that covers all the foundational pieces of Rust. Here is the list of modules:

• Module 1: Introduction to Rust
• Module 2: Language Fundamentals
• Module 3: Ownership
• Module 4: Compound Types
• Module 5: Common Types and Collections
• Module 6: Modules and Project Management
• Module 7: Error Handling
• Module 8: Generics and Traits
• Module 9: Smart Pointers
• Module 10: Functional Programming
• Module 11: Threads and Concurrency
• Module 12: Async and Await
• Module 13: Unsafe Rust and Interoperability
• Module 14: Macros
• Module 15: Lifetimes

Dates are listed below. The times are 11am-3pm EST (8am-12pm PST) (4pm-8pm UT)
March: 25, 27, 29, April: 1, 3, 5, 8, 10.

Cost: 850 USD (if paid by an individual), 1500 USD if paid by a company. Previous students in my classes get 10% off.

Special bonus for this course: anyone registering gets a 50% discount to any two courses at https://training.trainsec.net.

Registration

If you’d like to register, please send me an email to [email protected] and provide your full name, company (if any), preferred contact email, and your time zone.

The sessions will be recorded, so you can watch any part you may be missing, or that may be somewhat overwhelming in “real time”.

As usual, if you have any questions, feel free to send me an email, or DM on X (twitter) or Linkedin.

I promised this class a while back, and now it is happening. This is a brand new, 3 day class, split into 6 half-days, that covers the x64 processor architecture, programming in general, and programming in the context of Windows. The syllabus can be found here. It may change a bit, but should mostly be stable.

Dates are listed below. The times are 12pm-4pm EST (9am-1pm PST) (5pm-9pm UT)
January: 15, 17, 22, 24, 29, 31.

Cost: 750 USD (if paid by an individual), 1400 USD if paid by a company.

Registration

If you’d like to register, please send me an email to [email protected] and provide your full name, company (if any), preferred contact email, and your time zone. Previous participants in my classes get 10% off.

The sessions will be recorded, so you can watch any part you may be missing, or that may be somewhat overwhelming in “real time”.

As usual, if you have any questions, feel free to send me an email, or DM on X (twitter) or Linkedin.

It’s been a while since I have taught a public class. I am happy to launch a new class that combines Windows Kernel Programming and Advanced Windows Kernel Programming into a 6-day (48 hours) masterclass. The full syllabus can be found here.

There is a special bonus for those registering for this class: you get one free recorded course from Windows Internals and Programming (trainsec.net)!

For those who have attended the Windows Kernel Programming class, and wish to capture the more “advanced” stuff, I offer one of two options:

• Join the second part (3 days) of the training, at 60% of the entire course cost.
• Register for the entire course with a 20% discount, and get the free recorded course.
  

The course is planned to stretch from mid-December to late-January, in 4-hour chunks to make it easier to combine with other activities and also have the time to do lab exercises (very important for truly understanding the material). Yes, I know christmas is in the middle there, I’ll keep the last week of December free

The course will be conducted remotely using MS Teams or similar.

Dates and times (not final, but unlikely to change much, if at all):

• Dec 2023: 12, 14, 19, 21: 12pm-4pm EST (9am-1pm PST)
• Jan 2024: 2, 4, 9, 11, 16, 18, 23, 25: 12pm-4pm EST (9am-1pm PST)

Training cost:

• Early bird (until Nov 22): 1150 USD
• After Nov 22: 1450 USD

If you’d like to register, please write to [email protected] with your name, company name (if any), and time zone. If you have any question, use the same email or DM me on X (Twitter) or Linkedin.

Many developers and researcher are faimilar with the SetWindowsHookEx API that provides ways to intercept certain operations related to user interface, such as messages targetting windows. Most of these hooks can be set on a specific thread, or all threads attached to the current desktop. A short video showing how to use this API can be found here. One of the options is to inject a DLL to the target process(es) that is invoked inline to process the relevant events.

There is another mechanism, less known, that provides various events that relate to UI, that can similarly be processed by a callback. This can be attached to a specific thread or process, or to all processes that have threads attached to the current desktop. The API in question is SetWinEventHook:

HWINEVENTHOOK SetWinEventHook(
    _In_ DWORD eventMin,
    _In_ DWORD eventMax,
    _In_opt_ HMODULE hmodWinEventProc,
    _In_ WINEVENTPROC pfnWinEventProc,
    _In_ DWORD idProcess,
    _In_ DWORD idThread,
    _In_ DWORD dwFlags);

The function allows invoking a callback (pfnWinEventProc) when an event occurs. eventMin and eventMax provide a simple way to filter events. If all events are needed, EVENT_MIN and EVENT_MAX can be used to cover every possible event. The module is needed if the function is inside a DLL, so that hmodWinEventProc is the module handle loaded into the calling process. The DLL will automatically be loaded into target process(es) as needed, very similar to the way SetWindowsHookEx works.

idProcess and idThread allow targetting a specific thread, a specific process, or all processes in the current desktop (if both IDs are zero). Targetting all processes is possible even without a DLL. In that case, the event information is marshalled back to the caller’s process and invoked there. This does require to pass the WINEVENT_OUTOFCONTEXT flag to indicate this requirement. The following example shows how to install such event monitoring for all processes/threads in the current desktop:

auto hHook = ::SetWinEventHook(EVENT_MIN, EVENT_MAX, nullptr, 
    OnEvent, 0, 0,
	WINEVENT_OUTOFCONTEXT | 
    WINEVENT_SKIPOWNPROCESS | WINEVENT_SKIPOWNTHREAD);

::GetMessage(nullptr, nullptr, 0, 0);

The last two flags indicate that events from the caller’s process should not be reported. Notice the weird-looking GetMessage call – it’s required for the event handler to be called. The weird part is that a MSG structure is not needed, contrary to the function’s SAL that requires a non-NULL pointer.

The event handler itself can do anything, however, the information provided is fundamentally different than SetWindowsHookEx callbacks. For example, there is no way to “change” anything – it’s just notifying about things that already happended. These events are related to accessibility and are not directly related to windows messaging. Here is the event handler prototype:

void CALLBACK OnEvent(HWINEVENTHOOK hWinEventHook, DWORD event, 
    HWND hwnd, LONG idObject, LONG idChild, DWORD eventTid, DWORD time);

event is the event being reported. Various such events are defined in WinUser.h and there are many values that can be used by third paries and OEMs. It’s worthwile checking the header file because every Microsoft-defined event has details as to when such an event is raised, and the meaning of idObject, idChild and hwnd for that event. eventTid is the thread ID from which the event originated. hwnd is typically the window or constrol associated with the event (if any) – some events are general enough so that no hwnd is provided.

We can get more information on the object that is associated with the event by tapping into the accessibility API. Accessibility objects implement the IAccessible COM interface at least, but may implement other interfaces as well. To get an IAccesible pointer from an event handler, we can use AccessibleObjectFromEvent:

CComPtr<IAccessible> spAcc;
CComVariant child;
::AccessibleObjectFromEvent(hwnd, idObject, idChild, &spAcc, &child);

I’ve included <atlbase.h> to get the ATL client side support (smart pointers and COM type wrappers). Other APIs that can bring an IAccessible in other contexts include AccessibleObjectFromPoint and AccessibleObjectFromWindow.

Note that you must also include <oleacc.h> and link with oleacc.lib.

IAccessible has quite a few methods and properties, the simplest of which is Name that is mandatory for implementors to provide:

CComBSTR name;
spAcc->get_accName(CComVariant(idChild), &name);

Refer to the documentation for other members of IAccessible. We can also get the details of the process associated with the event by going through the window handle or the thread ID and retrieving the executable name. Here is an example with a window handle:

DWORD pid = 0;
WCHAR exeName[MAX_PATH];
PCWSTR pExeName = L"";

if (hwnd && ::GetWindowThreadProcessId(hwnd, &pid)) {
    auto hProcess = ::OpenProcess(PROCESS_QUERY_LIMITED_INFORMATION, FALSE, pid);
    if (hProcess) {
        DWORD size = _countof(exeName);
        if (::QueryFullProcessImageName(hProcess, 0, exeName, &size))
            pExeName = wcsrchr(exeName, L'\\') + 1;
        ::CloseHandle(hProcess);
    }
}

GetWindowThreadProcessId retrieves the process ID (and thread ID) associated with a window handle. We could go with the given thread ID – call OpenThread and then GetProcessIdOfThread. The interested reader is welcome to try this approach to retrieve the process ID. Here is the full event handler for this example dumping all using printf:

void CALLBACK OnEvent(HWINEVENTHOOK hWinEventHook, DWORD event, HWND hwnd,
    LONG idObject, LONG idChild, DWORD idEventThread, DWORD time) {
    CComPtr<IAccessible> spAcc;
    CComVariant child;
    ::AccessibleObjectFromEvent(hwnd, idObject, idChild, &spAcc, &child);
    CComBSTR name;
    if (spAcc)
        spAcc->get_accName(CComVariant(idChild), &name);
    DWORD pid = 0;
    WCHAR exeName[MAX_PATH];
    PCWSTR pExeName = L"";

    if (hwnd && ::GetWindowThreadProcessId(hwnd, &pid)) {
        auto hProcess = ::OpenProcess(PROCESS_QUERY_LIMITED_INFORMATION, FALSE, pid);
        if (hProcess) {
            DWORD size = _countof(exeName);
            if (::QueryFullProcessImageName(hProcess, 0, exeName, &size))
                pExeName = wcsrchr(exeName, L'\\') + 1;
            ::CloseHandle(hProcess);
        }
    }
    printf("Event: 0x%X (%s) HWND: 0x%p, ID: 0x%X Child: 0x%X TID: %u PID: %u (%ws) Time: %u Name: %ws\n",
        event, EventNameToString(event),
        hwnd, idObject, idChild, idEventThread, 
        pid, pExeName,
        time, name.m_str);
}

EventNameToString is a little helper converting some event IDs to names. If you run this code (SimpleWinEventHook project), you’ll see lots of output, because one of the reported events is EVENT_OBJECT_LOCATIONCHANGE that is raised (among other reasons) when the mouse cursor position changes:

Event: 0x800C (Name Change) HWND: 0x00000000000216F6, ID: 0xFFFFFFFC Child: 0x1DC TID: 39060 PID: 64932 (Taskmgr.exe) Time: 78492375 Name: (null)
Event: 0x8000 (Object Create) HWND: 0x00000000000216F6, ID: 0xFFFFFFFC Child: 0x1DD TID: 39060 PID: 64932 (Taskmgr.exe) Time: 78492375 Name: (null)
Event: 0x800C (Name Change) HWND: 0x00000000000216F6, ID: 0xFFFFFFFC Child: 0x1DD TID: 39060 PID: 64932 (Taskmgr.exe) Time: 78492375 Name: (null)
Event: 0x8000 (Object Create) HWND: 0x00000000000216F6, ID: 0xFFFFFFFC Child: 0x1DE TID: 39060 PID: 64932 (Taskmgr.exe) Time: 78492375 Name: (null)
Event: 0x800C (Name Change) HWND: 0x00000000000216F6, ID: 0xFFFFFFFC Child: 0x1DE TID: 39060 PID: 64932 (Taskmgr.exe) Time: 78492375 Name: (null)
...
Event: 0x800B (Location Changed) HWND: 0x0000000000000000, ID: 0xFFFFFFF7 Child: 0x0 TID: 72172 PID: 0 () Time: 78492562 Name: Normal
Event: 0x800B (Location Changed) HWND: 0x0000000000000000, ID: 0xFFFFFFF7 Child: 0x0 TID: 72172 PID: 0 () Time: 78492562 Name: Normal
...
Event: 0x800B (Location Changed) HWND: 0x0000000000000000, ID: 0xFFFFFFF7 Child: 0x0 TID: 72172 PID: 0 () Time: 78492718 Name: Vertical size
Event: 0x800B (Location Changed) HWND: 0x0000000000000000, ID: 0xFFFFFFF7 Child: 0x0 TID: 72172 PID: 0 () Time: 78492734 Name: Vertical size
Event: 0x800C (Name Change) HWND: 0x0000000000000000, ID: 0xFFFFFFF7 Child: 0x0 TID: 72172 PID: 0 () Time: 78492734 Name: Normal
Event: 0x800A (State Changed) HWND: 0x000000000001019E, ID: 0xFFFFFFFC Child: 0x16 TID: 15636 PID: 14060 (explorer.exe) Time: 78493000 Name: (null)
Event: 0x800A (State Changed) HWND: 0x00000000000101B0, ID: 0xFFFFFFFC Child: 0x6 TID: 15636 PID: 14060 (explorer.exe) Time: 78493000 Name: (null)
Event: 0x8004 () HWND: 0x0000000000010010, ID: 0xFFFFFFFC Child: 0x0 TID: 72172 PID: 1756 () Time: 78493000 Name: Desktop
Event: 0x8 (Capture Start) HWND: 0x0000000000271D5A, ID: 0x0 Child: 0x0 TID: 72172 PID: 67928 (WindowsTerminal.exe) Time: 78493000 Name: c:\Dev\Temp\WinEventHooks\x64\Debug\SimpleWinEventHook.exe
Event: 0x800B (Location Changed) HWND: 0x0000000000000000, ID: 0xFFFFFFF7 Child: 0x0 TID: 72172 PID: 0 () Time: 78493093 Name: Normal
Event: 0x8001 (Object Destroy) HWND: 0x00000000000216F6, ID: 0xFFFFFFFC Child: 0x45 TID: 39060 PID: 64932 (Taskmgr.exe) Time: 78493093 Name: (null)
Event: 0x8001 (Object Destroy) HWND: 0x00000000000216F6, ID: 0xFFFFFFFC Child: 0xB0 TID: 39060 PID: 64932 (Taskmgr.exe) Time: 78493093 Name: (null)
...
Event: 0x800C (Name Change) HWND: 0x00000000000216F6, ID: 0xFFFFFFFC Child: 0x1A TID: 39060 PID: 64932 (Taskmgr.exe) Time: 78493093 Name: (null)
Event: 0x800C (Name Change) HWND: 0x00000000000216F6, ID: 0xFFFFFFFC Child: 0x1B TID: 39060 PID: 64932 (Taskmgr.exe) Time: 78493109 Name: (null)
Event: 0x800B (Location Changed) HWND: 0x0000000000000000, ID: 0xFFFFFFF7 Child: 0x0 TID: 72172 PID: 0 () Time: 78493109 Name: Normal
Event: 0x9 (Capture End) HWND: 0x0000000000271D5A, ID: 0x0 Child: 0x0 TID: 72172 PID: 67928 (WindowsTerminal.exe) Time: 78493109 Name: c:\Dev\Temp\WinEventHooks\x64\Debug\SimpleWinEventHook.exe

DLL Injection

Instead of getting events on the SetWinEventHook caller’s thread, a DLL can be injected. Such a DLL must export the event handler so that the process setting up the handler can locate the function with GetProcAddress.

As an example, I created a simple DLL that implements the event handler similarly to the previous example (without the process name) like so:

extern "C" __declspec(dllexport)
void CALLBACK OnEvent(HWINEVENTHOOK hWinEventHook, DWORD event, HWND hwnd,
	LONG idObject, LONG idChild, DWORD idEventThread, DWORD time) {
	CComPtr<IAccessible> spAcc;
	CComVariant child;
	::AccessibleObjectFromEvent(hwnd, idObject, idChild, &spAcc, &child);
	CComBSTR name;
	if (spAcc)
		spAcc->get_accName(CComVariant(idChild), &name);

	printf("Event: 0x%X (%s) HWND: 0x%p, ID: 0x%X Child: 0x%X TID: %u Time: %u Name: %ws\n",
		event, EventNameToString(event),
        hwnd, idObject, idChild, idEventThread,
		time, name.m_str);
}

Note the function is exported. The code uses printf, but there is no guarantee that a target process has a console to use. The DllMain function creates such a console and attached the standard output handle to it (otherwise printf wouldn’t have an output handle, since the process wasn’t bootstraped with a console):

HANDLE hConsole;

BOOL APIENTRY DllMain(HMODULE hModule, DWORD reason, PVOID lpReserved) {
	switch (reason) {
		case DLL_PROCESS_DETACH:
			if (hConsole)   // be nice
				::CloseHandle(hConsole);
			break;

		case DLL_PROCESS_ATTACH:
			if (::AllocConsole()) {
				auto hConsole = ::CreateFile(L"CONOUT$", GENERIC_WRITE, 
                    0, nullptr, OPEN_EXISTING, 0, nullptr);
				if (hConsole == INVALID_HANDLE_VALUE)
					return FALSE;
				::SetStdHandle(STD_OUTPUT_HANDLE, hConsole);
			}
			break;
	}
	return TRUE;
}

The injector process (WinHookInject project) first grabs a target process ID (if any):

int main(int argc, const char* argv[]) {
	DWORD pid = argc < 2 ? 0 : atoi(argv[1]);
	if (pid == 0) {
		printf("Warning: injecting to potentially processes with threads connected to the current desktop.\n");
		printf("Continue? (y/n) ");
		char ans[3];
		gets_s(ans);
		if (tolower(ans[0]) != 'y')
			return 0;
	}

The warning is shown of no PID is provided, because creating consoles for certain processes could wreak havoc. If you do want to inject a DLL to all processes on the desktop, avoid creating consoles.

Once we have a target process (or not), we need to load the DLL (hardcoded for simplicity) and grab the exported event handler function:

auto hLib = ::LoadLibrary(L"Injected.Dll");
if (!hLib) {
	printf("DLL not found!\n");
	return 1;
}
auto OnEvent = (WINEVENTPROC)::GetProcAddress(hLib, "OnEvent");
if (!OnEvent) {
	printf("Event handler not found!\n");
	return 1;
}

The final step is to register the handler. If you’re targetting all processes, you’re better off limiting the events you’re interested in, especially the noisy ones. If you just want a DLL injected and you don’t care about any events, select a range that has no events and then call a relevant function to force the DLL to be loaded into the target process(es). I’ll let the interested reader figure these things out.

auto hHook = ::SetWinEventHook(EVENT_MIN, EVENT_MAX, 
	hLib, OnEvent, pid, 0, WINEVENT_INCONTEXT);
::GetMessage(nullptr, nullptr, 0, 0);

Note the arguments include the DLL module, the handler address, and the flag WINEVENT_INCONTEXT. Here is some output when using this DLL on a Notepad instance. A console is created the first time Notepad causes an event to be raised:

Event: 0x800B (Name Change) HWND: 0x0000000000000000, ID: 0xFFFFFFF7 Child: 0x0 TID: 34756 Time: 70717718 Name: Edit
Event: 0x800C (Name Change) HWND: 0x0000000000000000, ID: 0xFFFFFFF7 Child: 0x0 TID: 34756 Time: 70717718 Name: Horizontal size
Event: 0x800B (Name Change) HWND: 0x0000000000000000, ID: 0xFFFFFFF7 Child: 0x0 TID: 34756 Time: 70717718 Name: Horizontal size
Event: 0x800B (Name Change) HWND: 0x0000000000000000, ID: 0xFFFFFFF7 Child: 0x0 TID: 29516 Time: 70717734 Name: Horizontal size
Event: 0x800C (Name Change) HWND: 0x0000000000000000, ID: 0xFFFFFFF7 Child: 0x0 TID: 29516 Time: 70717734 Name: Edit
Event: 0x800B (Name Change) HWND: 0x0000000000000000, ID: 0xFFFFFFF7 Child: 0x0 TID: 29516 Time: 70717734 Name: Edit
Event: 0x800B (Name Change) HWND: 0x0000000000000000, ID: 0xFFFFFFF7 Child: 0x0 TID: 29516 Time: 70717734 Name: Edit
Event: 0x800B (Name Change) HWND: 0x0000000000000000, ID: 0xFFFFFFF7 Child: 0x0 TID: 29516 Time: 70717750 Name: Edit
Event: 0x800B (Name Change) HWND: 0x0000000000000000, ID: 0xFFFFFFF7 Child: 0x0 TID: 29516 Time: 70717765 Name: Edit
Event: 0x800B (Name Change) HWND: 0x0000000000000000, ID: 0xFFFFFFF7 Child: 0x0 TID: 29516 Time: 70717765 Name: Edit
Event: 0x800B (Name Change) HWND: 0x0000000000000000, ID: 0xFFFFFFF7 Child: 0x0 TID: 29516 Time: 70717781 Name: Edit
Event: 0x800B (Name Change) HWND: 0x0000000000000000, ID: 0xFFFFFFF7 Child: 0x0 TID: 29516 Time: 70717781 Name: Edit
Event: 0x800B (Name Change) HWND: 0x0000000000000000, ID: 0xFFFFFFF7 Child: 0x0 TID: 29516 Time: 70717796 Name: Edit
Event: 0x800B (Name Change) HWND: 0x0000000000000000, ID: 0xFFFFFFF7 Child: 0x0 TID: 29516 Time: 70717796 Name: Edit
Event: 0x800B (Name Change) HWND: 0x0000000000000000, ID: 0xFFFFFFF7 Child: 0x0 TID: 29516 Time: 70717812 Name: Edit
Event: 0x800B (Name Change) HWND: 0x0000000000000000, ID: 0xFFFFFFF7 Child: 0x0 TID: 29516 Time: 70717812 Name: Edit
Event: 0x800B (Name Change) HWND: 0x0000000000000000, ID: 0xFFFFFFF7 Child: 0x0 TID: 29516 Time: 70717828 Name: Edit
Event: 0x800B (Name Change) HWND: 0x0000000000000000, ID: 0xFFFFFFF7 Child: 0x0 TID: 29516 Time: 70717843 Name: Edit
Event: 0x8 (Capture Start) HWND: 0x0000000000091CAC, ID: 0x0 Child: 0x0 TID: 29516 Time: 70717843 Name: (null)
Event: 0x3 (Foreground) HWND: 0x00000000000A1D50, ID: 0x0 Child: 0x0 TID: 34756 Time: 70717843 Name: Untitled - Notepad
Event: 0x8004 () HWND: 0x0000000000010010, ID: 0xFFFFFFFC Child: 0x0 TID: 29516 Time: 70717859 Name: Desktop 1
Event: 0x800B (Name Change) HWND: 0x00000000000A1D50, ID: 0x0 Child: 0x0 TID: 34756 Time: 70717859 Name: Untitled - Notepad
...

The full code is at zodiacon/WinEventHooks: SetWinEventHook Sample (github.com)

Ever since I realized BASIC wasn’t the only living programming language, I thought about writing my own. Who wouldn’t? If you’re a developer, surely this idea popped into your mind at some point. No matter how much you love a particular programming language, you always have some ideas for improvement or even removal of annoying features.

The post assumes you have some background in compilers, and understand concepts like tokenizing (scanning), parsing, and Abstract Syntax Trees (ASTs)

Obviously, writing a programming language is not for the faint of heart. Even before you set out to implement your language, you have to design it first. Or maybe you have some fundamental ideas that would make your language unique, and you may decide to flesh out the details while you’re implementing it.

A new programming language does not have to be “general-purpose” – that is, it could be a “domain specific language” (DSL), which means it’s best suited for certain domain(s) or tasks. This makes your life (usually) at least somewhat easier; in addition, you’ll be unlikely to compete with the gazillion general-purpose languages out there. Still, a general-purpose language might be your goal.

Designing a programming language is a big topic, well outside the scope of this post. I’ll focus on the implementation details, so to speak. There are other considerations for a programming language beyond the language itself – its accompanying standard library, tooling (e.g., some IDE or at least syntax highlighting), debugging, testing, and few more. One decision is whether to make your language compiled or interpreted. This decision may not affect some aspects of the implementation, but it will definitely affect the language’s back-end. You can even support both interpretation and compilation for maximum flexibility.

I played around with the idea of creating a programming language for many years, never really getting very far beyond a basic parser and a minimal interpreter. Lately, I’ve read more about Pratt Parsing, that sparked my interest again. Pratt Parsing is one of many techniques for parsing expressions, something like “a+2*b”, and doing that correctly (parenthesis, operator precedence and associativity). Pratt parsing is really elegant, much more so than other techniques, and it’s also more flexible, supporting (indirectly) ternary operations and other unusual constructs. Once you have an expression parser, the rest of the parser is fairly easy to implement (relatively speaking) using the recursive-descent approach which is well suited for hand-crafted parsers.

Robert Nystrom gives a nice introduction to Pratt Parsing and an elegant idea for implementing it. His implementation is in Java, but there is a link to a C# implementation and even one in Rust. My go-to language is C++ (still), so you know where this is going. I’ve implemented a Pratt parser based on Robert’s ideas, and it turned out very well.

I’ve also been interested in visualization (a term which has way too much stuffed into it), but I thought I’d start small. A popular teaching language in the 80s was LOGO. Although it was treated as a “toy language”, it was a full-blown language, mostly resembling LISP internally.

However, LOGO became famous because of the “Turtle Graphics” built-in support that was provided, which allowed drawing with an imaginary turtle (you could even ask LOGO to show it) that would follow your commands like moving forward, backwards, rotating, lifting the pen and putting it back down. Why not create a fun version of Turtle Graphics with ideas from LOGO?

Here is an example from LOGO to draw a symmetric hexagon:

REPEAT 6 [ FD 100 RT 60 ]

You can probably guess what is going on here. “FD” is “forward” and “RT” is “right”, although it could be mistaken for “rotate”. LOGO supported functions as well, so you could create complex shapes by reusing functions.

My language, called “Logo2” for a lack of originality at this time, tries to capture that fun drawing, but put the syntax more inline with the C-family of functions, which I like more. The above hexagon is written with Logo2 like so:

repeat 6 {
    fd(100); rt(60);
}

Indentation is not significant, so it all could be placed on the same line. You can also define functions and execute them:

fn circle(size, steps) {
    repeat steps {
        fd(size); rt(360 / steps);
    }
}

repeat 10 {
    circle(80, 20); rt(36);
}

I also added support for colors, with the pencolor(r,g,b) function, something I don’t recall LOGO having in the 80s.

Implementation

There are 3 main projects in the solution (a fourth project in the works to create a simple IDE for easier experimentation):

• Logo2Core – contains the tokenizer, parser, and interpreter.
• Logo2Runtime – contains the runtime support for turtle graphics, currently using GDI+.
• Logo2 – is a simple REPL, that can parse and execute single line statements. If you provide a command line argument, it’s treated as file name to be parsed and executed. Anything not inside a function is executed directly (for now).

The Tokenizer

The tokenizer’s job (Tokenizer class) is to read text and turn it into a bunch of tokens. A token is a single unit of the language, like a number, keyword, identifier, operator, etc. To start tokenization, the Tokenize method can be invoked with the string to tokenize.

The Next() method returns the next token, whereas the Peek() method returns the next token without advancing the stream forward. This means the tokenizer is not doing all the work immediately, but only advanced to the next token when requested. The parser is the one “driving” the tokenizer.

The implementation of the tokenizer is not perfect, but it works well-enough. I didn’t want to use any existing tools like YACC (or BISON), for a couple reasons. For one, I don’t like generated code that I have little control colover. Second, I like to understand what I am writing. Writing a tokenizer is not rocket science, but it’s not trivial, either. And since one of my goals is to experiment, I need the freedom not available with generated code.

The Parser

The parser is much more interesting than the tokenizer (by far). This is where the syntax of the language is fleshed out. Just like with tokenization, usage of tools like LEX (or FLEX) is inappropriate. In fact, most languages have their own hand-written parser. The parser accepts a string to parse (Parse method) or a filename (ParseFile method) and begins the parsing. It calls on the tokenizer when the next token is needed.

The Init method of the parser initializes the tokenizer with the specific tokens it should be able to recognize (like specific keywords and operators), and also initializes its own “parslets” (defined in the above mentioned article) to make Pratt Parsing work. I will not show here the Pratt Parsing part since there’s quite a bit of code there, but here is an example of parsing the “repeat” statement:

std::unique_ptr<RepeatStatement> Parser::ParseRepeatStatement() {
	Next();		// eat "repeat"
	auto times = ParseExpression();

	m_LoopCount++;
	auto block = ParseBlock();
	m_LoopCount--;
    return std::make_unique<RepeatStatement>(
        std::move(times), std::move(block));
}

ParseExpression parses an expression to be used for the argument to repeat. Then ParseBlock is called to parse a curly-brace surrounded block of code. Finally, the result is an AST node representing a “repeat” statement is created, initialized, and returned to the caller.

The m_LoopCount variable is incremented when entering loop parsing and decremented afterwards. This is done so that parsing the keywords break and continue can check if there is any enclosing loop for these keywords to make sense.

Here is ParseBlock:

std::unique_ptr<BlockExpression>
Parser::ParseBlock(std::vector<std::string> const& args) {
	if (!Match(TokenType::OpenBrace))
		AddError(ParserError(ParseErrorType::OpenBraceExpected, Peek()));

	m_Symbols.push(std::make_unique<SymbolTable>(m_Symbols.top().get()));

	for (auto& arg : args) {
		Symbol sym;
		sym.Name = arg;
		sym.Flags = SymbolFlags::None;
		sym.Type = SymbolType::Argument;
		AddSymbol(sym);
	}

	auto block = std::make_unique<BlockExpression>();
	while (Peek().Type != TokenType::CloseBrace) {
		auto stmt = ParseStatement();
		if (!stmt)
			break;
		block->Add(std::move(stmt));
	}
	Next();		// eat close brace
	m_Symbols.pop();
	return block;
}

ParseBlock starts by making sure there is an open curly brace. Then it creates a symbol table and pushes it to be the “current” as there is a new scope within the block. The parameter to ParseBlock is used when parsing a function body, where these “args” are the parameters to the function. If this is the case, they are added to the symbol table as local variables.

The main work is to call ParseStatement as many times as needed until a close brace is encountered. The block is a vector of statements being filled up. Finally, the symbol table is popped and the AST node is returned.

ParseStatement is a big switch that calls the appropriate specific parsing method based on the first token encountered. Here is an excerpt:

std::unique_ptr<Statement> Parser::ParseStatement() {
	auto peek = Peek();
	if (peek.Type == TokenType::Invalid) {
		return nullptr;
	}

	switch (peek.Type) {
		case TokenType::Keyword_Var: 
             return ParseVarConstStatement(false);
		case TokenType::Keyword_Const: 
             return ParseVarConstStatement(true);
		case TokenType::Keyword_Repeat: 
             return ParseRepeatStatement();
		case TokenType::Keyword_While: 
             return ParseWhileStatement();
		case TokenType::Keyword_Fn: 
             return ParseFunctionDeclaration();
		case TokenType::Keyword_Return: 
             return ParseReturnStatement();
        case TokenType::Keyword_Break: 
             return ParseBreakContinueStatement(false);
        case TokenType::Keyword_Continue:
             return ParseBreakContinueStatement(true);
	}
	auto expr = ParseExpression();
	if (expr) {
		Match(TokenType::SemiColon);
		return std::make_unique<ExpressionStatement>(std::move(expr));
	}
	AddError(ParserError(ParseErrorType::InvalidStatement, peek));
	return nullptr;
}

If a statement is not recognized, an expression parsing is attempted. This allows using Logo2 as a simple calculator, for example. ParseStatement is where the support for more statements is added based on an initial token.

Once an AST is built by the parser, the next step is to execute the AST by some interpreter. In a more complex language (maybe once it grows some more), some semantic analysis may be appropriate, which is about looking at the usage of the language beyond the syntax. For now, we’ll just interpret what we have, and if any error is encountered it’s going to be a runtime error. Some parsing errors can be caught without semantic analysis, but some cannot.

The Interpreter

The Interpreter class provides the runtime behavior, by “executing” the AST. It receives the root of the AST tree constructed by the parser by implementing the well-known Visitor design pattern, whose purpose here is to decouple between the AST node types and the way they are handled by the interpreter. Alternatively, it would be possible to add a virtual “Execute” or “Eval” method to AST nodes, so the nodes can “evaluate” themselves, but that creates coupling, and goes against the single-responsibility principle (SRP) that states that a class should have one and only one job. Using the visitor pattern also makes it easier to add semantic analysis later without modifying the AST node types.

The gist of the visitor pattern is to have an “Accept” method in the AST nodes that calls back to whoever (the visitor) with the current node details. For example, here it is for a binary operator:

class BinaryExpression : public Expression {
public:
    BinaryExpression(std::unique_ptr<Expression> left, 
        Token op, std::unique_ptr<Expression> right);
	Value Accept(Visitor* visitor) const override;

	std::string ToString() const override;

	Expression* Left() const;
	Expression* Right() const;
	Token const& Operator() const;

private:
	std::unique_ptr<Expression> m_Left, m_Right;
	Token m_Operator;
};

Value BinaryExpression::Accept(Visitor* visitor) const {
	return visitor->VisitBinary(this);
}

This same idea is repeated for all concrete AST nodes. The Visitor type is abstract, implemented by the Interpreter class having methods like: VisitBinary, VisitRepeat, etc.

Each one of these “Visit” method’s purpose is to “execute” (or evaluate) that node. Here is an excerpt for the binary expression visiting:

Value Interpreter::VisitBinary(BinaryExpression const* expr) {
    switch (expr->Operator().Type) {
    case TokenType::Add: 
       return expr->Left()->Accept(this) + expr->Right()->Accept(this);
    case TokenType::Sub:
       return expr->Left()->Accept(this) - expr->Right()->Accept(this);
    case TokenType::Mul:
       return expr->Left()->Accept(this) * expr->Right()->Accept(this);
    case TokenType::Div:
       return expr->Left()->Accept(this) / expr->Right()->Accept(this);
    }
    return Value();
}

Here it is for “repeat”:

Value Interpreter::VisitRepeat(RepeatStatement const* expr) {
    auto count = Eval(expr->Count());
    if (!count.IsInteger())
        throw RuntimeError(ErrorType::TypeMismatch, expr->Count());

    auto n = count.Integer();
    while (n-- > 0) {
        try {
            Eval(expr->Block());
        }
        catch (BreakOrContinue const& bc) {
            if (!bc.Continue)
                break;
        }
    }
    return nullptr;     // repeat has no return value
}

You should get the idea at this point. (Eval is just a simple wrapper that calls Accept with the provided node).

The Value type used with the above code (the one returned from Accept methods is the way to represent “values” in Logo2. Logo2 is a dynamically typed language (at least for now), so variables can hold any one of a listed of supported types, encapsulated in Value. You can think of that as a C-style union. Specifically, it wraps a std::variant<> C++17 type that currently supports the following: 64-bit integer, 64-bit floating point (double), bool, string (std::string), and null (representing no value). The list of possibilities will increase, allowing user-defined types as well.

Turtle Graphics

The Logo2Runtime project contains the support for managing turtles, and displaying their “drawings”. The Turtle class is a graphics-free type to manage the state of the turtle – its position and heading, but also a list of “command” indicating operations the turtle has been instructed to do, such as drawing a line, changing color, or changing width of drawing. This list is necessary whenever a window’s output needs to be refreshed.

The Window class servers as a wrapper for an HWND, that also has the “power” to draw a set of turtle commands. Here is its DrawTurtle method:

void Window::DrawTurtle(Gdiplus::Graphics& g, Turtle* t) const {
    for (auto& cmd : t->GetCommands()) {
        DrawTurtleCommand(g, t, cmd);
    }
}

Each command does the right thing:

void Window::DrawTurtleCommand(Gdiplus::Graphics& g, Turtle* t, 
    TurtleCommand const& cmd) const {
    switch (cmd.Type) {
        case TurtleCommandType::DrawLine:
            g.DrawLine(m_Pen.get(), cmd.Line.From.X, 
               cmd.Line.From.Y, cmd.Line.To.X, cmd.Line.To.Y);
            break;

        case TurtleCommandType::SetWidth:
        {
            Color color;
            m_Pen->GetColor(&color);
            m_Pen.reset(new Pen(color, cmd.Width));
            break;
        }

        case TurtleCommandType::SetColor:
        {
            Color color;
            color.SetValue(cmd.Color);
            m_Pen.reset(new Pen(color, m_Pen->GetWidth()));
            break;
        }
    }
}

The graphical objects are GDI+ objects provided by the Windows API. Of course, it would be possible to switch to a different API. I chose GDI+ for its flexibility and 2D capabilities.

The Runtime class ties a turtle and a window together. It holds on to a (single) Turtle object and single Window object. In the future, this is going to be more dynamic, so any number of windows and turtles can be created, even more than one turtle in the same window.

The REPL

A simple REPL is implemented in the Logo2 project. It’s not trivial, as there is a user interface that must be kept alive, meaning messages have to be pumped. This means using functions like gets_s is not good enough, as they block the calling thread. Assuming the UI is on the same thread, this will cause the UI to become non-responsive. For now, the same thread is used, so that no special synchronization is required. The downside is that a custom input “loop” has to be written, and currently it’s very simple, and only supports the BACKSPACE key for typing error correction.

The first step is to get the input, key by key. If there is no key available, messages are pumped. A WM_QUIT message indicates it’s time to exit. Not very elegant, but here goes:

Tokenizer t;
Parser parser(t);
Interpreter inter;
Runtime runtime(inter);
runtime.Init();
runtime.CreateLogoWindow(L"Logo 2", 800, 800);

for (;;) {
	std::print(">> ");
	std::string input;
	int ch = 0;
	MSG msg{};
	while (ch != 13) {
		while (::PeekMessage(&msg, nullptr, 0, 0, PM_REMOVE) && 
                 msg.message != WM_QUIT) {
			::TranslateMessage(&msg);
			::DispatchMessage(&msg);
		}
		if (msg.message == WM_QUIT)
			break;

		if (_kbhit()) {
			ch = _getch();
			if (isprint(ch)) {
				input += (char)ch;
				printf("%c", ch);
			}
			else if (ch == 8) {		// backspace
				printf("\b \b");
				input = input.substr(0, input.length() - 1);
			}
			else {
				if (_kbhit())
					_getch();
			}
		}
	}

	if (msg.message == WM_QUIT)
		break;

Once we have a line of input, it’s time to parse and (if no errors occur), execute:

try {
	printf("\n");
	auto ast = parser.Parse(input);
	if (parser.HasErrors()) {
		for (auto& err : parser.Errors()) {
			printf("Error (%d,%d): %d\n", 
               err.ErrorToken.Line, err.ErrorToken.Col, err.Error);
		}
		continue;
	}
	try {
		auto result = ast->Accept(&inter); // execute!
		if (result != nullptr)
			std::println("{}", result.ToString());
	}
	catch (RuntimeError const& err) {
		printf("Runtime error: %d\n", (int)err.Error);
	}
}
catch (ParserError const& err) {
	printf("Error (%d,%d): %d\n", err.ErrorToken.Line, 
         err.ErrorToken.Col, err.Error);
	continue;
}

Some parser errors are accumulated in a vector, some throw an exception (errors where it would be difficult for the parser to recover confidently). At runtime, errors could occur as well, such as the wrong types being used with certain operations.

Conclusion

Writing a language can be lots of fun. You can invent your “dream” language. For me, the Logo2 experiment is ongoing. I’m planning to build a simple IDE, to extend the language to support user-defined types, lambdas (with closures), and much more. Your ideas are welcome as well!

The project is at zodiacon/Logo2 (github.com)

When a thread is created, it has some priority, which sets its importance compared to other threads competing for CPU time. The thread priority range is 0 to 31 (31 being the highest), where priority zero is used by the memory manager’s zero-page thread(s), whose purpose is to zero out physical pages (for reasons outside the scope of this post), so technically the allowed priority range is 1 to 31.

It stands to reason (to some extent), that a developer could change a thread’s priority to some valid value in the range of 1 to 31, but this is not the case. The Windows API sets up rules as to how thread priorities may change. First, there is a process priority class (sometimes called Base Priority), that specifies the default thread priority within that process. Processes don’t run – threads do, but still this is a process property and affects all threads in the process. You can see the value of this property very simply with Task Manager’s Base Priority column (not visible by default):

Base Priority column in Task Manager

There are six priority classes (the priority of which is specified after the colon):

• Idle (called Low in Task Manager, probably not to give the wrong impression): 4
• Below Normal (6)
• Normal (8)
• Above Normal (10)
• Highest (13)
• Realtime (24)

A few required notes:

• Normal is the default priority class unless overridden in some way. For example, double-clicking an executable in Explorer will launch a new process with priority class of Normal (8).
• The term “Realtime” does not imply Windows is a real-time OS; it’s not. “Real-time” just means “higher than all the others”.
• To set the Realtime priority class, the process in question must have the SeIncreaseBasePriorityPrivilege, normally granted to administrators. If “Realtime” is requested, but the process’s token does not poses that privilege, the result is “High”. The reason has do to with the fact that many kernel threads have priorities in the real-time range, and it could be problematic if too many threads spend a lot of time running in these priorities, potentially leading to kernel threads getting less time than they need.

Is this the end of the story? Not quite. For example, looking at Task Manager, processes like Csrss.exe (Windows subsystem process) or Smss.exe (Session manager) seem to have a priority class of Normal as well. Is this really the case? Yes and no (everyone likes that kind of answer, right?) We’ll get to that soon.

Setting a Thread’s priority

Changing the process priority class is possible with the SetPriorityClass API. For example, a process can change its own priority class like so:

::SetPriorityClass(::GetCurrentProcess(), HIGH_PRIORITY_CLASS);

You can do the same in .NET by utilizing the System.Diagnostics.Process class:

Process.GetCurrentProcess().PriorityClass = ProcessPriorityClass.High;

You can also change priority class using Task Manager or Process Explorer, by right-clicking a process and selecting “Set Priority”.

Once the priority class is changed, it affects all threads in that process. But how?

It turns out that a specific thread’s priority can be changed around the process priority class. The following diagram shows the full picture:

Every small rectangle in the above diagram indicates a valid thread priority. For example, the Normal priority classes allows setting thread priorities to 1, 6, 7, 8, 9, 10, 15. To be more generic, here are the rules for all except the Realtime class. A thread priority is by default the same as the process priority class, but it can be -1, -2, +1, +2 from that base, or have two extreme values (internally called “Saturation”) with the values 1 and 15.

The Realtime range is unique, where the base priority is 24, but all priorities from 16 to 31 are available. The SetThreadPriority API that can be used to change an individual thread’s priority accepts an enumeration value (as its second argument) rather than an absolute value. Here are the macro definitions:

#define THREAD_PRIORITY_LOWEST         // -2  
#define THREAD_PRIORITY_BELOW_NORMAL   // -1
#define THREAD_PRIORITY_NORMAL         // 0
#define THREAD_PRIORITY_HIGHEST        // + 2
#define THREAD_PRIORITY_ABOVE_NORMAL   // + 1
#define THREAD_PRIORITY_TIME_CRITICAL  // 15 or 31
#define THREAD_PRIORITY_IDLE           // 1 or 16

Here is an example of changing the current thread’s priority to +2 compared to the process priority class:

::SetThreadPriority(::GetCurrentThread(), THREAD_PRIORITY_HIGHEST);

And a C# version:

Thread.CurrentThread.Priority = ThreadPriority.Highest;

You can see threads priorities in Process Explorer‘s bottom view:

Thread priorities in Process Explorer

There are two columns for priorities – A base priority and a Dynamic priority. The base priority is the priority set by code (SetThreadPriority) or the default, while the dynamic priority is the current thread’s priority, which could be slightly higher than the base (temporarily), and is changed because of certain decisions made by the kernel scheduler and other components and drivers that can produce such an effect. These thread boosting scenarios are outside the scope of this post.

If you want to see all threads in the system with their priorities, you can use my System Explorer tool, and select System / Threads menu item:

System Explorer showing all threads in the system

The two priority column are shown (Priority is the same as Dynamic Priority in Process Explorer). You can sort by any column, including the priority to see which threads have the highest priority.

Native APIs

If you look in Process Explorer, there is a column named Base Priority under the Process Performance tab:

Process Performance tab

With this column visible, it indicates a process priority with a number. It’s mostly the corresponding number to the priority class (e.g. 10 for Above Normal, 13 for High, etc.), but not always. For example, Smss.exe has a value of 11, which doesn’t correspond to any priority class. Csrss.exe processes have a value of 13.

Changing to these numbers can only be done with the Native API. Specifically, NtSetInformationProcess with the ProcessBasePriority enumeration value can make that change. Weirdly enough, if the value is higher than the current process priority, the same privilege mentioned earlier is required. The weird part, is that calling SetPriorityClass to change Normal to High always works, but calling NtSetInformationProcess to change from 8 to 13 (the same as Normal to High) requires that privilege; oh, well.

What about a specific thread? The native API allows changing a priority of a thread to any given value directly without the need to depend on the process priority class. Choosing a priority in the realtime range (16 or higher) still requires that privilege. But at least you get the flexibility to choose any priority value. The call to use is NtSetInformationThread with ThreadPriority enumeration. For example:

KPRIORITY priority = 14;
NtSetInformationThread(NtCurrentThread(), ThreadPriority, 
    &priority, sizeof(priority));

Note: the definitions for the native API can be obtained from the phnt project.

What happens if you need a high priority (16 or higher) but don’t have admin privileges in the process? Enter the Multimedia Class Scheduler.

The MMCSS Service

The multimedia class service coupled with a driver (mmcss.sys) provide a thread priority service intended for “multimedia” applications that would like to get some guarantee when “playing” multimedia. For example, if you have Spotify running locally, you’ll find there is one thread with priority 22, although the process itself has a priority class Normal:

Spotify threads

You can use the MMCSS API to get that kind of support. There is a Registry key that defines several “tasks” applications can use. Third parties can add more tasks:

MMCSS tasks

The base key is: HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Multimedia\SystemProfile\Tasks

The selected “Audio” task has several properties that are read by the MMCSS service. The most important is Priority, which is between 1 (low) and 8 (high) representing the relative priority compared to other “tasks”. Some values aren’t currently used (GPU Priority, SFIO Priority), so don’t expect anything from these.

Here is an example that uses the MMCSS API to increase the current thread’s priority:

#include <Windows.h>
#include <avrt.h>

#pragma comment(lib, "avrt")

int main() {
	DWORD index = 0;
    HANDLE h = AvSetMmThreadCharacteristics(L"Audio", &index);
	AvSetMmThreadPriority(h, AVRT_PRIORITY_HIGH);

The priority itself is an enumeration, where each value corresponds to a range of priorities (all above 15).

The returned HANDLE by the way, is to the MMCSS device (\Device\MMCSS). The argument to AvSetMmThreadCharacteristics must correspond to one of the “Tasks” registered. Calling AvRevertMmThreadCharacteristics reverts the thread to “normal”. There are more APIs in that set, check the docs.

Happy Threading!

A while back I blogged about the differences between the virtual desktop feature exposed to users on Windows 10/11, and the Desktops tool from Sysinternals. In this post, I’d like to shed some more light on Window Stations, desktops, and windows. I assume you have read the aforementioned blog post before continuing.

We know that Window Stations are contained in sessions. Can we enumerate these? The EnumWindowStations API is available in the Windows API, but it only returns the Windows Stations in the current session. There is no “EnumSessionWindowStations”. Window Stations, however, are named objects, and so are visible in tools such as WinObj (running elevated):

Window stations in session 0

The Window Stations in session 0 are at \Windows\WindowStations
The Window Stations in session x are at \Sessions\x\Windows\WindowStations

The OpenWindowStation API only accepts a “local” name, under the callers session. The native NtUserOpenWindowStation API (from Win32u.dll) is more flexible, accepting a full object name:

HWINSTA NtUserOpenWindowStation(POBJECT_ATTRIBUTES attr, ACCESS_MASK access);

Here is an example that opens the “msswindowstation” Window Station:

#include <Windows.h>
#include <winternl.h>

#pragma comment(lib, "ntdll")

HWINSTA NTAPI _NtUserOpenWindowStation(_In_ POBJECT_ATTRIBUTES attr, _In_ ACCESS_MASK access);
int main() {
	// force Win32u.DLL to load
	::LoadLibrary(L"user32");
	auto NtUserOpenWindowStation = (decltype(_NtUserOpenWindowStation)*)
		::GetProcAddress(::GetModuleHandle(L"win32u"), "NtUserOpenWindowStation");

	UNICODE_STRING winStaName;
	RtlInitUnicodeString(&winStaName, L"\\Windows\\WindowStations\\msswindowstation");
	OBJECT_ATTRIBUTES winStaAttr;
	InitializeObjectAttributes(&winStaAttr, &winStaName, 0, nullptr, nullptr);
	auto hWinSta = NtUserOpenWindowStation(&winStaAttr, READ_CONTROL);
	if (hWinSta) {
        // do something with hWinSta
        ::CloseWindowStation(hWinSta);
    }

You may or may not have enough power to open a handle with the required access – depending on the Window Station in question. Those in session 0 are hardly accessible from non-session 0 processes, even with the SYSTEM account. You can examine their security descriptor with the kernel debugger (as other tools will return access denied):

lkd> !object \Windows\WindowStations\msswindowstation
Object: ffffe103f5321c00  Type: (ffffe103bb0f0ae0) WindowStation
    ObjectHeader: ffffe103f5321bd0 (new version)
    HandleCount: 4  PointerCount: 98285
    Directory Object: ffff808433e412b0  Name: msswindowstation
lkd> dt nt!_OBJECT_HEADER ffffe103f5321bd0

   +0x000 PointerCount     : 0n98285
   +0x008 HandleCount      : 0n4
   +0x008 NextToFree       : 0x00000000`00000004 Void
   +0x010 Lock             : _EX_PUSH_LOCK
   +0x018 TypeIndex        : 0xa2 ''
   +0x019 TraceFlags       : 0 ''
   +0x019 DbgRefTrace      : 0y0
   +0x019 DbgTracePermanent : 0y0
   +0x01a InfoMask         : 0xe ''
   +0x01b Flags            : 0 ''
   +0x01b NewObject        : 0y0
   +0x01b KernelObject     : 0y0
   +0x01b KernelOnlyAccess : 0y0
   +0x01b ExclusiveObject  : 0y0
   +0x01b PermanentObject  : 0y0
   +0x01b DefaultSecurityQuota : 0y0
   +0x01b SingleHandleEntry : 0y0
   +0x01b DeletedInline    : 0y0
   +0x01c Reserved         : 0
   +0x020 ObjectCreateInfo : 0xfffff801`21c53940 _OBJECT_CREATE_INFORMATION
   +0x020 QuotaBlockCharged : 0xfffff801`21c53940 Void
   +0x028 SecurityDescriptor : 0xffff8084`3da8aa6c Void
   +0x030 Body             : _QUAD
lkd> !sd 0xffff8084`3da8aa60
->Revision: 0x1
->Sbz1    : 0x0
->Control : 0x8014
            SE_DACL_PRESENT
            SE_SACL_PRESENT
            SE_SELF_RELATIVE
->Owner   : S-1-5-18
->Group   : S-1-5-18
->Dacl    : 
->Dacl    : ->AclRevision: 0x2
->Dacl    : ->Sbz1       : 0x0
->Dacl    : ->AclSize    : 0x1c
->Dacl    : ->AceCount   : 0x1
->Dacl    : ->Sbz2       : 0x0
->Dacl    : ->Ace[0]: ->AceType: ACCESS_ALLOWED_ACE_TYPE
->Dacl    : ->Ace[0]: ->AceFlags: 0x0
->Dacl    : ->Ace[0]: ->AceSize: 0x14
->Dacl    : ->Ace[0]: ->Mask : 0x0000011b
->Dacl    : ->Ace[0]: ->SID: S-1-1-0

You can become SYSTEM to help with access by using PsExec from Sysinternals to launch a command window (or whatever) as SYSTEM but still run in the interactive session:

psexec -s -i -d cmd.exe

If all else fails, you may need to use the “Take Ownership” privilege to make yourself the owner of the object and change its DACL to allow yourself full access. Apparently, even that won’t work, as getting something from a Window Station in another session seems to be blocked (see replies in Twitter thread). READ_CONTROL is available to get some basic info.

Here is a screenshot of Object Explorer running under SYSTEM that shows some details of the “msswindowstation” Window Station:

Guess which processes hold handles to this hidden Windows Station?

Once you are able to get a Window Station handle, you may be able to go one step deeper by enumerating desktops, if you managed to get at least WINSTA_ENUMDESKTOPS access mask:

::EnumDesktops(hWinSta, [](auto deskname, auto param) -> BOOL {
	printf(" Desktop: %ws\n", deskname);
	auto h = (HWINSTA)param;
	return TRUE;
	}, (LPARAM)hWinSta);

Going one level deeper, you can enumerate the top-level windows in each desktop (if any). For that you will need to connect the process to the Window Station of interest and then call EnumDesktopWindows:

void DoEnumDesktopWindows(HWINSTA hWinSta, PCWSTR name) {
	if (::SetProcessWindowStation(hWinSta)) {
		auto hdesk = ::OpenDesktop(name, 0, FALSE, DESKTOP_READOBJECTS);
		if (!hdesk) {
			printf("--- failed to open desktop %ws (%d)\n", name, ::GetLastError());
			return;
		}
		static WCHAR pname[MAX_PATH];
		::EnumDesktopWindows(hdesk, [](auto hwnd, auto) -> BOOL {
			static WCHAR text[64];
			if (::IsWindowVisible(hwnd) && ::GetWindowText(hwnd, text, _countof(text)) > 0) {
				DWORD pid;
				auto tid = ::GetWindowThreadProcessId(hwnd, &pid);
				auto hProcess = ::OpenProcess(PROCESS_QUERY_LIMITED_INFORMATION, FALSE, pid);
				BOOL exeNameFound = FALSE;
				PWSTR exeName = nullptr;
				if (hProcess) {
					DWORD size = MAX_PATH;
					exeNameFound = ::QueryFullProcessImageName(hProcess, 0, pname, &size);
					::CloseHandle(hProcess);
					if (exeNameFound) {
						exeName = ::wcsrchr(pname, L'\\');
						if (exeName == nullptr)
							exeName = pname;
						else
							exeName++;
					}
				}
				printf("  HWND: 0x%08X PID: 0x%X (%d) %ws TID: 0x%X (%d): %ws\n", 
					(DWORD)(DWORD_PTR)hwnd, pid, pid, 
					exeNameFound ? exeName : L"", tid, tid, text);
			}
			return TRUE;
			}, 0);
		::CloseDesktop(hdesk);
	}
}

Calling SetProcessWindowStation can only work with a Windows Station that belongs to the current session.

Here is an example output for the interactive session (Window Stations enumerated with EnumWindowStations):

Window station: WinSta0
 Desktop: Default
  HWND: 0x00010E38 PID: 0x4D04 (19716) Zoom.exe TID: 0x5FF8 (24568): ZPToolBarParentWnd
  HWND: 0x000A1C7A PID: 0xB804 (47108) VsDebugConsole.exe TID: 0xDB50 (56144): D:\Dev\winsta\x64\Debug\winsta.exe
  HWND: 0x00031DE8 PID: 0xBF40 (48960) devenv.exe TID: 0x94E8 (38120): winsta - Microsoft Visual Studio Preview
  HWND: 0x00031526 PID: 0x1384 (4996) msedge.exe TID: 0xE7C (3708): zodiacon/ObjectExplorer: Explore Kernel Objects on Windows and
  HWND: 0x00171A9A PID: 0xA40C (41996)  TID: 0x9C08 (39944): WindowStation (\Windows\WindowStations\msswindowstation)
  HWND: 0x000319D0 PID: 0xA40C (41996)  TID: 0x9C08 (39944): Object Manager - Object Explorer 2.0.2.0 (Administrator)
  HWND: 0x001117DC PID: 0x253C (9532) ObjExp.exe TID: 0x9E10 (40464): Object Manager - Object Explorer 2.0.2.0 (Administrator)
  HWND: 0x00031CA8 PID: 0xBE5C (48732) devenv.exe TID: 0xC250 (49744): OpenWinSta - Microsoft Visual Studio Preview (Administrator)
  HWND: 0x000B1884 PID: 0xA8A0 (43168) DbgX.Shell.exe TID: 0xA668 (42600):  - KD '', Local Connection  - WinDbg 1.2306.12001.0 (Administra
...
  HWND: 0x000101C8 PID: 0x3598 (13720) explorer.exe TID: 0x359C (13724): Program Manager
Window station: Service-0x0-45193$
 Desktop: sbox_alternate_desktop_0x6A80
 Desktop: sbox_alternate_desktop_0xA94C
 Desktop: sbox_alternate_desktop_0x3D8C
 Desktop: sbox_alternate_desktop_0x7EF8
 Desktop: sbox_alternate_desktop_0x72FC
 Desktop: sbox_alternate_desktop_0x27B4
 Desktop: sbox_alternate_desktop_0x6E80
 Desktop: sbox_alternate_desktop_0x6C54
 Desktop: sbox_alternate_desktop_0x68C8
 Desktop: sbox_alternate_desktop_0x691C
 Desktop: sbox_alternate_desktop_0x4150
 Desktop: sbox_alternate_desktop_0x6254
 Desktop: sbox_alternate_desktop_0x5B9C
 Desktop: sbox_alternate_desktop_0x59B4
 Desktop: sbox_alternate_desktop_0x1384
 Desktop: sbox_alternate_desktop_0x5480

The desktops in the Window Station “Service-0x0-45193$” above don’t seem to have top-level visible windows.

You can also access the clipboard and atom table of a given Windows Station, if you have a powerful enough handle. I’ll leave that as an exercise as well.

Finally, what about session enumeration? That’s the easy part – no need to call NtOpenSession with Session objects that can be found in the “\KernelObjects” directory in the Object Manager’s namespace – the WTS family of functions can be used. Specifically, WTSEnumerateSessionsEx can provide some important properties of a session:

void EnumSessions() {
	DWORD level = 1;
	PWTS_SESSION_INFO_1 info;
	DWORD count = 0;
	::WTSEnumerateSessionsEx(WTS_CURRENT_SERVER_HANDLE, &level, 0, &info, &count);
	for (DWORD i = 0; i < count; i++) {
		auto& data = info[i];
		printf("Session %d (%ws) Username: %ws\\%ws State: %s\n", data.SessionId, data.pSessionName, 
			data.pDomainName ? data.pDomainName : L"NT AUTHORITY", data.pUserName ? data.pUserName : L"SYSTEM", 
			StateToString((WindowStationState)data.State));
    }
	::WTSFreeMemory(info);
}

What about creating a process to use a different Window Station and desktop? One member of the STARTUPINFO structure passed to CreateProcess (lpDesktop) allows setting a desktop name and an optional Windows Station name separated by a backslash (e.g. “MyWinSta\MyDesktop”).

There is more to Window Stations and Desktops that meets the eye… this should give interested readers a head start in doing further research.

A few people have asked me if I provide mentoring. I didn’t consider this avenue of contribution, but after some thought I am happy to open a software development personal mentoring program. My goal is to give from my knowledge and experience in software development, but not just that.

Being a great software developer is not just about writing high-quality code. I will not elaborate on the qualities of great software engineers, as there are many such lists in various articles. Many of the qualities of great software developers are the same as other roles in the IT industry (and some in any industry for that matter).

This is what I can offer:

• Guidance on how to tackle new topics
• How to be more productive
• Building self-confidence
• Working on foundational/core software related pieces to boost professionalism and performance
• Specific help in subjects I know well enough (C, C++, C#, Rust, Windows, Kernel, Hardware, Graphics, Algorithms, UI, math, writing, …)
• Anything I can help with that aligns with your goals!

Program details

• Initial meeting to discuss goals, purpose, and expectations.
• Program length: 3, 6, 8, or 12 months.
• 1×1 meetings (see below).
• Discussions on activities, challenges, support, resources, and hoe to measure success.
• Chat access between 1×1 meetings.

1×1 Meetings

• 40 min/week (first month)
• 1 hour/2 weeks. (month 2-3)
• 1 hour/3 weeks. (month 4-6)
• 1 hour/4 weeks. (month 7+)

Program cost

• 3 months: 3900 USD (paid in 3 installments)
• 6 months: 4900 USD (paid in 4 installments)
• 8 months: 5900 USD (paid in 5 installments)
• 12 months: 6900 USD (paid in 6 installments)

Get in touch

If you’re interested, send me an email to [email protected], and we’ll get the process going. If you have any questions or doubts, just email me. I’ll try to help as best I can.

Much of the Windows kernel functionality is exposed via kernel objects. Processes, threads, events, desktops, semaphores, and many other object types exist. Some object types can have string-based names, which means they can be “looked up” by that name. In this post, I’d like to consider some subtleties that concern object names.

Let’s start by examining kernel object handles in Process Explorer. When we select a process of interest, we can see the list of handles in one of the bottom views:

Handles view in Process Explorer

However, Process Explorer shows what it considers handles to named objects only by default. But even that is not quite right. You will find certain object types in this view that don’t have string-based names. The simplest example is processes. Processes have numeric IDs, rather than string-based names. Still, Process Explorer shows processes with a “name” that shows the process executable name and its unique process ID. This is useful information, for sure, but it’s not the object’s name.

Same goes for threads: these are displayed, even though threads (like processes) have numeric IDs rather than string-based names.

If you wish to see all handles in a process, you need to check the menu item Show Unnamed Handles and Mappings in the View menu.

Object Name Lifetime

What is the lifetime associated with an object’s name? This sounds like a weird question. Kernel objects are reference counted, so obviously when an object reference count drops to zero, it is destroyed, and its name is deleted as well. This is correct in part. Let’s look a bit deeper.

The following example code creates a Notepad process, and puts it into a named Job object (error handling omitted for brevity):

PROCESS_INFORMATION pi;
STARTUPINFO si = { sizeof(si) };

WCHAR name[] = L"notepad";
::CreateProcess(nullptr, name, nullptr, nullptr, FALSE, 0, 
	nullptr, nullptr, &si, &pi);

HANDLE hJob = ::CreateJobObject(nullptr, L"MyTestJob");
::AssignProcessToJobObject(hJob, pi.hProcess);

After running the above code, we can open Process Explorer, locate the new Notepad process, double-click it to get to its properties, and then navigate to the Job tab:

We can clearly see the job object’s name, prefixed with “\Sessions\1\BaseNamedObjects” because simple object names (like “MyTestJob”) are prepended with a session-relative directory name, making the name unique to this session only, which means processes in other sessions can create objects with the same name (“MyTestJob”) without any collision. Further details on names and sessions is outside the scope of this post.

Let’s see what the kernel debugger has to say regarding this job object:

lkd> !process 0 1 notepad.exe
PROCESS ffffad8cfe3f4080
    SessionId: 1  Cid: 6da0    Peb: 175b3b7000  ParentCid: 16994
    DirBase: 14aa86d000  ObjectTable: ffffc2851aa24540  HandleCount: 233.
    Image: notepad.exe
    VadRoot ffffad8d65d53d40 Vads 90 Clone 0 Private 524. Modified 0. Locked 0.
    DeviceMap ffffc28401714cc0
    Token                             ffffc285355e9060
    ElapsedTime                       00:04:55.078
    UserTime                          00:00:00.000
    KernelTime                        00:00:00.000
    QuotaPoolUsage[PagedPool]         214720
    QuotaPoolUsage[NonPagedPool]      12760
    Working Set Sizes (now,min,max)  (4052, 50, 345) (16208KB, 200KB, 1380KB)
    PeakWorkingSetSize                3972
    VirtualSize                       2101395 Mb
    PeakVirtualSize                   2101436 Mb
    PageFaultCount                    4126
    MemoryPriority                    BACKGROUND
    BasePriority                      8
    CommitCharge                      646
    Job                               ffffad8d14503080

lkd> !object ffffad8d14503080
Object: ffffad8d14503080  Type: (ffffad8cad8b7900) Job
    ObjectHeader: ffffad8d14503050 (new version)
    HandleCount: 1  PointerCount: 32768
    Directory Object: ffffc283fb072730  Name: MyTestJob

Clearly, there is a single handle to the job object. The PointerCount value is not the real reference count because of the kernel’s tracking of the number of usages each handle has (outside the scope of this post as well). To get the real reference count, we can click the PointerCount DML link in WinDbg (the !truref command):

kd> !trueref ffffad8d14503080
ffffad8d14503080: HandleCount: 1 PointerCount: 32768 RealPointerCount: 3

We have a reference count of 3, and since we have one handle, it means there are two references somewhere to this job object.

Now let’s see what happens when we close the job handle we’re holding:

::CloseHandle(hJob);

Reopening the Notepad’s process properties in Process Explorer shows this:

Running the !object command again on the job yields the following:

lkd> !object ffffad8d14503080
Object: ffffad8d14503080  Type: (ffffad8cad8b7900) Job
    ObjectHeader: ffffad8d14503050 (new version)
    HandleCount: 0  PointerCount: 1
    Directory Object: 00000000  Name: MyTestJob

The handle count dropped to zero because we closed our (only) existing handle to the job. The job object’s name seem to be intact at first glance, but not really: The directory object is NULL, which means the object’s name is no longer visible in the object manager’s namespace.

Is the job object alive? Clearly, yes, as the pointer (reference) count is 1. When the handle count it zero, the Pointer Count is the correct reference count, and there is no need to run the !truref command. At this point, you should be able to guess why the object is still alive, and where is that one reference coming from.

If you guessed “the Notepad process”, then you are right. When a process is added to a job, it adds a reference to the job object so that it remains alive if at least one process is part of the job.

We, however, have lost the only handle we have to the job object. Can we get it back knowing the object’s name?

hJob = ::OpenJobObject(JOB_OBJECT_QUERY, FALSE, L"MyTestJob");

This call fails, and GetLastError returns 2 (“the system cannot find the file specified”, which in this case is the job object’s name). This means that the object name is destroyed when the last handle of the object is closed, even if there are outstanding references on the object (the object is alive!).

This the job object example is just that. The same rules apply to any named object.

Is there a way to “preserve” the object name even if all handles are closed? Yes, it’s possible if the object is created as “Permanent”. Unfortunately, this capability is not exposed by the Windows API functions like CreateJobObject, CreateEvent, and all other create functions that accept an object name.

Quick update: The native NtMakePermanentObject can make an object permanent given a handle, if the caller has the SeCreatePermanent privilege. This privilege is not granted to any user/group by default.

A permanent object can be created with kernel APIs, where the flag OBJ_PERMANENT is specified as one of the attribute flags part of the OBJECT_ATTRIBUTES structure that is passed to every object creation API in the kernel.

A “canonical” kernel example is the creation of a callback object. Callback objects are only usable in kernel mode. They provide a way for a driver/kernel to expose notifications in a uniform way, and allow interested parties (drivers/kernel) to register for notifications based on that callback object. Callback objects are created with a name so that they can be looked up easily by interested parties. In fact, there are quite a few callback objects on a typical Windows system, mostly in the Callback object manager namespace:

Most of the above callback objects’ usage is undocumented, except three which are documented in the WDK (ProcessorAdd, PowerState, and SetSystemTime). Creating a callback object with the following code creates the callback object but the name disappears immediately, as the ExCreateCallback API returns an object pointer rather than a handle:

PCALLBACK_OBJECT cb;
UNICODE_STRING name = RTL_CONSTANT_STRING(L"\\Callback\\MyCallback");
OBJECT_ATTRIBUTES cbAttr = RTL_CONSTANT_OBJECT_ATTRIBUTES(&name, 
    OBJ_CASE_INSENSITIVE);
status = ExCreateCallback(&cb, &cbAttr, TRUE, TRUE);

The correct way to create a callback object is to add the OBJ_PERMANENT flag:

PCALLBACK_OBJECT cb;
UNICODE_STRING name = RTL_CONSTANT_STRING(L"\\Callback\\MyCallback");
OBJECT_ATTRIBUTES cbAttr = RTL_CONSTANT_OBJECT_ATTRIBUTES(&name, 
    OBJ_CASE_INSENSITIVE | OBJ_PERMANENT);
status = ExCreateCallback(&cb, &cbAttr, TRUE, TRUE);

A permanent object must be made “temporary” (the opposite of permanent) before actually dereferencing it by calling ObMakeTemporaryObject.

Aside: Getting to an Object’s Name in WinDbg

For those that wonder how to locate an object’s name give its address. I hope that it’s clear enough… (watch the bold text).

lkd> !object ffffad8d190c0080
Object: ffffad8d190c0080  Type: (ffffad8cad8b7900) Job
    ObjectHeader: ffffad8d190c0050 (new version)
    HandleCount: 1  PointerCount: 32770
    Directory Object: ffffc283fb072730  Name: MyTestJob
lkd> dt nt!_OBJECT_HEADER ffffad8d190c0050
   +0x000 PointerCount     : 0n32770
   +0x008 HandleCount      : 0n1
   +0x008 NextToFree       : 0x00000000`00000001 Void
   +0x010 Lock             : _EX_PUSH_LOCK
   +0x018 TypeIndex        : 0xe9 ''
   +0x019 TraceFlags       : 0 ''
   +0x019 DbgRefTrace      : 0y0
   +0x019 DbgTracePermanent : 0y0
   +0x01a InfoMask         : 0xa ''
   +0x01b Flags            : 0 ''
   +0x01b NewObject        : 0y0
   +0x01b KernelObject     : 0y0
   +0x01b KernelOnlyAccess : 0y0
   +0x01b ExclusiveObject  : 0y0
   +0x01b PermanentObject  : 0y0
   +0x01b DefaultSecurityQuota : 0y0
   +0x01b SingleHandleEntry : 0y0
   +0x01b DeletedInline    : 0y0
   +0x01c Reserved         : 0
   +0x020 ObjectCreateInfo : 0xffffad8c`d8e40cc0 _OBJECT_CREATE_INFORMATION
   +0x020 QuotaBlockCharged : 0xffffad8c`d8e40cc0 Void
   +0x028 SecurityDescriptor : 0xffffc284`3dd85eae Void
   +0x030 Body             : _QUAD
lkd> db nt!ObpInfoMaskToOffset L10
fffff807`72625e20  00 20 20 40 10 30 30 50-20 40 40 60 30 50 50 70  .  @.00P @@`0PPp
lkd> dx (nt!_OBJECT_HEADER_NAME_INFO*)(0xffffad8d190c0050 - ((char*)0xfffff807`72625e20)[(((nt!_OBJECT_HEADER*)0xffffad8d190c0050)->InfoMask & 3)])
(nt!_OBJECT_HEADER_NAME_INFO*)(0xffffad8d190c0050 - ((char*)0xfffff807`72625e20)[(((nt!_OBJECT_HEADER*)0xffffad8d190c0050)->InfoMask & 3)])                 : 0xffffad8d190c0030 [Type: _OBJECT_HEADER_NAME_INFO *]
    [+0x000] Directory        : 0xffffc283fb072730 [Type: _OBJECT_DIRECTORY *]
    [+0x008] Name             : "MyTestJob" [Type: _UNICODE_STRING]
    [+0x018] ReferenceCount   : 0 [Type: long]
    [+0x01c] Reserved         : 0x0 [Type: unsigned long]

I’m happy to announce 3 upcoming remote training classes to be held in June and July.

Windows System Programming

This is a 5-day class, split into 10 half-days. The syllabus can be found here.

All times are 11am to 3pm ET (8am to 11am, PT) (4pm to 8pm, London time)

June: 7, 8, 12, 14, 15, 19, 21, 22, 26, 28

Cost: 950 USD if paid by an individual, 1900 USD if paid by a company.

COM Programming

This is a 3-day course, split into 6 half-days. The syllabus can be found here.

All times are 11am to 3pm ET (8am to 11am, PT) (4pm to 8pm, London time)

July: 10, 11, 12, 17, 18, 19

Cost: 750 USD (if paid by an individual), 1500 USD if paid by a company.

x64 Architecture and Programming

This is a brand new, 3 day class, split into 6 half-days, that covers the x64 processor architecture, programming in general, and programming in the context of Windows. The syllabus is not finalized yet, but it will cover at least the following topics:

• General architecture and brief history
• Registers
• Addressing modes
• Stand-alone assembly programs
• Mixing assembly with C/C++
• MSVC compiler-generated assembly
• Operating modes: real, protected, long (+paging)
• Major instruction groups
• Macros
• Shellcode
• BIOS and assembly

July: 24, 25, 26, 31, August: 1, 2

Cost: 750 USD (if paid by an individual), 1500 USD if paid by a company.

Registration

If you’d like to register, please send me an email to [email protected] and provide the name of the training class of interest, your full name, company (if any), preferred contact email, and your time zone. Previous participants in my classes get 10% off. If you register for more than one class, the second (and third) are 10% off as well.

The sessions will be recorded, so you can watch any part you may be missing, or that may be somewhat overwhelming in “real time”.

As usual, if you have any questions, feel free to send me an email, or DM on twitter (@zodiacon) or Linkedin (https://www.linkedin.com/in/pavely/).

I love Graphical User Interfaces, especially the good ones Some people feel more comfortable with a terminal and command line arguments – I prefer a graphical representation, especially when visualization of information can be much more effective than text (even if colorful).

Most of the tools I write are GUI tools; I like colors and graphics – computers are capable of so much graphic and visualization power – why not see it in all its glory? GUIs are not a silver bullet by any means. Sometimes bad GUIs are encountered, which might send the user to the command terminal. I’m not going to discuss here what makes up a good GUI. This post is about technologies to create GUIs.

Disclaimer: much of the rest of this post is subjective – my experience with Windows GUIs. I’m also not discussing web UI – not really in the same scope. I’m interested in taking advantage of the machine, not being constrained or affected by some browser or HTML/CSS/JS engine. The discussion is not exhaustive, either; there is a limit to a post

In the old days, the Win32 User Interface reined supreme. It was created in the days where memory was scarce, colors were few, hardware acceleration did not exist, and consistency was the name of the game. Modern GUIs were just starting to come up.

Windows supports all the standard controls (widgets) a typical GUI application would need. From buttons and menus, to list views and tree views, to edit controls, the standard set of typical application usage was covered. The basis of the Win32 GUI model was (and still is) the might Handle to Window (HWND). This entity represented the surface on which the window (typically a control) would render its graphical representation and handle its interaction logic. This worked fairly well throughout the 1990s and early 2000s.

The model was not perfect, but any means. Customizing controls was difficult, and in some cases downright impossible. Built-in customization was minimal, any substantial customization required subclassing – essentially taking control of handling some window messages differently in the hope of not breaking integration with the default message processing. It was a lot of work at best, and imperfect or impossible at worse. Messages like WM_PAINT and WM_ERASEBKGND were commonly overridden, but also mouse and keyboard-related messages. In some cases, there was no good option for customization and full blown control had to be written from scratch.

Here is a simple example: say you want to change the background color of a button. This should in theory be simple – change some property and you’re done. Not so easy with the Win32 button – it had to be owner-drawn or custom-drawn (WM_CUSTOMDRAW) in later versions of Windows. And that’s really a simple example.

Layout didn’t really exist. Controls were placed at an (x,y) coordinate measured from the top-left corner of the parent window – in pixels, mind you – with a specified width and height. There were no “panels” to handle more complex layout, in a grid for example, horizontally, or vertically, etc.

From a programmatic perspective, working directly with the Windows GUI API was no picnic either. Microsoft realized this, and developed The Microsoft Foundation Classes (MFC) library in the early 1990s to make working with Win32 GUI somewhat easier, by wrapping some of the functionality in C++ classes, and adding some nice features like docking windows. MFC was very popular at the time, as it was easier to use when getting started with building GUIs. It didn’t solve anything fundamental, as it was just using the Win32 GUI API under the covers. Several third-party libraries were written on top of MFC to provide even more functionality out of the box. MFC can still be used today, with Visual Studio still providing wizards and other helpers for MFC developers.

MFC wasn’t perfect of course. Beyond the obvious usage of the Win32 UI controls, it was fairly bloated, dragging with it a large DLL or adding a big static chunk if linked statically. Another library came out, the Windows Template Library (WTL), that provided a thin layer around the Windows GUI API, based on template classes, meaning that there was no “runtime” in the same sense as MFC – no library to link with – just whatever is compiled directly.

Personally, I like WTL a lot. In fact, my tools in recent years use WTL exclusively. It’s much more flexible than MFC, and doesn’t impose a particular way of working as MFC strongly did. The downside is that WTL wasn’t an official Microsoft library, mostly developed by good people inside the company in their spare time. Visual Studio has no special support for WTL. That said, WTL is still being maintained, and had some incremental features added throughout the years.

At the same time as MFC and WTL were used by C++ developers, another might tool entered the scene: Visual Basic. This environment was super successful for primary two reasons:

• The programming language was based on BASIC, which many people had at least acquaintance with, as it was the most common programming language for personal computers in the 1980s and early 1990s.
• The “Visual” aspect of Visual Basic was new and compelling. Just drag controls from a toolbox onto a surface, change properties in the designer and/or at runtime, connect to events easily, and you’re good to go.

To this day, I sometimes encounter customers and applications still built with Visual Basic 6, even though its official support date is long gone.

The .NET Era

At around 2002, .NET and C# were introduced by Microsoft as a response to the Java language and ecosystem that came out in 1995. With .NET, the Windows Forms (WinForms) library was provided, which was very similar to the Visual Basic experience, but with the more modern and powerful .NET Framework. And with .NET 2 in 2005, where .NET really kicked in (generics and other important features released), Windows Forms was the go-to UI framework while Visual Basic’s popularity somewhat waning.

However, WinForms was still based around the Win32 GUI model – HWNDs, no easy customization, etc. However, Microsoft did a lot of work to make WinForms more customizable than pure Win32 or MFC by subclassing many of the existing controls and adding functionality available with simple properties. Support was added to customize menus with colors and icons, buttons with images and custom colors, and more. The drag-n-drop experience from Visual Basic was available as well, making it relatively easy to migrate from Visual Basic.

.NET 3 and WPF

The true revolution came in 2006 when .NET 3 was released. .NET 3 had 3 new technologies that were greatly advertised:

• Windows Communication Foundation (WCF) – networking and communication framework
• Windows Presentation Foundation (WPF) – UI and graphics framework
• Windows Workflow Foundation (WF) – workflow framework

WCF was hugely successful, and took over older technologies as it unified all types of communications, whether based on remoting, HTTP, sockets, or whatever. WF had only moderate success.

WPF was the new UI framework, and it was revolutionary. WPF ditched the Win32 UI model – a WPF “main” window still had an HWND – you can’t get away with that – but all the controls were drawn by WPF – the Win32 UI controls were not used. From Win32’s perspective there was just one HWND. Compare that to Win32 UI model, where every control is an HWND – buttons, list boxes, list views, toolbars, etc.

With the HWND restrictions gone, WPF used DirectX for rendering purposes, compared to the aging Graphics Device Interface (GDI) API used by Win32 GUIs. Without the artificial boundaries of HWNDs, WPF could do anything – combine anything – 2D, 3D, animation, media, unlimited customization – without any issues, as the entire HWND surface belonged to WPF.

I remember when I was introduced to WPF (at that time code name “Avalon”) – I was blown away. It was a far cry from the old, predictable, non-customizable model of Win32 GUIs.

WPF wasn’t just about the graphics and visuals. It also provided powerful data binding, much more powerful than the limited model supported by WinForms. I would even go so far as say it’s one of the most important of WPF’s features. WPF introduced XAML – an XML based language to declaratively build UIs, with object creation, properties, and even declarative data binding. Customizing controls could be done in several ways, including existing properties, control templates and data templates. WPF was raw power.

So, is WPF the ultimate GUI framework? It certainly looked like a prime candidate.

WPF made progress, ironing out issues, adding some features in .NET 3.5 and .NET 4. But then it seemed to have grinded to a halt. WPF barely made some minor improvements in .NET 4.5. One can say that it was pretty complete, so perhaps nothing much to add?

One aspect of WPF not dealt with well was performance. WPF could be bogged down by many control with complex data bindings – data bindings were mostly implemented with Reflection – a flexible but relatively slow .NET mechanism. There was certainly opportunities for improvement. Additionally, some controls were inherently slow, most notable the DataGrid, which was useful, but problematic as it was painfully slow. Third party libraries came in to the rescue and provided improved Data Grids of their own (most not free).

WPF had a strong following, with community created controls, and other goodies. Microsoft, however, seemed to have lost interest in WPF, the reason perhaps being the “Metro” revolution of 2012.

“Metro” and Going Universal

Windows 8 was a major release for Microsoft where UI is concerned. The “Metro” minimal language was all the rage at the time. Touch devices started to appear and Microsoft did not want to lose the battle. I noticed that Microsoft tends to move from one extreme to another, finally settling somewhere in the middle – but that usually takes years. Windows 8 is a perfect example. Metro applications (as they were called at the time) were always full screen – even on desktops with big displays. A new framework was built, based around the Windows Runtime – a new library based on the old but trusty Component Object Model (COM), with metadata used with the .NET metadata format.

The Windows Runtime UI model was built on similar principles as WPF – XAML (not the same one, mind you; that would be too easy), data binding, control templates, and other similar (but simplified) concepts from WPF. The Windows Runtime was internally built in C++, with “convenient” language projections provided out of the box for C++ (C++/CX at the time), .NET (C# and VB), and even JavaScript.

Generally, Windows 8 and the Universal applications (as they were later renamed) were pretty terrible. The “Metro design language”, with its monochromatic simplistic icons and graphics was ridiculous. Colors were gone. I felt like I’m sliding back to the 1980s when colors were limited. This “Metro” style spread everywhere as far as Microsoft is concerned. For example, Visual Studio 2012 that was out at the time was monochromatic – all icons in black only! It was a nightmare. Microsoft’s explanation was “to focus the developer attention to the code, remove distractions”. In actually, it failed miserably. I remember the control toolbox for WinForms and WPF in VS 2012 – all icons were gray – there was just no way to distinguish between them at a glance – which destroys the point of having icons in the first place. Microsoft boasted that their designers managed to make all these once colorful icons with a single color! What an achievement.

With Visual Studio 2013, they started to bring some colors back… the whole thing was so ridiculous.

The “Universal” model was created at least to address the problem of creating applications with the same code for Windows 8 and Windows Phone 8. To that end, it was successful, as the Win32 GUI was not implemented on Windows Phone, presumably because it was outdated, with lots and lots of code that is not well-suited for a small, much less powerful, form factor like the phone and other small devices.

Working with Universal applications (now called Universal Windows Platform applications) was similar to WPF to some extent, but the controls were geared towards touch devices, where fingers are mostly used. Controls were big, list views were scrolling smoothly but had very few lines of content. For desktop applications, it was a nightmare. Not to mention that Windows 7 (still very popular at the time) was not supported.

WPF was still the best option in the Microsoft space at the time, even though it stagnated. At least it worked on Windows 7, and its default control rendering was suited to desktop applications.

Windows 8.1 made some improvements in Universal apps – at least a minimize button was added! Windows 10 fixed the Universal fiasco by allowing windows to be resized normally like in the “old” days. There was a joke at the time saying that “Windows 10 returned windows to Windows. Before that it was Window – singular”.

That being said, Windows 10’s own UI was heavily influenced by Metro. The settings up use monochrome icons – how can anyone think this is better than colorful icons for easy recognition. This trend continues with Windows 11 where various classic windows are “converted” to the new “design language”. At least the settings app uses somewhat colorful icons on Windows 11.

The Universal apps could only run with a single instance, something that has since changed, but still employed. For example, the settings app in Windows 10 and 11 is single instance. Why on earth should it be in an OS named “Windows”? Give me more than one Settings window at a time!

Current State of Affairs

WPF is not moving forward. With the introduction of .NET Core (later renamed to simply .NET), WPF was open sourced, and is available in .NET 5+. It’s not cross platform, as most of the other .NET 5+ pieces.

UWP is a failure, even Microsoft admits that. It’s written in C++ (it’s based on the Windows Runtime after all), which should give it good performance not bogged down by .NET’s garbage collector and such. But its projections for C++ is awful, and in my opinion unusable. If you create a new UWP application with C++ in Visual Studio, you’ll get plenty of files, including IDL (Interface Definition Language), some generated files, and all that for a single button in a window. I tried writing something more complex, and gave up. It’s too slow and convoluted. The only real option is to use .NET – something I may not want to do with all its dependencies and overhead.

Regardless, the controls default look and feel is geared towards touch devices. I don’t care about the little animations – I want to be able to use a proper list view. For example, the Windows 11 new Task Manager that is built with the new WinUI technology (described next) uses the Win32 classic list view – because it’s fast and appropriate for this kind of tool. The rest is WinUI – the tabs are gone, there are monochromatic icons – it’s just ridiculous. The WinUI adds nothing except a dark theme option.

Task manager in Windows 11

The WinUI technology is similar to UWP in concept and implementation. The current state of UI affairs is messy – there is WinUI, UWP, .NET Maui (to replace Xamarin for mobile devices but not just) – what are people supposed to use?

All these UI libraries don’t really cater for desktop apps. This is why I’m still using WTL (which is wrapping the Win32 classic GUI API). There is no good alternative from Microsoft.

But perhaps not all is lost – Avalonia is a fairly new library attempting to bring WPF style UI and capabilities to more than just Windows. But it’s not a Microsoft library, but built by people in the community as open source – there is no telling if at some point it will stop being supported. On the other hand, WPF – a Microsoft library – stopped being supported.

Other Libraries

At this point you may be wondering why use a Microsoft library at all for desktop GUI – Microsoft has dropped the ball, as they continue to make a mess. Maybe use Blazor on the desktop? Out of scope for this post.

There are other options. many GUI libraries that use C or C++ exist – wxWidgets, GTK, and Qt, to name a few. wxWidgets supports Windows fairly well. Installing GTK successfully is a nightmare. Qt is very powerful and takes control of drawing everything, similar to the WPF model. It has powerful tools for designing GUIs, with its own declarative language based on JavaScript. With Qt you also have to use its own classes for non-UI stuff, like strings and lists. It’s also pricey for closed source.

Another alternative which has a lot of promise (some of which is already delivered) is Dear ImGui. This library is different from most others, as it’s Immediate Mode GUI, rather than Retained Mode which most other are. It’s cross platform, very flexible and fast. Just look at some of the GUIs built with it – truly impressive.

I’ll probably migrate to using ImGui. Is it the ultimate GUI framework? Not yet, but I feel it’s the closest to attain that goal. A couple of years back I implemented a mini-Process Explorer like tool with ImGui. Its list view is flexible and rich, and the library in general gets better all the time. It has great support from the authors and the community. It’s not perfect yet, there are still rough edges, and in some cases you have to work harder because of its cross-platform nature.

I should also mention Uno Platform, another cross-platform UI framework built on top of .NET, that made great strides in recent years.

What’s Next?

Microsoft has dropped the ball on desktop apps. The Win32 classic model is not being maintained. Just try to create a “dark mode” UI. I did that to some extent for the Sysinternals tools at the time. It was hard. Some things I just couldn’t do right – the scrollbars that are attached to list views and tree views, for example.

Prior to common controls version 6 (Vista), Microsoft had a “flat scroll bars” feature that allowed customization of scrollbars fairly easily (colors, for example). But surprisingly, common controls version 6 dropped this feature! Flat scroll bars are no longer supported. I had to go through hoops to implement dark scroll bars for Sysinternals – and even that was imperfect.

In my own tools, I created a theme engine as well – implemented differently – and I decided to forgo customizing scroll bars. Let them remain as is – it’s just too difficult and fragile.

I do hope Microsoft changes something in the way they look at desktop apps. This is where most Windows users are! Give us WPF in C++. Or enhance the Win32 model. The current UI mess is not helping, either.

I’m going to set some time to work on building some tools that use Dear ImGui – I feel it has the most bang for the buck.

You may have been asked this question many times: “How much memory does this process consume?” The question seems innocent enough. Your first instinct might be to open Task Manager, go to the Processes tab, find the process in the list, and look at the column marked “Memory“. What could be simpler?

A complication is hinted at when looking in the Details tab. The default memory-related column is named “Memory (Active Private Working Set)”, which seems more complex than simply “Memory”. Opening the list of columns from the Details tab shows more columns where the term “Memory” is used. What gives?

The Processes’ tab Memory column is the same as the Details’ tab Memory (active private working set). But what does it mean? Let’s break it down:

• Working set – the memory is accessible by the processor with no page fault exception. Simply put, the memory is in RAM (physical memory).
• Private – the memory is private to the process. This is in contrast to shared memory, which is (at least can be) shared with other processes. The canonical example of shared memory is PE images – DLLs and executables. A DLL that is mapped to multiple processes will (in most cases) have a single presence in physical memory.
• Active – this is an artificial term used by Task Manager related to UWP (Universal Windows Platform) processes. If a UWP process’ window is minimized, this column shows zero memory consumption, because in theory, since all the process’ threads are suspended, that memory can be repurposed for other processes to use. You can try it by running Calculator, and minimizing its window. You’ll see this column showing zero. Restore the window, and it will show some non-zero value. In fact, there is a column named Memory (private working set), which shows the same thing but does not take into consideration the “active” aspect of UWP processes.

So what does all this mean? The fact that this column shows only private memory is a good thing. That’s because the shared memory size (in most cases) is not controllable and is fixed – for example, the size of a DLL – it’s out of our control – the process just needs to use the DLL. The downside of this active private working set column is that fact it only shows memory current part of the process working set – in RAM. A process may allocate a large junk of memory, but most of it may not be in RAM right now, but it is still consumed, and counts towards the commit limit of the system.

Here is a simple example. I’m writing the following code to allocate (commit) 64 GM of memory:

auto ptr = VirtualAlloc(nullptr, 64LL << 30, MEM_COMMIT | MEM_RESERVE, PAGE_READWRITE);

Here is what Task manager shows in its Performance/Memory tab before the call:

“In Use” indicates current RAM (physical memory) usage – it’s 34.6 GB. The “Committed” part is more important – it indicates how much memory I can totally commit on the system, regardless of whether it’s in physical memory now or not. It shows “44/128 GB” – 44 GB are committed now (34.6 of that in RAM), and my commit limit is 128 GB (it’s the sum of my total RAM and the configured page files sizes). Here is the same view after I commit the above 64 GB:

Notice the physical memory didn’t change much, but the committed memory “jumped” by 64 GB, meaning there is now only 20 GB left for other processes to use before the system runs out of memory (or page file expansion occurs). Looking at the Details that for this Test process shows the active private working set column indicating a very low memory consumption because it’s looking at private RAM usage only:

Only when the process starts “touching” (using) the committed memory, physical pages will start being used by the process. The name “committed” indicates the commitment of the system to providing that entire memory block if required no matter what.

Where is that 64 GB shown? The column to use is called in Task Manager Commit Size, which is in fact private committed memory:

Commit Size is the correct column to look at when trying to ascertain memory consumption in processes. The sad thing is that it’s not the default column shown, and that’s why many people use the misleading active private working set column. My guess is the reason the misleading column is shown by default is because physical memory is easy to understand for most people, whereas virtual memory – (some of which is in RAM and some which is not) is not trivially understood.

Compare Commit Size to active private working set sometimes reveals a big difference – an indication that most of the private memory of a process is not in RAM right now, but the memory is still consumed as far as the memory manager is concerned.

A related confusion exists because of different terminology used by different tools. Specifically, Commit Size in Task Manager is called Private Bytes in Process Explorer and Performance Monitor.

Task Manager’s other memory columns allow you to look at more memory counters such as Working Set (total RAM used by a process, including private and shared memory), Peak Working Set, Memory (shared working set), and Working Set Delta.

There are other subtleties I am not expanding on in this post. Hopefully, I’ll touch on these in a future post.

Bottom line: Commit Size is the way to go.

Here is a simple experiment to try: open Visual Studio and create a C++ console application. All that app is doing is display “hello world” to the console:

#include <stdio.h>

int main() {
	printf("Hello, world!\n");
	return 0;
}

Build the executable in Release build and check its size. I get 11KB (x64). Not too bad, perhaps. However, if we check the dependencies of this executable (using the dumpbin command line tool or any PE Viewer), we’ll find the following in the Import directory:

There are two dependencies: Kernel32.dll and VCRuntime140.dll. This means these DLLs will load at process start time no matter what. If any of these DLLs is not found, the process will crash. We can’t get rid of Kernel32 easily, but we may be able to link statically to the CRT. Here is the required change to VS project properties:

After building, the resulting executable jumps to 136KB in size! Remember, it’s a “hello, world” application. The Imports directory in a PE viewer now show Kernel32.dll as the only dependency.

Is that best we can do? Why do we need the CRT in the first place? One obvious reason is the usage of the printf function, which is implemented by the CRT. Maybe we can use something else without depending on the CRT. There are other reasons the CRT is needed. Here are a few:

• The CRT is the one calling our main function with the correct argc and argv. This is expected behavior by developers.
• Any C++ global objects that have constructors are executed by the CRT before the main function is invoked.
• Other expected behaviors are provided by the CRT, such as correct handling of the errno (global) variable, which is not really global, but uses Thread-Local-Storage behind the scenes to make it per-thread.
• The CRT implements the new and delete C++ operators, without which much of the C++ standard library wouldn’t work without major customization.

Still, we may be OK doing things outside the CRT, taking care of ourselves. Let’s see if we can pull it off. Let’s tell the linker that we’re not interested in the CRT:

Setting “Ignore All Default Libraries” tells the linker we’re not interested in linking with the CRT in any way. Building the app now gives some linker errors:

1>Test2.obj : error LNK2001: unresolved external symbol __security_check_cookie
1>Test2.obj : error LNK2001: unresolved external symbol __imp___acrt_iob_func
1>Test2.obj : error LNK2001: unresolved external symbol __imp___stdio_common_vfprintf
1>LINK : error LNK2001: unresolved external symbol mainCRTStartup
1>D:\Dev\Minimal\x64\Release\Test2.exe : fatal error LNK1120: 4 unresolved externals

One thing we expected is the missing printf implementation. What about the other errors? We have the missing “security cookie” implementation, which is a feature of the CRT to try to detect stack overrun by placing a “cookie” – some number – before making certain function calls and making sure that cookie is still there after returning. We’ll have to settle without this feature. The main missing piece is mainCRTStartup, which is the default entry point that the linker is expecting. We can change the name, or overwrite main to have that name.

First, let’s try to fix the linker errors before reimplementing the printf functionality. We’ll remove the printf call and rebuild. Things are improving:

>Test2.obj : error LNK2001: unresolved external symbol __security_check_cookie
1>LINK : error LNK2001: unresolved external symbol mainCRTStartup
1>D:\Dev\Minimal\x64\Release\Test2.exe : fatal error LNK1120: 2 unresolved externals

The “security cookie” feature can be removed with another compiler option:

When rebuilding, we get a warning about the “/sdl” (Security Developer Lifecycle) option conflicting with removing the security cookie, which we can remove as well. Regardless, the final linker error remains – mainCRTStartup.

We can rename main to mainCRTStartup and “implement” printf by going straight to the console API (part of Kernel32.Dll):

#include <Windows.h>

int mainCRTStartup() {
	char text[] = "Hello, World!\n";
	::WriteConsoleA(::GetStdHandle(STD_OUTPUT_HANDLE),
		text, (DWORD)strlen(text), nullptr, nullptr);

	return 0;
}

This compiles and links ok, and we get the expected output. The file size is only 4KB! An improvement even over the initial project. The dependencies are still just Kernel32.DLL, with the only two functions used:

You may be thinking that although we replaced printf, that’s wasn’t the full power of printf – it supports various format specifiers, etc., which are going to be difficult to reimplement. Is this just a futile exercise?

Not necessarily. Remember that every user mode process always links with NTDLL.dll, which means the API in NtDll is always available. As it turns out, a lot of functionality that is implemented by the CRT is also implemented in NTDLL. printf is not there, but the next best thing is – sprintf and the other similar formatting functions. They would fill a buffer with the result, and then we could call WriteConsole to spit it to the console. Problem solved!

Removing the CRT

Well, almost. Let’s add a definition for sprintf_s (we’ll be nice and go with the “safe” version), and then use it:

#include <Windows.h>

extern "C" int __cdecl sprintf_s(
	char* buffer,
	size_t sizeOfBuffer,
	const char* format,	...);

int mainCRTStartup() {
	char text[64];
	sprintf_s(text, _countof(text), "Hello, world from process %u\n", ::GetCurrentProcessId());
	::WriteConsoleA(::GetStdHandle(STD_OUTPUT_HANDLE),
		text, (DWORD)strlen(text), nullptr, nullptr);

	return 0;
}

Unfortunately, this does not link: sprintf_s is an unresolved external, just like strlen. It makes sense, since the linker does not know where to look for it. Let’s help out by adding the import library for NtDll:

#pragma comment(lib, "ntdll")

This should work, but one error persists – sprintf_s; strlen however, is resolved. The reason is that the import library for NtDll provided by Microsoft does not have an import entry for sprintf_s and other CRT-like functions. Why? No good reason I can think of. What can we do? One option is to create an NtDll.lib import library of our own and use it. In fact, some people have already done that. One such file can be found as part of my NativeApps repository (it’s called NtDll64.lib, as the name does not really matter). The other option is to link dynamically. Let’s do that:

int __cdecl sprintf_s_f(
	char* buffer, size_t sizeOfBuffer, const char* format, ...);

int mainCRTStartup() {
	auto sprintf_s = (decltype(sprintf_s_f)*)::GetProcAddress(
        ::GetModuleHandle(L"ntdll"), "sprintf_s");
	if (sprintf_s) {
		char text[64];
		sprintf_s(text, _countof(text), "Hello, world from process %u\n", ::GetCurrentProcessId());
		::WriteConsoleA(::GetStdHandle(STD_OUTPUT_HANDLE),
			text, (DWORD)strlen(text), nullptr, nullptr);
	}

	return 0;
}

Now it works and runs as expected.

You may be wondering why does NTDLL implement the CRT-like functions in the first place? The CRT exists, after all, and can be normally used. “Normally” is the operative word here. Native applications, those that can only depend on NTDLL cannot use the CRT. And this is why these functions are implemented as part of NTDLL – to make it easier to build native applications. Normally, native applications are built by Microsoft only. Examples include Smss.exe (the session manager), CSrss.exe (the Windows subsystem process), and UserInit.exe (normally executed by WinLogon.exe on a successful login).

One thing that may be missing in our “main” function are command line arguments. Can we just add the classic argc and argv and go about our business? Let’s try:

int mainCRTStartup(int argc, const char* argv[]) {
//...
char text[64];
sprintf_s(text, _countof(text), 
    "argc: %d argv[0]: 0x%p\n", argc, argv[0]);
::WriteConsoleA(::GetStdHandle(STD_OUTPUT_HANDLE),
	text, (DWORD)strlen(text), nullptr, nullptr);

Seems simple enough. argv[0] should be the address of the executable path itself. The code carefully displays the address only, not trying to dereference it as a string. The result, however, is perplexing:

argc: -359940096 argv[0]: 0x74894808245C8948

This seems completely wrong. The reason we see these weird values (if you try it, you’ll get different values. In fact, you may get different values in every run!) is that the expected parameters by a true entry point of an executable is not based on argc and argv – this is part of the CRT magic. We don’t have a CRT anymore. There is in fact just one argument, and it’s the Process Environment Block (PEB). We can add some code to show some of what is in there (non-relevant code omitted):

#include <Windows.h>
#include <winternl.h>
//...
int mainCRTStartup(PPEB peb) {
	char text[256];
	sprintf_s(text, _countof(text), "PEB: 0x%p\n", peb);
	::WriteConsoleA(::GetStdHandle(STD_OUTPUT_HANDLE),
		text, (DWORD)strlen(text), nullptr, nullptr);

	sprintf_s(text, _countof(text), "Executable: %wZ\n", 
        peb->ProcessParameters->ImagePathName);
	::WriteConsoleA(::GetStdHandle(STD_OUTPUT_HANDLE),
		text, (DWORD)strlen(text), nullptr, nullptr);

	sprintf_s(text, _countof(text), "Commandline: %wZ\n", 
        peb->ProcessParameters->CommandLine);
	::WriteConsoleA(::GetStdHandle(STD_OUTPUT_HANDLE),
		text, (DWORD)strlen(text), nullptr, nullptr);

<Winternl.h> contains some NTDLL definitions, such as a partially defined PEB. In it, there is a ProcessParameters member that holds the image path and the full command line. Here is the result on my console:

PEB: 0x000000EAC01DB000
Executable: D:\Dev\Minimal\x64\Release\Test3.exe
Commandline: "D:\Dev\Minimal\x64\Release\Test3.exe"

The PEB is the argument provided by the OS to the entry point, whatever its name is. This is exactly what native applications get as well. By the way, we could have used GetCommandLine from Kernel32.dll to get the command line if we didn’t add the PEB argument. But for native applications (that can only depend on NTDLL), GetCommandLine is not an option.

Going Native

How far are we from a true native application? What would be the motivation for such an application anyway, besides small file size and reduced dependencies? Let’s start with the first question.

To make our executable truly native, we have to do two things. The first is to change the subsystem of the executable (stored in the PE header) to Native. VS provides this option via a linker setting:

The second thing is to remove the dependency on Kernel32.Dll. No more WriteConsole and no GetCurrentProcessId. We will have to find some equivalent in NTDLL, or write our own implementation leveraging what NtDll has to offer. This is obviously not easy, given that most of NTDLL is undocumented, but most function prototypes are available as part of the Process Hacker/phnt project.

For the second question – why bother? Well, one reason is that native applications can be configured to run very early in Windows boot – these in fact run by Smss.exe itself when it’s the only existing user-mode process at that time. Such applications (like autochk.exe, a native chkdsk.exe) must be native – they cannot depend on the CRT or even on kernel32.dll, since the Windows Subsystem Process (csrss.exe) has not been launched yet.

For more information on Native Applications, you can view my talk on the subject.

I may write a blog post on native application to give more details. The examples shown here can be found here.

Happy minimization!

Doing any kind of research into the Windows kernel requires working with a kernel debugger, mostly WinDbg (or WinDbg Preview). There are at least 3 “levels” of debugging the kernel.

Level 1: Local Kernel Debugging

The first is using a local kernel debugger, which means configuring WinDbg to look at the kernel of the local machine. This can be configured by running the following command in an elevated command window, and restarting the system:

bcdedit -debug on

You must disable Secure Boot (if enabled) for this command to work, as Secure Boot protects against putting the machine in local kernel debugging mode. Once the system is restarted, WinDbg launched elevated, select File/Kernel Debug and go with the “Local” option (WinDbg Preview shown):

If all goes well, you’ll see the “lkd>” prompt appearing, confirming you’re in local kernel debugging mode.

What can you in this mode? You can look at anything in kernel and user space, such as listing the currently existing processes (!process 0 0), or examining any memory location in kernel or user space. You can even change kernel memory if you so desire, but be careful, any “bad” change may crash your system.

The downside of local kernel debugging is that the system is a moving target, things change while you’re typing commands, so you don’t want to look at things that change quickly. Additionally, you cannot set any breakpoint; you cannot view any CPU registers, since these are changing constantly, and are on a CPU-basis anyway.

The upside of local kernel debugging is convenience – setting it up is very easy, and you can still get a lot of information with this mode.

Level 2: Remote Debugging of a Virtual Machine

The next level is a full kernel debugging experience of a virtual machine, which can be running locally on your host machine, or perhaps on another host somewhere. Setting this up is more involved. First, the target VM must be set up to allow kernel debugging and set the “interface” to the host debugger. Windows supports several interfaces, but for a VM the best to use is network (supported on Windows 8 and later).

First, go to the VM and ping the host to find out its IP address. Then type the following:

bcdedit /dbgsettings net hostip:172.17.32.1 port:55000 key:1.2.3.4

Replace the host IP with the correct address, and select an unused port on the host. The key can be left out, in which case the command will generate something for you. Since that key is needed on the host side, it’s easier to select something simple. If the target VM is not local, you might prefer to let the command generate a random key and use that.

Next, launch WinDbg elevated on the host, and attach to the kernel using the “Net” option, specifying the correct port and key:

Restart the target, and it should connect early in its boot process:

Microsoft (R) Windows Debugger Version 10.0.25200.1003 AMD64
Copyright (c) Microsoft Corporation. All rights reserved.

Using NET for debugging
Opened WinSock 2.0
Waiting to reconnect...
Connected to target 172.29.184.23 on port 55000 on local IP 172.29.176.1.
You can get the target MAC address by running .kdtargetmac command.
Connected to Windows 10 25309 x64 target at (Tue Mar  7 11:38:18.626 2023 (UTC - 5:00)), ptr64 TRUE
Kernel Debugger connection established.  (Initial Breakpoint requested)

************* Path validation summary **************
Response                         Time (ms)     Location
Deferred                                       SRV*d:\Symbols*https://msdl.microsoft.com/download/symbols
Symbol search path is: SRV*d:\Symbols*https://msdl.microsoft.com/download/symbols
Executable search path is: 
Windows 10 Kernel Version 25309 MP (1 procs) Free x64
Edition build lab: 25309.1000.amd64fre.rs_prerelease.230224-1334
Machine Name:
Kernel base = 0xfffff801`38600000 PsLoadedModuleList = 0xfffff801`39413d70
System Uptime: 0 days 0:00:00.382
nt!DebugService2+0x5:
fffff801`38a18655 cc              int     3

Enter the g command to let the system continue. The prompt is “kd>” with the current CPU number on the left. You can break at any point into the target by clicking the “Break” toolbar button in the debugger. Then you can set up breakpoints, for whatever you’re researching. For example:

1: kd> bp nt!ntWriteFile
1: kd> g
Breakpoint 0 hit
nt!NtWriteFile:
fffff801`38dccf60 4c8bdc          mov     r11,rsp
2: kd> k
 # Child-SP          RetAddr               Call Site
00 fffffa03`baa17428 fffff801`38a81b05     nt!NtWriteFile
01 fffffa03`baa17430 00007ff9`1184f994     nt!KiSystemServiceCopyEnd+0x25
02 00000095`c2a7f668 00007ff9`0ec89268     0x00007ff9`1184f994
03 00000095`c2a7f670 0000024b`ffffffff     0x00007ff9`0ec89268
04 00000095`c2a7f678 00000095`c2a7f680     0x0000024b`ffffffff
05 00000095`c2a7f680 0000024b`00000001     0x00000095`c2a7f680
06 00000095`c2a7f688 00000000`000001a8     0x0000024b`00000001
07 00000095`c2a7f690 00000095`c2a7f738     0x1a8
08 00000095`c2a7f698 0000024b`af215dc0     0x00000095`c2a7f738
09 00000095`c2a7f6a0 0000024b`0000002c     0x0000024b`af215dc0
0a 00000095`c2a7f6a8 00000095`c2a7f700     0x0000024b`0000002c
0b 00000095`c2a7f6b0 00000000`00000000     0x00000095`c2a7f700
2: kd> .reload /user
Loading User Symbols
.....................
2: kd> k
 # Child-SP          RetAddr               Call Site
00 fffffa03`baa17428 fffff801`38a81b05     nt!NtWriteFile
01 fffffa03`baa17430 00007ff9`1184f994     nt!KiSystemServiceCopyEnd+0x25
02 00000095`c2a7f668 00007ff9`0ec89268     ntdll!NtWriteFile+0x14
03 00000095`c2a7f670 00007ff9`08458dda     KERNELBASE!WriteFile+0x108
04 00000095`c2a7f6e0 00007ff9`084591e6     icsvc!ICTransport::PerformIoOperation+0x13e
05 00000095`c2a7f7b0 00007ff9`08457848     icsvc!ICTransport::Write+0x26
06 00000095`c2a7f800 00007ff9`08452ea3     icsvc!ICEndpoint::MsgTransactRespond+0x1f8
07 00000095`c2a7f8b0 00007ff9`08452abc     icsvc!ICTimeSyncReferenceMsgHandler+0x3cb
08 00000095`c2a7faf0 00007ff9`084572cf     icsvc!ICTimeSyncMsgHandler+0x3c
09 00000095`c2a7fb20 00007ff9`08457044     icsvc!ICEndpoint::HandleMsg+0x11b
0a 00000095`c2a7fbb0 00007ff9`084574c1     icsvc!ICEndpoint::DispatchBuffer+0x174
0b 00000095`c2a7fc60 00007ff9`08457149     icsvc!ICEndpoint::MsgDispatch+0x91
0c 00000095`c2a7fcd0 00007ff9`0f0344eb     icsvc!ICEndpoint::DispatchThreadFunc+0x9
0d 00000095`c2a7fd00 00007ff9`0f54292d     ucrtbase!thread_start<unsigned int (__cdecl*)(void *),1>+0x3b
0e 00000095`c2a7fd30 00007ff9`117fef48     KERNEL32!BaseThreadInitThunk+0x1d
0f 00000095`c2a7fd60 00000000`00000000     ntdll!RtlUserThreadStart+0x28
2: kd> !process -1 0
PROCESS ffffc706a12df080
    SessionId: 0  Cid: 0828    Peb: 95c27a1000  ParentCid: 044c
    DirBase: 1c57f1000  ObjectTable: ffffa50dfb92c880  HandleCount: 123.
    Image: svchost.exe

In this “level” of debugging you have full control of the system. When in a breakpoint, nothing is moving. You can view register values, call stacks, etc., without anything changing “under your feet”. This seems perfect, so do we really need another level?

Some aspects of a typical kernel might not show up when debugging a VM. For example, looking at the list of interrupt service routines (ISRs) with the !idt command on my Hyper-V VM shows something like the following (truncated):

2: kd> !idt

Dumping IDT: ffffdd8179e5f000

00:	fffff80138a79800 nt!KiDivideErrorFault
01:	fffff80138a79b40 nt!KiDebugTrapOrFault	Stack = 0xFFFFDD8179E95000
02:	fffff80138a7a140 nt!KiNmiInterrupt	Stack = 0xFFFFDD8179E8D000
03:	fffff80138a7a6c0 nt!KiBreakpointTrap
...
2e:	fffff80138a80e40 nt!KiSystemService
2f:	fffff80138a75750 nt!KiDpcInterrupt
30:	fffff80138a733c0 nt!KiHvInterrupt
31:	fffff80138a73720 nt!KiVmbusInterrupt0
32:	fffff80138a73a80 nt!KiVmbusInterrupt1
33:	fffff80138a73de0 nt!KiVmbusInterrupt2
34:	fffff80138a74140 nt!KiVmbusInterrupt3
35:	fffff80138a71d88 nt!HalpInterruptCmciService (KINTERRUPT ffffc70697f23900)

36:	fffff80138a71d90 nt!HalpInterruptCmciService (KINTERRUPT ffffc70697f23a20)

b0:	fffff80138a72160 ACPI!ACPIInterruptServiceRoutine (KINTERRUPT ffffdd817a1ecdc0)
...

Some things are missing, such as the keyboard interrupt handler. This is due to certain things handled “internally” as the VM is “enlightened”, meaning it “knows” it’s a VM. Normally, it’s a good thing – you get nice support for copy/paste between the VM and the host, seamless mouse and keyboard interaction, etc. But it does mean it’s not the same as another physical machine.

Level 3: Remote debugging of a physical machine

In this final level, you’re debugging a physical machine, which provides the most “authentic” experience. Setting this up is the trickiest. Full description of how to set it up is described in the debugger documentation. In general, it’s similar to the previous case, but network debugging might not work for you depending on the network card type your target and host machines have.

If network debugging is not supported because of the limited list of network cards supported, your best bet is USB debugging using a dedicated USB cable that you must purchase. The instructions to set up USB debugging are provided in the docs, but it may require some trial and error to locate the USB ports that support debugging (not all do). Once you have that set up, you’ll use the “USB” tab in the kernel attachment dialog on the host. Once connected, you can set breakpoints in ISRs that may not exist on a VM:

: kd> !idt

Dumping IDT: fffff8022f5b1000

00:	fffff80233236100 nt!KiDivideErrorFault
...
80:	fffff8023322cd70 i8042prt!I8042KeyboardInterruptService (KINTERRUPT ffffd102109c0500)
...
Dumping Secondary IDT: ffffe5815fa0e000 

01b0:hidi2c!OnInterruptIsr (KMDF) (KINTERRUPT ffffd10212e6edc0)

0: kd> bp i8042prt!I8042KeyboardInterruptService
0: kd> g
Breakpoint 0 hit
i8042prt!I8042KeyboardInterruptService:
fffff802`6dd42100 4889542410      mov     qword ptr [rsp+10h],rdx
0: kd> k
 # Child-SP          RetAddr               Call Site
00 fffff802`2f5cdf48 fffff802`331453cb     i8042prt!I8042KeyboardInterruptService
01 fffff802`2f5cdf50 fffff802`3322b25f     nt!KiCallInterruptServiceRoutine+0x16b
02 fffff802`2f5cdf90 fffff802`3322b527     nt!KiInterruptSubDispatch+0x11f
03 fffff802`2f5be9f0 fffff802`3322e13a     nt!KiInterruptDispatch+0x37
04 fffff802`2f5beb80 00000000`00000000     nt!KiIdleLoop+0x5a

Happy debugging!

I’ve recently posted about the upcoming training classes, the first of which is Advanced Windows Kernel Programming in April. Some people have asked me how can they participate if they have not taken the Windows Kernel Programming fundamentals class, and they might not have the required time to read the book.

Since I don’t plan on providing the fundamentals training class before April, after some thought, I decided to do the following.

I am selling one of the previous Windows Kernel Programming class recordings, along with the course PDF materials, the labs, and solutions to the labs. This is the first time I’m selling recordings of my public classes. If this “experiment” goes well, I might consider doing this with other classes as well. Having recordings is not the same as doing a live training class, but it’s the next best thing if the knowledge provided is valuable and useful. It’s about 32 hours of video, and plenty of labs to keep you busy

As an added bonus, I am also giving the following to those purchasing the training class:

• You get 10% discount for the Advanced Windows Kernel Programming class in April.
• You will be added to a discord server that will host all the Alumni from my public classes (an idea I was given by some of my students which will happen soon)
• A live session with me sometime in early April (I’ll do a couple in different times of day so all time zones can find a comfortable session) where you can ask questions about the class, etc.

These are the modules covered in the class recordings:

• Module 0: Introduction
• Module 1: Windows Internals Overview
• Module 2: The I/O System
• Module 3: Device Driver Basics
• Module 4: The I/O Request Packet
• Module 5: Kernel Mechanisms
• Module 6: Process and Thread Monitoring
• Module 7: Object and Registry Notifications
• Module 8: File System Mini-Filters Fundamentals
• Module 9: Miscellaneous Techniques

If you’re interested in purchasing the class, send me an email to [email protected] with the title “Kernel Programming class recordings” and I will reply with payment details. Once paid, reply with the payment information, and I will share a link with the course. I’m working on splitting the recordings into meaningful chunks, so not all are ready yet, but these will be completed in the next day or so.

Here are the rules after a purchase:

• No refunds – once you have access to the recordings, this is it.
• No sharing – the content is for your own personal viewing. No sharing of any kind is allowed.
• No reselling – I own the copyright and all rights.

The cost is 490 USD for the entire class. That’s the whole 32 hours.

If you’re part of a company (or simply have friends) that would like to purchase multiple “licenses”, contact me for a discount.