Some people asked me if I had a schedule of trainings I plan to do in the coming months. Well, here it is. At this point I would like to gauge interest, and plan the course hours (time zone) to accommodate the majority of participants.
I am moving to classes that are partly full days and partly half days. The full day sessions are going to be recorded for the participants (so that if anyone misses something because of urgent work, inconvenient time zone, etc., the recording should help). The half days are not recorded, and should be easier to handle, since they are only about 4 hours long.
Here is the planned course list with dates (f=full day, all others are half-days). The cost is in USD (paid by individual / paid by a company):
COM Programming with C++ (3 days): April 25, 26, 27, 28, May 2, 3 (Cost: 700/1300)
Windows System Programming (5 days): May 16 (f), 17, 18, 19, 23 (f), 24, 25, 26 (Cost: 800/1500)
Windows Kernel Programming (4 days): June 6 (f), 8, 9, 13 (f), 14 (Cost: 800/1500)
Windows Internals (5 days): July 11 (f), 12, 13, 14, 18 (f), 19, 20, 21 (Cost: 800/1500)
“Advanced Kernel Programming” is a new class I’m planning, suitable for those who participated in “Windows Kernel Programming” (or have equivalent knowledge). This course will cover file system mini-filters, NDIS filters, and the Windows Filtering Platform (WFP), along with other advanced programming techniques.
I may add more classes after September, but it’s too far from now to make such a commitment.
If you are interested in one or more of these classes, please write an email to [email protected], and provide your name, preferred contact email, and your time zone. It’s not a commitment on your part, you may change your mind later on, but it should be genuine, where the dates and topics work for you.
Also, if you have other classes you would be happy if I deliver, you are welcome to suggest them. No promises, of course, but if there is enough interest, I will consider creating and delivering them.
If you’d like a private class for your team, get in touch. Syllabi can be customized as needed.
One of the exercises I gave at the recent COM Programming class was to build an Icon Handler that integrates with the Windows Shell, where DLLs should have an icon based on their “bitness” – whether they’re 64-bit or 32-bit Portable Executable (PE).
The Shell provides many opportunities for extensibility. An Icon Handler is one of the simplest, but still requires writing a full-fledged COM component that implements certain interfaces that the shell expects. Here is the result of using the Icon Handler DLL, showing the folders c:\Windows\System32 and c:\Windows\SysWow64 (large icons for easier visibility).
The first step is to create a new ATL project in Visual Studio. I’ll be using Visual Studio 2022, but any recent version would work essentially the same way (e.g. VS 2019, or 2017). Locate the ATL project type by searching in the terrible new project dialog introduced in VS 2019 and still horrible in VS 2022.
ATL (Active Template Library) is certainly not the only way to build COM components. “Pure” C++ would work as well, but ATL provides all the COM boilerplate such as the required exported functions, class factories, IUnknown implementations, etc. Since ATL is fairly “old”, it lacks the elegance of other libraries such as WRL and WinRT, as it doesn’t take advantage of C++11 and later features. Still, ATL has withstood the test of time, is robust, and full featured when it comes to COM, something I can’t say for these other alternatives.
If you can’t locate the ATL project, you may not have ATL installed propertly. Make sure the C++ Desktop development workload is installed using the Visual Studio Installer.
Click Next and select a project name and location:
Click Create to launch the ATL project wizard. Leave all defaults (Dynamic Link Library) and click OK. Shell extensions of all kinds must be DLLs, as these are loaded by Explorer.exe. It’s not ideal in terms of Explorer’s stability, as aun unhandled exception can bring down the entire process, but this is necessary to get good performance, as no inter-process calls are made.
Two projects are created, named DllIconHandler and DllIconHandlerPS. The latter is a proxy/stub DLL that maybe useful if cross-apartment COM calls are made. This is not needed for shell extensions, so the PS project should simply be removed from the solution.
A detailed discussion of COM is way beyond the scope of this post.
The remaining project contains the COM DLL required code, such as the mandatory exported function, DllGetClassObject, and the other optional but recommended exports (DllRegisterServer, DllUnregisterServer, DllCanUnloadNow and DllInstall). This is one of the nice benefits of working with ATL project for COM component development: all the COM boilerplate is implemented by ATL.
The next step is to add a COM class that will implement our icon handler. Again, we’ll turn to a wizard provided by Visual Studio that provides the fundamentals. Right-click the project and select Add Item… (don’t select Add Class as it’s not good enough). Select the ATL node on the left and ATL Simple Object on the right. Set the name to something like IconHandler:
Click Add. The ATL New Object wizard opens up. The name typed in the Add New Item dialog is used as a basis for generating names for source code elements (like the C++ class) and COM elements (that would be written into the IDL file and the resulting type library). Since we’re not going to define a new interface (we need to implement explorer-defined interfaces), there is no real need to tweak anything. You can click Finish to generate the class.
Three files are added with this last step: IconHandler.h, IconHandler.cpp and IconHandler.rgs. The C++ source files role is obvious – implementing the Icon Handler. The rgs file contains a script in an ATL-provided “language” indicating what information to write to the Registry when this DLL is registered (and what to remove if it’s unregistered).
The IDL (Interface Definition Language) file has also been modified, adding the definitions of the wizard generated interface (which we don’t need) and the coclass. We’ll leave the IDL alone, as we do need it to generate the type library of our component because the ATL registration code uses it internally.
If you look in IconHandler.h, you’ll see that the class implements the IIconHandler empty interface generated by the wizard that we don’t need. It even derives from IDispatch:
class ATL_NO_VTABLE CIconHandler :
public CComObjectRootEx<CComSingleThreadModel>,
public CComCoClass<CIconHandler, &CLSID_IconHandler>,
public IDispatchImpl<IIconHandler, &IID_IIconHandler, &LIBID_DLLIconHandlerLib, /*wMajor =*/ 1, /*wMinor =*/ 0> {
We can leave the IDispatchImpl-inheritance, since it’s harmless. But it’s useless as well, so let’s delete it and also delete the interfaces IIconHandler and IDispatch from the interface map located further down:
class ATL_NO_VTABLE CIconHandler :
public CComObjectRootEx<CComSingleThreadModel>,
public CComCoClass<CIconHandler, &CLSID_IconHandler> {
public:
BEGIN_COM_MAP(CIconHandler)
END_COM_MAP()
(I have rearranged the code a bit). Now we need to add the interfaces we truly have to implement for an icon handler: IPersistFileand IExtractIcon. To get their definitions, we’ll add an #include for <shlobj_core.h> (this is documented in MSDN). We add the interfaces to the inheritance hierarchy, the COM interface map, and use the Visual Studio feature to add the interface members for us by right-clicking the class name (CIconHandler), pressing Ctrl+. (dot) and selecting Implement all pure virtuals of CIconHandler. The resulting class header looks something like this (some parts omitted for clarity) (I have removed the virtual keyword as it’s inherited and doesn’t have to be specified in derived types):
Now for the implementation. The IPersistFile interface seems non-trivial, but fortunately we just need to implement the Load method for an icon handler. This is where we get the file name we need to inspect. To check whether a DLL is 64 or 32 bit, we’ll add a simple enumeration and a helper function to the CIconHandler class:
The method receives the full path of the DLL we need to examine. How do we know that only DLL files will be delivered? This has to do with the registration we’ll make for the icon handler. We’ll register it for DLL file extensions only, so that other file types will not be provided. Calling GetModuleBitness (shown later) performs the real work of determining the DLL’s bitness and stores the result in m_Bitness (a data member of type ModuleBitness).
All that’s left to do is tell explorer which icon to use. This is the role of IExtractIcon. The Extract method can be used to provide an icon handle directly, which is useful if the icon is “dynamic” – perhaps generated by different means in each case. In this example, we just need to return one of two icons which have been added as resources to the project (you can find those in the project source code. This is also an opportunity to provide your own icons).
For our case, it’s enough to return S_FALSE from Extract that causes explorer to use the information returned from GetIconLocation. Here is its implementation:
The method’s purpose is to return the current (our icon handler DLL) module’s path and the icon index to use. This information is enough for explorer to load the icon itself from the resources. First, we get the module path to where our DLL has been installed. Since this doesn’t change, it’s only retrieved once (with GetModuleFileName) and stored in a static variable (s_ModulePath).
If this fails (unlikely) or the bitness could not be determined (maybe the file was not a PE at all, but just had such an extension), then we return S_FALSE. This tells explorer to use the default icon for the file type (DLL). Otherwise, we store 0 or 1 in piIndex, based on the IDs of the icons (0 corresponds to the lower of the IDs).
Finally, we need to set a flag inside pwFlags to indicate to explorer that this icon extraction is required for every file (GIL_PERINSTANCE). Otherwise, explorer calls IExtractIcon just once for any DLL file, which is the opposite of what we want.
The final piece of the puzzle (in terms of code) is how to determine whether a PE is 64 or 32 bit. This is not the point of this post, as any custom algorithm can be used to provide different icons for different files of the same type. For completeness, here is the code with comments:
CIconHandler::ModuleBitness CIconHandler::GetModuleBitness(PCWSTR path) {
auto bitness = ModuleBitness::Unknown;
//
// open the DLL as a data file
//
auto hFile = ::CreateFile(path, GENERIC_READ, FILE_SHARE_READ,
nullptr, OPEN_EXISTING, 0, nullptr);
if (hFile == INVALID_HANDLE_VALUE)
return bitness;
//
// create a memory mapped file to read the PE header
//
auto hMemMap = ::CreateFileMapping(hFile, nullptr, PAGE_READONLY, 0, 0, nullptr);
::CloseHandle(hFile);
if (!hMemMap)
return bitness;
//
// map the first page (where the header is located)
//
auto p = ::MapViewOfFile(hMemMap, FILE_MAP_READ, 0, 0, 1 << 12);
if (p) {
auto header = ::ImageNtHeader(p);
if (header) {
auto machine = header->FileHeader.Machine;
bitness = header->Signature == IMAGE_NT_OPTIONAL_HDR64_MAGIC ||
machine == IMAGE_FILE_MACHINE_AMD64 || machine == IMAGE_FILE_MACHINE_ARM64 ?
ModuleBitness::Bit64 : ModuleBitness::Bit32;
}
::UnmapViewOfFile(p);
}
::CloseHandle(hMemMap);
return bitness;
}
To make all this work, there is still one more concern: registration. Normal COM registration is necessary (so that the call to CoCreateInstanceissued by explorer has a chance to succeed), but not enough. Another registration is needed to let explorer know that this icon handler exists, and is to be used for files with the extension “DLL”.
Fortunately, ATL provides a convenient mechanism to add Registry settings using a simple script-like configuration, which does not require any code. The added keys/values have been placed in DllIconHandler.rgs like so:
This sets an icon handler in HKEY_CLASSES_ROOT\DllFile\ShellEx, where the IconHandler value specifies the CLSID of our component. You can find the CLSID in the IDL file where the coclass element is defined:
Replace your own CLSID if you’re building this project from scratch. Registration itself is done with the RegSvr32 built-in tool. With an ATL project, a successful build also causes RegSvr32 to be invoked on the resulting DLL, thus performing registration. The default behavior is to register in HKEY_CLASSES_ROOT which uses HKEY_LOCAL_MACHINE behind the covers. This requires running Visual Studio elevated (or an elevated command window if called from outside VS). It will register the icon handler for all users on the machine. If you prefer to register for the current user only (which uses HKEY_CURRENT_USER and does not require running elevated), you can set the per-user registration in VS by going to project properties, clinking on the Linker element and setting per-user redirection:
If you’re registering from outside VS, the per-user registration is achieved with:
regsvr32 /n /i:user <dllpath>
This is it! The full source code is available here.
I am announcing the next 5 day Windows Internals remote training to be held in January 2022, starting on the 24th according to the followng schedule:
Jan 24 – 2pm to 10pm (all times are based on London time)
Jan 25, 26, 27 – 2pm to 6pm
Jan 31 – 2pm to 10pm
Feb 1, 2, 3 – 2pm to 6pm
The syllabus can be found here (slight changes are possible if new important topics come up).
Cost and Registration
I’m keeping the cost of these training classes relatively low. This is to make these classes accessible to more people, especially in these unusual and challenging times.
Cost: 800 USD if paid by an individual, 1500 USD if paid by a company. Multiple participants from the same company are entitled to a discount (email me for the details). Previous students of my classes are entitled to a 10% discount.
To register, send an email to [email protected] and specify “Windows Internals Training” in the title. The email should include your name, contact email, time zone, and company name (if any).
Today I’m announcing a new training – COM (Component Object Model) Programming, to be held in November.
COM is a well established technology, debuted back in 1993, and is still very much in use. In fact, the Windows Runtime is based on an enhanced version of COM. There is quite a bit of confusion and misconceptions about COM, which is one reason I decided to offer this class.
The syllabus for the 3 day class can be found here. This is the first time I will be offering this class, and also will try a new format: 6 half-days instead of 3 full days.
When: November: 8, 9, 10, 11, 15, 16. 2pm to 6pm, UT. (Technically it’s more than 3 days, as with a full day there is a lunch break not present in half days). The class will be conducted remotely using Microsoft Teams or a similar platform.
What you need to know before the class: You should be comfortable using Windows on a Power User level. Concepts such as processes, threads, DLLs, and virtual memory should be understood fairly well. You should have experience writing code in C and some C++. You don’t have to be an expert, but you must know C and basic C++ to get the most out of the class. In case you have doubts, talk to me.
Obviously, participants in my Windows Internals and (especially) Windows System Programming classes have the required knowledge for the class.
We’ll start by looking at why COM was created in the first place, and then build clients and servers, digging into various mechanisms COM provides. See the syllabus for more details.
Registration will be different than previous classes: Early bird (paid by September 30th): 500 USD (if paid by an individual), 1100 USD (if paid by a company). Normal (paid after September 30th): 700USD (if paid by an individual), 1300 USD (if paid by a company).
To register, send an email to [email protected] with the title “COM Training”, and write the name(s), email(s) and time zone(s) of the participants.Multiple participants from the same company get a discount. Previous participants (individuals) of my classes get 10% off. I will reply with further instructions.
The relationship between processes and threads is fairly well known – a process contains one or more threads, running within the process, using process wide resources, like handles and address space.
Where do windows (those usually-rectangular elements, not the OS) come into the picture?
The relationship between a process, its threads, and windows, along with desktop and Window Stations, can be summarized in the following figure:
A process, threads, windows, desktops and a Window Station
A window is created by a thread, and that thread is designated as the owner of the window. The creating thread becomes a UI thread (if it’s the first User/GDI call it makes), getting a message queue (provided internally by Win32k.sys in the kernel). Every occurrence in the created window causes a message to be placed in the message queue of the owner thread. This means that if a thread creates 100 windows, it’s responsible for all these windows.
“Responsible” here means that the thread must perform something known as “message pumping” – pulling messages from its message queue and processing them. Failure to do that processing would lead to the infamous “Not Responding” scenario, where all the windows created by that thread become non-responsive. Normally, a thread can read its own message queue only – other threads can’t do it for that thread – this is a single threaded UI scheme used by the Windows OS by default. The simplest message pump would look something like this:
The call to GetMessagedoes not return until there is a message in the queue (filling up the MSG structure), or the WM_QUIT message has been retrieved, causing GetMessage to return FALSE and the loop exited. WM_QUIT is the message typically used to signal an application to close. Here is the MSG structure definition:
Once a message has been retrieved, the important call is to DispatchMessage, that looks at the window handle in the message (hwnd) and looks up the window class that this window instance is a member of, and there, finally, the window procedure – the callback to invoke for messages destined to these kinds of windows – is invoked, handling the message as appropriate or passing it along to default processing by calling DefWindowProc. (I may expand on that if there is interest in a separate post).
You can enumerate the windows owned by a thread by calling EnumThreadWindows. Let’s see an example that uses the Toolhelp API to enumerate all processes and threads in the system, and lists the windows created (owned) by each thread (if any).
We’ll start by creating a snapshot of all processes and threads (error handling mostly omitted for clarity):
#include <Windows.h>
#include <TlHelp32.h>
#include <unordered_map>
#include <string>
int main() {
auto hSnapshot = ::CreateToolhelp32Snapshot(
TH32CS_SNAPPROCESS | TH32CS_SNAPTHREAD, 0);
We’ll build a map of process IDs to names for easy lookup later on when we show details of a process:
PROCESSENTRY32 pe;
pe.dwSize = sizeof(pe);
// we can skip the idle process
::Process32First(hSnapshot, &pe);
std::unordered_map<DWORD, std::wstring> processes;
processes.reserve(500);
while (::Process32Next(hSnapshot, &pe)) {
processes.insert({ pe.th32ProcessID, pe.szExeFile });
}
Now we loop through all the threads, calling EnumThreadWindows and showing the results:
THREADENTRY32 te;
te.dwSize = sizeof(te);
::Thread32First(hSnapshot, &te);
static int count = 0;
struct Data {
DWORD Tid;
DWORD Pid;
PCWSTR Name;
};
do {
if (te.th32OwnerProcessID <= 4) {
// idle and system processes have no windows
continue;
}
Data d{
te.th32ThreadID, te.th32OwnerProcessID,
processes.at(te.th32OwnerProcessID).c_str()
};
::EnumThreadWindows(te.th32ThreadID, [](auto hWnd, auto param) {
count++;
WCHAR text[128], className[32];
auto data = reinterpret_cast<Data*>(param);
int textLen = ::GetWindowText(hWnd, text, _countof(text));
int classLen = ::GetClassName(hWnd, className, _countof(className));
printf("TID: %6u PID: %6u (%ws) HWND: 0x%p [%ws] %ws\n",
data->Tid, data->Pid, data->Name, hWnd,
classLen == 0 ? L"" : className,
textLen == 0 ? L"" : text);
return TRUE; // bring in the next window
}, reinterpret_cast<LPARAM>(&d));
} while (::Thread32Next(hSnapshot, &te));
printf("Total windows: %d\n", count);
EnumThreadWindows requires the thread ID to look at, and a callback that receives a window handle and whatever was passed in as the last argument. Returning TRUE causes the callback to be invoked with the next window until the window list for that thread is complete. It’s possible to return FALSE to terminate the enumeration early.
I’m using a lambda function here, but only a non-capturing lambda can be converted to a C function pointer, so the param value is used to provide context for the lambda, with a Data instance having the thread ID, its process ID, and the process name.
The call to GetClassNamereturns the name of the window class, or type of window, if you will. This is similar to the relationship between objects and classes in object orientation. For example, all buttons have the class name “button”. Of course, the windows enumerated are only top level windows (have no parent) – child windows are not included. It is possible to enumerate child windows by calling EnumChildWindows.
Going the other way around – from a window handle to its owner thread and *its* owner process is possible with GetWindowThreadProcessId.
Desktops
A window is placed on a desktop, and is bound to it. More precisely, a thread is bound to a desktop once it creates its first window (or has any hook installed with SetWindowsHookEx). Before that happens, a thread can attach itself to a desktop by calling SetThreadDesktop.
Normally, a thread will use the “default” desktop, which is where a logged in user sees his or her windows appearing. It is possible to create additional desktops using the CreateDesktopAPI. A classic example is the desktopsSysinternals tool. See my post on desktops and windows stations for more information.
It’s possible to redirect a new process to use a different desktop (and optionally a window station) as part of the CreateProcess call. The STARTUPINFO structure has a member named lpDesktop that can be set to a string with the format “WindowStationName\DesktopName” or simply “DesktopName” to keep the parent’s window station. This could look something like this:
STARTUPINFO si = { sizeof(si) };
// WinSta0 is the interactive window station
WCHAR desktop[] = L"Winsta0\\MyOtherDesktop";
si.lpDesktop = desktop;
PROCESS_INFORMATION pi;
::CreateProcess(..., &si, &pi);
“WinSta0” is always the name of the interactive Window Station (one of its desktops can be visible to the user).
The following example creates 3 desktops and launches notepad in the default desktop and the other desktops:
If yoy run the above function, you’ll see one Notepad window on your desktop. If you dig deeper, you’ll find 4 notepad.exe processes in Task Manager. The other three have created their windows in different desktops. Task Manager gives no indication of that, nor does the process list in Process Explorer. However, we see a hint in the list of handles of these “other” notepad processes. Here is one:
Handle named \Desktop1
Curiously enough, desktop objects are not stored as part of the kernel’s Object Manager namespace. If you double-click the handle, copy its address, and look it up in the kernel debugger, this is the result:
Notice the directory object address is zero – it’s not part of any directory. It has meaning in the context of its containing Window Station.
You might think that we can use the EnumThreadWindows as demonstrated at the beginning of this post to locate the other notepad windows. Unfortunately, access is restricted and those notepad windows are not listed (more on that possibly in a future post).
We can try a different approach by using yet another enumration function – EnumDesktopWindows. Given a powerful enough desktop handle, we can enumerate all top level windows in that desktop:
The lambda body is very similar to the one we’ve seen earlier. The difference is that we start with a window handle with no thread ID. But it’s fairly easy to get the owner thread and process with GetWindowThreadProcessId. Here is one way to invoke this function:
auto hDesktop = ::OpenDesktop(L"Desktop1", 0, FALSE, DESKTOP_ENUMERATE);
ListDesktopWindows(hDesktop);
A Notepad window is clearly there. We can repeat the same exercise with the other two desktops, which I have named “Desktop2” and “Desktop3”.
Can we view and interact with these “other” notepads? The SwitchDesktopAPI exists for that purpose. Given a desktop handle with the DESKTOP_SWITCHDESKTOP access, SwitchDesktop changes the input desktop to the requested desktop, so that input devices redirect to that desktop. This is exactly how the Sysinternals Desktops tool works. I’ll leave the interested reader to try this out.
Window Stations
The last piece of the puzzle is a Window Station. I assume you’ve read my earlier post and understand the basics of Window Stations.
A Thread can be associated with a desktop. A process is associated with a window station, with the constraint being that any desktop used by a thread must be within the process’ window station. In other words, a process cannot use one window station and any one of its threads using a desktop that belongs to a different window station.
A process is associated with a window station upon creation – the one specified in the STARTUPINFO structure or the creator’s window station if not specified. Still, a process can associate itself with a different window station by calling SetProcessWindowStation. The constraint is that the provided window station must be part of the current session. There is one window station per session named “WinSta0”, called the interactive window station. One of its desktop can be the input desktop and have user interaction. All other window stations are non-interactive by definition, which is perfectly fine for processes that don’t require any user interface. In return, they get better isolation, because a window station contains its own set of desktops, its own clipboard and its own atom table. Windows handles, by the way, have Window Station scope.
Just like desktops, window stations can be created by calling CreateWindowStation. Window stations can be enumerated as well (in the current session) with EnumerateWindowStations.
Summary
This discussion of threads, processes, desktops, and window stations is not complete, but hopefully gives you a good idea of how things work. I may elaborate on some advanced aspects of Windows UI system in future posts.
Today I’m announcing the next public remote Windows Kernel Programming training. This is a 5-day training scheduled for October: 4, 5, 7, 11, 13. Times: 12pm to 8pm, London Time.
The syllabus can be found here. It may be slightly modified by the time the class starts, but not by much. This is a development-heavy course, so be prepared to write lots of code!
Cost: 800 USD if paid by an individual, 1500 USD if paid by a company. Previous participants of the my classes get 10% discount. Multiple participants from the same company are entitled to a discount (email me for the details).
To register, send an email to [email protected] and specify “Windows Kernel Programming Training” in the title. The email should include your name, preferred email for communication, and company name (if any).
The training sessions will be recorded and provided to the participants.
Please read carefully the pre-requisites for this class. You should especially be comfortable coding in C (any C++ used in the class will be explained). In case of any doubt, talk to me. If you have any questions, feel free to shoot me an email, or DM me on twitter (@zodiacon) or Linkedin (https://www.linkedin.com/in/pavely/).
While teaching a Windows Internals class recently, I came across a situation which looked like a bug to me, but turned out to be something I didn’t know about – dynamic symbolic links.
Symbolic links are Windows kernel objects that point to another object. The weird situation in question was when running WinObj from Sysinternals and navigating to the KenrelObjects object manager directory.
WinObj from Sysinternals
You’ll notice some symbolic link objects that look weird: MemoryErrors, PhysicalMemoryChange, HighMemoryCondition, LowMemoryCondition and a few others. The weird thing that is fairly obvious is that these symbolic link objects have empty targets. Double-clicking any one of them confirms no target, and also shows a curious zero handles, as well as quota change of zero:
Symbolic link properties
To add to the confusion, searching for any of them with Process Explorer yields something like this:
It seems these objects are events, and not symbolic links!
My first instinct was that there is a bug in WinObj (I rewrote it recently for Sysinternals, so was certain I introduced a bug). I ran an old WinObj version, but the result was the same. I tried other tools with similar functionality, and still got the same results. Maybe a bug in Process Explorer? Let’s see in the kernel debugger:
Definitely an event and not a symbolic link. What’s going on? I debugged it in WinObj, and indeed the reported object type is a symbolic link. Maybe it’s a bug in the NtQueryDirectoryObject used to query a directory object for an object.
I asked Mark Russinovich, could there be a bug in Windows? Mark remembered that this is not a bug, but a feature of symbolic links, where objects can be created/resolved dynamically when accessing the symbolic link. Let’s see if we can see something in the debugger:
Clearly, there is target, but notice the flags value 0x10. This is the flag indicating the symbolic link is a dynamic one. To get further information, we need to look at the object with a “symbolic link lenses” by using the data structure the kernel uses to represent symbolic links:
The Callback member shows the function that is being called (MiResolveMemoryEvent) that “resolves” the symbolic link to the relevant event. There are currently 11 such events, their names visible with the following:
I am announcing the next Windows Internals remote training to be held in July 2021 on the 12, 14, 15, 19, 21. Times: 11am to 7pm, London time.
The syllabus can be found here (slight changes are possible if new important topics come up).
Cost and Registration
I’m keeping the cost of these training classes relatively low. This is to make these classes accessible to more people, especially in these unusual and challenging times.
Cost: 800 USD if paid by an individual, 1500 USD if paid by a company. Multiple participants from the same company are entitled to a discount (email me for the details). Previous students of my classes are entitled to a 10% discount.
To register, send an email to [email protected] and specify “Windows Internals Training” in the title. The email should include your name, contact email, and company name (if any).
Later this year I plan a Windows Kernel Programming class. Stay tuned!
Normally, a process created by another process in Windows establishes a parent-child relationship between them. This relationship can be viewed in tools such as Process Explorer. Here is an example showing FoxitReader as a child process of Explorer, implying Explorer created FoxitReader. That must be the case, right?
First, it’s important to emphasize that there is no dependency between a child and a parent process in any way. For example, if the parent process terminates, the child is unaffected. The parent is used for inheriting many properties of the child process, such as current directory and environment variables (if not specified explicitly).
Windows stores the parent process ID in the process management object of the child in the kernel for posterity. This also means that the process ID, although correct, could point to a “wrong” process. This can happen if the parent dies, and the process ID is reused.
Process Explorer does not get confused in such a case, and left-justifies the process in its tree view. It knows to check the process creation time. If the “parent” was created after the child, there is no way it could be the real parent. The parent process ID can still be viewed in the process properties:
Notice the “<Non-existent Parent>” indication. Although the parent process ID is known (13636 in this case), there is no other information on that process, as it does not exist anymore, and its ID may or may not have been reused.
By the way, don’t be alarmed that Explorer has no living parent. This is expected, as it’s normally created by a process running the image UserInit.exe, that exits normally after finishing its duties.
Returning to the question raised earlier – is the parent process always the actual creator? Not necessarily.
The canonical example is when a process is launched elevated (for example, by right-clicking it in Explorer and selecting “Run as Administrator”. Here is a diagram showing the major components in an elevation procedure:
First, the user right-clicks in Explorer and asks to run some App.Exe elevated. Explorer calls ShellExecute(Ex) with the verb “runas” that requests this elevation. Next, The AppInfo service is contacted to perform the operation if possible. It launches consent.exe, which shows the Yes/No message box (for true administrators) or a username/password dialog to get an admin’s approval for the elevation. Assuming this is granted, the AppInfo service calls CreateProcessAsUser to launch the App.exe elevated. To maintain the illusion that Explorer created App.exe, it stores the parent ID of Explorer into App.exe‘s new process management block. This makes it look as if Explorer had created App.exe, but is not really what happened. However, that “re-parenting” is probably a good idea in this case, giving the user the expected result.
Is it possible to specify a different parent when creating a process with CreateProcess?
It is indeed possible by using a process attribute. Here is a function that does just that (with minimal error handling):
bool CreateProcessWithParent(DWORD parentId, PWSTR commandline) {
auto hProcess = ::OpenProcess(PROCESS_CREATE_PROCESS, FALSE, parentId);
if (!hProcess)
return false;
SIZE_T size;
//
// call InitializeProcThreadAttributeList twice
// first, get required size
//
::InitializeProcThreadAttributeList(nullptr, 1, 0, &size);
//
// now allocate a buffer with the required size and call again
//
auto buffer = std::make_unique<BYTE[]>(size);
auto attributes = reinterpret_cast<PPROC_THREAD_ATTRIBUTE_LIST>(buffer.get());
::InitializeProcThreadAttributeList(attributes, 1, 0, &size);
//
// add the parent attribute
//
::UpdateProcThreadAttribute(attributes, 0,
PROC_THREAD_ATTRIBUTE_PARENT_PROCESS,
&hProcess, sizeof(hProcess), nullptr, nullptr);
STARTUPINFOEX si = { sizeof(si) };
//
// set the attribute list
//
si.lpAttributeList = attributes;
PROCESS_INFORMATION pi;
//
// create the process
//
BOOL created = ::CreateProcess(nullptr, commandline, nullptr, nullptr,
FALSE, EXTENDED_STARTUPINFO_PRESENT, nullptr, nullptr,
(STARTUPINFO*)&si, &pi);
//
// cleanup
//
::CloseHandle(hProcess);
::DeleteProcThreadAttributeList(attributes);
return created;
}
The first step is to open a handle to the “prospected” parent by calling OpenProcess with the PROCESS_CREATE_PROCESS access mask. This means you cannot choose an arbitrary process, you must have at least that access to the process. Protected (and protected light) processes, for example, cannot be used as parents in this way, as PROCESS_CREATE_PROCESS cannot be obtained for such processes.
Next, an attribute list is created with a single attribute of type PROC_THREAD_ATTRIBUTE_PARENT_PROCESS by calling InitializeProcThreadAttributeList followed by UpdateProcThreadAttribute to set up the parent handle. Finally, CreateProcess is called with the extended STARTUPINFOEX structure where the attribute list is stored. CreateProcess must know about the extended version by specifying EXTENDED_STARTUPINFO_PRESENT in its flags argument.
Here is an example where Excel supposedly created Notepad (it really didn’t). The “real” process creator is nowhere to be found.
The obvious question perhaps is whether there is any way to know which process was the real creator?
The only way that seems to be available is for kernel drivers that register for process creation notifications with PsSetCreateProcessNotifyRoutineEx (or one of its variants). When such a notification is invoked in the driver, the following structure is provided:
On the one hand, there is the ParentProcessId member (although it’s typed as HANDLE, it actually the parent process ID). This is the parent process as set by CreateProcess based on the code snippet in CreateProcessWithParent.
However, there is also the CreatingThreadId member of type CLIENT_ID, which is a small structure containing a process and thread IDs. This is the real creator. In most cases it’s going to have the same process ID as ParentProcessId, but not in the case where a different parent has been specified.
After this point, that information goes away, leaving only the parent ID, as it’s the one stored in the EPROCESS kernel structure of the child process.
Update: (thanks to @jdu2600)
The creator is available by listening to the process create ETW event, where the creator is the one raising the event. Here is a snapshot from my own ProcMonXv2:
I am also announcing a new training, requested by quite a few people, Windows System Programming. The dates are in February 2021: 8, 9, 11, 15, 17. Times: 12pm to 8pm, London time.
I’m keeping the cost of these training classes relatively low. This is to make these classes accessible to more people, especially in these challenging times. If you register for both classes, you get 10% off the second class. Previous students of my classes get 10% off as well.
Cost: 750 USD if paid by an individual, 1500 USD if paid by a company. Multiple participants from the same company are entitled to a discount (email me for the details).
To register, send an email to [email protected] and specify “Training” in the title. The email should include the training(s) you’re interested in, your name, contact email, company name (if any) and preferred time zone. The training times have already been selected, but it’s still good to know which time zone you live in.
I have recently completed another successful iteration of the Windows Internals training – thank you those who participated!
I am announcing two upcoming training classes, Windows Internals and Windows Kernel Programming.
Windows Internals (5 days)
I promised some folks that the next Internals training would be convenient to US-based time zones. That said, all time zones are welcome!
Dates: Sep 29, Oct 1, 5, 7, 8 Times: 8am to 4pm Pacific time (11am to 7pm Eastern)
The syllabus can be found here. I may make small changes in the final topics, but the major topics remain the same.
Windows Kernel Programming (4 days)
Dates: Oct 13, 15, 19, 21 Times: TBA
The syllabus can be found here. Again, slight changes are possible. This is a development-heavy course, so be prepared to write lots of code!
The selected time zone will be based on the majority of participants’ preference.
Cost and Registration
The cost for each class is kept relatively low (as opposed to other, perhaps similar offerings), as I’ve done in the past year or so. This is to make these classes accessible to more people, especially in these challenging times. If you register for both classes, you get 10% off the second class. Previous students of my classes get 10% off as well.
Cost: 750 USD if paid by an individual, 1500 USD if paid by a company. Multiple participants from the same company are entitled to a discount (email me for the details).
To register, send an email to [email protected] and specify “Training” in the title. The email should include your name, company name (if any) and preferred time zone.
Please read carefully the pre-requisites of each class, especially for Windows Kernel Programming. In case of doubt, talk to me.
Companies that are interested in such (or other) training classes receive special prices. Topics can also be customized according to specific needs.
Other classes I provide include: Modern C++ Programming, Windows System Programming, COM Programming, C#/.NET Programming (Basic and Advanced), Advanced Windows Debugging, and more. Contact me for detailed syllabi if interested.
The standard Windows Registry contains some keys that are not real keys, but instead are symbolic links (or simply, links) to other keys. For example, the key HKEY_LOCAL_MACHINE\System\CurrentControlSetis a symbolic link to HKEY_LOCAL_MACHINE\System\ControlSet001 (in most cases). When working with the standard Registry editor, RegEdit.exe, symbolic links look like normal keys, in the sense that they behave as the link’s target. The following figure shows the above mentioned keys. They look exactly the same (and they are).
There are several other existing links in the Registry. As another example, the hive HKEY_CURRENT_CONFIG is a link to (HKLM is HKEY_LOCAL_MACHINE) HKLM\SYSTEM\CurrentControlSet\Hardware Profiles\Current.
But how to do you create such links yourself? The official Microsoft documentation has partial details on how to do it, and it misses two critical pieces of information to make it work.
Let’s see if we can create a symbolic link. One rule of Registry links, is that the link must point to somewhere within the same hive where the link is created; we can live with that. For demonstration purposes, we’ll create a link in HKEY_CURRENT_USER named DesktopColors that links to HKEY_CURRENT_USER\Control Panel\Desktop\Colors.
The first step is to create the key and specify it to be a link rather than a normal key (error handling omitted):
The important part is that REG_OPTION_CREATE_LINK flag that indicates this is supposed to be a link rather than a standard key. The KEY_WRITE access mask is required as well, as we are about to set the link’s target.
Now comes the first tricky part. The documentation states that the link’s target should be written to a value named “SymbolicLinkValue” and it must be an absolute registry path. Sounds easy enough, right? Wrong. The issue here is the “absolute path” – you might think that it should be something like “HKEY_CURRENT_USER\Control Panel\Desktop\Colors” just like we want, but hey – maybe it’s supposed to be “HKCU” instead of “HKEY_CURRENT_USER” – it’s just a string after all.
It turns out both these variants are wrong. The “absolute path” required here is a native Registry path that is not visible in RegEdit.exe, but it is visible in my own Registry editing tool, RegEditX.exe, downloadable from https://github.com/zodiacon/AllTools. Here is a screenshot, showing the “real” Registry vs. the view we get with RegEdit.
This top view is the “real” Registry is seen by the Windows kernel. Notice there is no HKEY_CURRENT_USER, there is a USER key where subkeys exist that represent users on this machine based on their SIDs. These are mostly visible in the standard Registry under the HKEY_USERS hive.
The “absolute path” needed is based on the real view of the Registry. Here is the code that writes the correct path based on my (current user’s) SID:
The above code shows the second (undocumented, as far as I can tell) piece of crucial information – the length of the link path (in bytes) must NOT include the NULL terminator. Good luck guessing that
And that’s it. We can safely close the key and we’re done.
Well, almost. If you try to delete your newly created key using RegEdit.exe – the target is deleted, rather than the link key itself! So, how do you delete the key link? (My RegEditX does not support this yet).
The standard RegDeleteKey and RegDeleteKeyEx APIs are unable to delete a link. Even if they’re given a key handle opened with REG_OPTION_OPEN_LINK – they ignore it and go for the target. The only API that works is the native NtDeleteKey function (from NtDll.Dll).
First, we add the function’s declaration and the NtDll import:
extern "C" int NTAPI NtDeleteKey(HKEY);
#pragma comment(lib, "ntdll")
As a final note, RegCreateKeyEx cannot open an existing link key – it can only create one. This in contrast to standard keys that can be created OR opened with RegCreateKeyEx. This means that if you want to change an existing link’s target, you have to call RegOpenKeyEx first (with REG_OPTION_OPEN_LINK) and then make the change (or delete the link key and re-create it).
Many of you are probably familiar with Process Explorer‘s ability to close a handle in any process. How can this be accomplished programmatically?
The standard CloseHandle function can close a handle in the current process only, and most of the time that’s a good thing. But what if you need, for whatever reason, to close a handle in another process?
There are two routes than can be taken here. The first one is using a kernel driver. If this is a viable option, then nothing can prevent you from doing the deed. Process Explorer uses that option, since it has a kernel driver (if launched with admin priveleges at least once). In this post, I will focus on user mode options, some of which are applicable to kernel mode as well.
The first issue to consider is how to locate the handle in question, since its value is unknown in advance. There must be some criteria for which you know how to identify the handle once you stumble upon it. The easiest (and probably most common) case is a handle to a named object.
Let take a concrete example, which I believe is now a classic, Windows Media Player. Regardless of what opnions you may have regarding WMP, it still works. One of it quirks, is that it only allows a single instance of itself to run. This is accomplished by the classic technique of creating a named mutex when WMP comes up, and if it turns out the named mutex already exists (presumabley created by an already existing instance of itself), send a message to its other instance and then exits.
The following screenshot shows the handle in question in a running WMP instance.
This provides an opportunity to close that mutex’ handle “behind WMP’s back” and then being able to launch another instance. You can try this by manually closing the handle with Process Explorer and then launch another WMP instance successfully.
If we want to achieve this programmatically, we have to locate the handle first. Unfortunately, the documented Windows API does not provide a way to enumerate handles, not even in the current process. We have to go with the (officially undocumented) Native API if we want to enumerate handles. There two routes we can use:
Enumerate all handles in the system with NtQuerySystemInformation, search for the handle in the PID of WMP.
Enumerate all handles in the WMP process only, searching for the handle yet again.
Inject code into the WMP process to query handles one by one, until found.
Option 3 requires code injection, which can be done by using the CreateRemoteThreadEx function, but requires a DLL that we inject. This technique is very well-known, so I won’t repeat it here. It has the advantage of not requring some of the native APIs we’ll be using shortly.
Options 1 and 2 look very similar, and for our purposes, they are. Option 1 retrieves too much information, so it’s probably better to go with option 2.
Let’s start at the beginning: we need to locate the WMP process. Here is a function to do that, using the Toolhelp API for process enumeration:
#include <windows.h>
#include <TlHelp32.h>
#include <stdio.h>
DWORD FindMediaPlayer() {
HANDLE hSnapshot = ::CreateToolhelp32Snapshot(TH32CS_SNAPPROCESS, 0);
if (hSnapshot == INVALID_HANDLE_VALUE)
return 0;
PROCESSENTRY32 pe;
pe.dwSize = sizeof(pe);
// skip the idle process
::Process32First(hSnapshot, &pe);
DWORD pid = 0;
while (::Process32Next(hSnapshot, &pe)) {
if (::_wcsicmp(pe.szExeFile, L"wmplayer.exe") == 0) {
// found it!
pid = pe.th32ProcessID;
break;
}
}
::CloseHandle(hSnapshot);
return pid;
}
int main() {
DWORD pid = FindMediaPlayer();
if (pid == 0) {
printf("Failed to locate media player\n");
return 1;
}
printf("Located media player: PID=%u\n", pid);
return 0;
}
Now that we have located WMP, let’s get all handles in that process. The first step is opening a handle to the process with PROCESS_QUERY_INFORMATION and PROCESS_DUP_HANDLE (we’ll see why that’s needed in a little bit):
HANDLE hProcess = ::OpenProcess(PROCESS_QUERY_INFORMATION | PROCESS_DUP_HANDLE,
FALSE, pid);
if (!hProcess) {
printf("Failed to open WMP process handle (error=%u)\n",
::GetLastError());
return 1;
}
If we can’t open a proper handle, then something is terribly wrong. Maybe WMP closed in the meantime?
Now we need to work with the native API to query the handles in the WMP process. We’ll have to bring in some definitions, which you can find in the excellent phnt project on Github (I added extern "C" declaration because we use a C++ file).
The #include <memory> is for using unique_ptr<> as we’ll do soon enough. The #parma links the NTDLL import library so that we don’t get an “unresolved external” when calling NtQueryInformationProcess. Some people prefer getting the functions address with GetProcAddress so that linking with the import library is not necessary. I think using GetProcAddress is important when using a function that may not exist on the system it’s running on, otherwise the process will crash at startup, when the loader (code inside NTDLL.dll) tries to locate a function. It does not care if we check dynamically whether to use the function or not – it will crash. Using GetProcAddress will just fail and the code can handle it. In our case, NtQueryInformationProcess existed since the first Windows NT version, so I chose to go with the simplest route.
Our next step is to enumerate the handles with the process information class I plucked from the full list in the phnt project (ntpsapi.h file):
ULONG size = 1 << 10;
std::unique_ptr<BYTE[]> buffer;
for (;;) {
buffer = std::make_unique<BYTE[]>(size);
auto status = ::NtQueryInformationProcess(hProcess, ProcessHandleInformation,
buffer.get(), size, &size);
if (NT_SUCCESS(status))
break;
if (status == STATUS_INFO_LENGTH_MISMATCH) {
size += 1 << 10;
continue;
}
printf("Error enumerating handles\n");
return 1;
}
The Query* style functions in the native API request a buffer and return STATUS_INFO_LENGTH_MISMATCH if it’s not large enough or not of the correct size. The code allocates a buffer with make_unique<BYTE[]> and tries its luck. If the buffer is not large enough, it receives back the required size and then reallocates the buffer before making another call.
Now we need to step through the handles, looking for our mutex. The information returned from each handle does not include the object’s name, which means we have to make yet another native API call, this time to NtQyeryObject along with some extra required definitions:
NtQueryObject has several information classes, but we only need the name. But what handle do we provide NtQueryObject? If we were going with option 3 above and inject code into WMP’s process, we could loop with handle values starting from 4 (the first legal handle) and incrementing the loop handle by four.
Here we are in an external process, so handing out the handles provided by NtQueryInformationProcess does not make sense. What we have to do is duplicate each handle into our own process, and then make the call. First, we set up a loop for all handles and duplicate each one:
auto info = reinterpret_cast<PROCESS_HANDLE_SNAPSHOT_INFORMATION*>(buffer.get());
for (ULONG i = 0; i < info->NumberOfHandles; i++) {
HANDLE h = info->Handles[i].HandleValue;
HANDLE hTarget;
if (!::DuplicateHandle(hProcess, h, ::GetCurrentProcess(), &hTarget,
0, FALSE, DUPLICATE_SAME_ACCESS))
continue; // move to next handle
}
We duplicate the handle from WMP’s process (hProcess) to our own process. This function requires the handle to the process opened with PROCESS_DUP_HANDLE.
Now for the name: we need to call NtQueryObject with our duplicated handle and buffer that should be filled with UNICODE_STRING and whatever characters make up the name.
BYTE nameBuffer[1 << 10];
auto status = ::NtQueryObject(hTarget, ObjectNameInformation,
nameBuffer, sizeof(nameBuffer), nullptr);
::CloseHandle(hTarget);
if (!NT_SUCCESS(status))
continue;
Once we query for the name, the handle is not needed and can be closed, so we don’t leak handles in our own process. Next, we need to locate the name and compare it with our target name. But what is the target name? We see in Process Explorer how the name looks. It contains the prefix used by any process (except UWP processes): “\Sessions\<session>\BasedNameObjects\<thename>”. We need the session ID and the “real” name to build our target name:
WCHAR targetName[256];
DWORD sessionId;
::ProcessIdToSessionId(pid, &sessionId);
::swprintf_s(targetName,
L"\\Sessions\\%u\\BaseNamedObjects\\Microsoft_WMP_70_CheckForOtherInstanceMutex",
sessionId);
auto len = ::wcslen(targetName);
This code should come before the loop begins, as we only need to build it once.
Not for the real comparison of names:
auto name = reinterpret_cast<UNICODE_STRING*>(nameBuffer);
if (name->Buffer &&
::_wcsnicmp(name->Buffer, targetName, len) == 0) {
// found it!
}
The name buffer is cast to a UNICODE_STRING, which is the standard string type in the native API (and the kernel). It has a Length member which is in bytes (not characters) and does not have to be NULL-terminated. This is why the function used is _wcsnicmp, which can be limited in its search for a match.
Assuming we find our handle, what do we do with it? Fortunately, there is a trick we can use that allows closing a handle in another process: call DuplicateHandle again, but add the DUPLICATE_CLOSE_SOURCE to close the source handle. Then close our own copy, and that’s it! The mutex is gone. Let’s do it:
// found it!
::DuplicateHandle(hProcess, h, ::GetCurrentProcess(), &hTarget,
0, FALSE, DUPLICATE_CLOSE_SOURCE);
::CloseHandle(hTarget);
printf("Found it! and closed it!\n");
return 0;
This is it. If we get out of the loop, it means we failed to locate the handle with that name. The general technique of duplicating a handle and closing the source is applicable to kernel mode as well. It does require a process handle with PROCESS_DUP_HANDLE to make it work, which is not always possible to get from user mode. For example, protected and PPL (protected processes light) processes cannot be opened with this access mask, even by administrators. In kernel mode, on the other hand, any process can be opened with full access.
It’s been a while since I gave the Windows Internals training, so it’s time for another class of my favorite topics!
This time I decided to make it more afordable, to allow more people to participate. The cost is based on whether paid by an individual vs. a company. The training includes lab exercises – some involve working with tools, while others involve coding in C/C++.
Public 5-day remote class
Dates: April 20, 22, 23, 27, 30
Time: 8 hours / day. Exact hours TBD
Price: 750 USD (payed by individual) / 1500 USD (payed by company)
Register by emailing [email protected] and specifying “Windows Internals Training” in the title
Provide names of participants (discount available for multiple participants from the same company), company name (if any) and preferred time zone.
You’ll receive instructions for payment and other details
Virtual space is limited!
The training time zone will be finalized closer to the start date.
Objectives:
Understand the Windows system architectureExplore the internal workings of process, threads, jobs, virtual memory, the I/O system and other mechanisms fundamental to the way Windows works
Write a simple software device driver to access/modify information not available from user mode
Target Audience:
Experienced windows programmers in user mode or kernel mode, interested in writing better programs, by getting a deeper understanding of the internal mechanisms of the windows operating system.Security researchers interested in gaining a deeper understanding of Windows mechanisms (security or otherwise), allowing for more productive research
Pre-Requisites:
Basic knowledge of OS concepts and architecture.Power user level working with Windows
Practical experience developing windows applications is an advantage
C/C++ knowledge is an advantage
Module 1: System Architecture
Brief Windows NT History
Windows Versions
Tools: Windows, Sysinternals, Debugging Tools for Windows
Processes and Threads
Virtual Memory
User mode vs. Kernel mode
Architecture Overview
Key Components
User/kernel transitions
APIs: Win32, Native, .NET, COM, WinRT
Objects and Handles
Sessions
Introduction to WinDbg
Lab: Task manager, Process Explorer, WinDbg
Module 2: Processes & Jobs
Process basics
Creating and terminating processes
Process Internals & Data Structures
The Loader
DLL explicit and implicit linking
Process and thread attributes
Protected processes and PPL
UWP Processes
Minimal and Pico processes
Jobs
Nested jobs
Introduction to Silos
Server Silos and Docker
Lab: viewing process and job information; creating processes; setting job limits
Module 3: Threads
Thread basics
Thread Internals & Data Structures
Creating and terminating threads
Thread Stacks
Thread Priorities
Thread Scheduling
CPU Sets
Direct Switch
Deep Freeze
Thread Synchronization
Lab: creating threads; thread synchronization; viewing thread information; CPU sets
Module 4: Kernel Mechanisms
Trap Dispatching
Interrupts
Interrupt Request Level (IRQL)
Deferred Procedure Calls (DPCs)
Exceptions
System Crash
Object Management
Objects and Handles
Sharing Objects
Thread Synchronization
Synchronization Primitives (Mutex, Semaphore, Events, and more)
Signaled vs. Non-Signaled
High IRQL Synchronization
Windows Global Flags
Kernel Event Tracing
Wow64
Lab: Viewing Handles, Interrupts; creating maximum handles; Thread synchronization
Module 5: Memory Management
Overview
Small, large and huge pages
Page states
Memory Counters
Address Space Layout
Address Translation Mechanisms
Heaps
APIs in User mode and Kernel mode
Page Faults
Page Files
Commit Size and Commit Limit
Workings Sets
Memory Mapped Files (Sections)
Page Frame Database
Other memory management features
Lab: committing & reserving memory; using shared memory; viewing memory related information
Module 6: Management Mechanisms
The Registry
Services
Starting and controlling services
Windows Management Instrumentation
Lab: Viewing and configuring services; Process Monitor
Module 7: I/O System
I/O System overview
Device Drivers
Plug & Play
The Windows Driver Model (WDM)
The Windows Driver Framework (WDF)
WDF: KMDF and UMDF
Device and Driver Objects
I/O Processing and Data Flow
IRPs
Power Management
Driver Verifier
Writing a Software Driver
Labs: viewing driver and device information; writing a software driver
System calls on Windows go through NTDLL.dll, where each system call is invoked by a syscall (x64) or sysenter (x86) CPU instruction, as can be seen from the following output of NtCreateFile from NTDLL:
0:000> u
ntdll!NtCreateFile:
00007ffc`c07fcb50 4c8bd1 mov r10,rcx
00007ffc`c07fcb53 b855000000 mov eax,55h
00007ffc`c07fcb58 f604250803fe7f01 test byte ptr [SharedUserData+0x308 (00000000`7ffe0308)],1
00007ffc`c07fcb60 7503 jne ntdll!NtCreateFile+0x15 (00007ffc`c07fcb65)
00007ffc`c07fcb62 0f05 syscall
00007ffc`c07fcb64 c3 ret
00007ffc`c07fcb65 cd2e int 2Eh
00007ffc`c07fcb67 c3 ret
The important instructions are marked in bold. The value set to EAX is the system service number (0x55 in this case). The syscall instruction follows (the condition tested does not normally cause a branch). syscall causes transition to the kernel into the System Service Dispatcher routine, which is responsible for dispatching to the real system call implementation within the Executive. I will not go to the exact details here, but eventually, the EAX register must be used as a lookup index into the System Service Dispatch Table (SSDT), where each system service number (index) should point to the actual routine.
On x64 versions of Windows, the SSDT is available in the kernel debugger in the nt!KiServiceTable symbol:
You might expect the values in the SSDT to be 64-bit pointers, pointing directly to the system services (this is the scheme used on x86 systems). On x64 the values are 32 bit, and are used as offsets from the start of the SSDT itself. However, the offset does not include the last hex digit (4 bits): this last value is the number of arguments to the system call.
Let’s see if this holds with NtCreateFile. Its service number is 0x55 as we’ve seen from user mode, so to get to the actual offset, we need to perform a simple calculation:
Indeed – this is NtCreateFile. What about the argument count? The value stored is 7. Here is the prototype of NtCreateFile (documented in the WDK as ZwCreateFile):
Clearly, there are 11 parameters, not just 7. Why the discrepency? The stored value is the number of parameters that are passed using the stack. In x64 calling convention, the first 4 arguments are passed using registers: RCX, RDX, R8, R9 (in this order).
Now back to the title of this post. Here are the first few entries in the SSDT again:
The first two entries look different, with much larger numbers. Let’s try to apply the same logic for the first value (index 0):
kd> u nt!KiServiceTable+fced720
fffff804`2392c340 ?? ???
^ Memory access error in 'u nt!KiServiceTable+fced720'
Clearly a bust. The value is in fact a negative value (in two’s complement), so we need to sign-extend it to 64 bit, and then perform the addition (leaving out the last hex digit as before):
kd> u nt!KiServiceTable+ffffffff`ffced720
nt!NtAccessCheck:
fffff804`1392c340 4c8bdc mov r11,rsp
fffff804`1392c343 4883ec68 sub rsp,68h
fffff804`1392c347 488b8424a8000000 mov rax,qword ptr [rsp+0A8h]
This is NtAccessCheck. The function’s implementation is in lower addresses than the SSDT itself. Let’s try the same exercise with index 1:
One of the new Windows 10 features visible to users is the support for additional “Desktops”. It’s now possible to create additional surfaces on which windows can be used. This idea is not new – it has been around in the Linux world for many years (e.g. KDE, Gnome), where users have 4 virtual desktops they can use. The idea is that to prevent clutter, one desktop can be used for web browsing, for example, and another desktop can be used for all dev work, and yet a third desktop could be used for all social / work apps (outlook, WhatsApp, Facebook, whatever).
To create an additional virtual desktop on Windows 10, click on the Task View button on the task bar, and then click the “New Desktop” button marked with a plus sign.
Now you can switch between desktops by clicking the appropriate desktop button and then launch apps as usual. It’s even possible (by clicking Task View again) to move windows from desktop to desktop, or to request that a window be visible on all desktops.
The Sysinternals tools had a tool called “Desktops” for many years now. It too allows for creation of up to 4 desktops where applications can be launched. The question is – is this Desktops tool the same as the Windows 10 virtual desktops feature? Not quite.
First, some background information. In the kernel object hierarchy under a session object, there are window stations, desktops and other objects. Here’s a diagram summarizing this tree-like relationship:
As can be seen in the diagram, a session contains a set of Window Stations. One window station can be interactive, meaning it can receive user input, and is always called winsta0. If there are other window stations, they are non-interactive.
Each window station contains a set of desktops. Each of these desktops can hold windows. So at any given moment, an interactive user can interact with a single desktop under winsta0. Upon logging in, a desktop called “Default” is created and this is where all the normal windows appear. If you click Ctrl+Alt+Del for example, you’ll be transferred to another desktop, called “Winlogon”, that was created by the winlogon process. That’s why your normal windows “disappear” – you have been switched to another desktop where different windows may exist. This switching is done by a documented function – SwitchDesktop.
And here lies the difference between the Windows 10 virtual desktops and the Sysinternals desktops tool. The desktops tool actually creates desktop objects using the CreateDesktop API. In that desktop, it launches Explorer.exe so that a taskbar is created on that desktop – initially the desktop has nothing on it. How can desktops launch a process that by default creates windows in a different desktop? This is possible to do with the normal CreateProcess function by specifying the desktop name in the STARTUPINFO structure’s lpDesktop member. The format is “windowstation\desktop”. So in the desktops tool case, that’s something like “winsta0\Sysinternals Desktop 1”. How do I know the name of the Sysinternals desktop objects? Desktops can be enumerated with the EnumDesktops API. I’ve written a small tool, that enumerates window stations and desktops in the current session. Here’s a sample output when one additional desktop has been created with “desktops”:
In the Windows 10 virtual desktops feature, no new desktops are ever created. Win32k.sys just manipulates the visibility of windows and that’s it. Can you guess why? Why doesn’t Window 10 use the CreateDesktop/SwitchDesktop APIs for its virtual desktop feature?
The reason has to do with some limitations that exist on desktop objects. For one, a window (technically a thread) that is bound to a desktop cannot be switched to another; in other words, there is no way to transfer a windows from one desktop to another. This is intentional, because desktops provide some protection. For example, hooks set with SetWindowsHookEx can only be set on the current desktop, so cannot affect other windows in other desktops. The Winlogon desktop, as another example, has a strict security descriptor that prevents non system-level users from accessing that desktop. Otherwise, that desktop could have been tampered with.
The virtual desktops in Windows 10 is not intended for security purposes, but for flexibility and convenience (security always “contradicts” convenience). That’s why it’s possible to move windows between desktops, because there is no real “moving” going on at all. From the kernel’s perspective, everything is still on the same “Default” desktop.
The next public remote Windows kernel Programming class I will be delivering is scheduled for April 15 to 18. It’s going to be very similar to the first one I did at the end of January (with some slight modifications and additions).
Cost: 1950 USD. Early bird (register before March 30th): 1650 USD
I have not yet finalized the time zone the class will be “targeting”. I will update in a few weeks on that.
If you’re interested in registering, please email [email protected] with the subject “Windows Kernel Programming class” and specify your name, company (if any) and time zone. I’ll reply by providing more information.
Feel free to contact me for questions using the email or through twitter (@zodiacon).
The complete syllabus is outlined below:
Duration:
4 Days
Target Audience:
Experienced windows developers, interested in developing kernel mode drivers
Objectives:
· Understand the Windows kernel driver programming model
· Write drivers for monitoring processes, threads, registry and some types of objects
· Use documented kernel hooking mechanisms
· Write basic file system mini-filter drivers
Pre Requisites:
· At least 2 years of experience working with the Windows API
· Basic understanding of Windows OS concepts such as processes, threads, virtual memory and DLLs
Software requirements:
· Windows 10 Pro 64 bit (latest stable version)
· Visual Studio 2017 + latest update
· Windows 10 SDK (latest)
· Windows 10 WDK (latest)
· Virtual Machine for testing and debugging
Instructor: Pavel Yosifovich
Abstract
The cyber security industry has grown considerably in recent years, with more sophisticated attacks and consequently more defenders. To have a fighting chance against these kinds of attacks, kernel mode drivers must be employed, where nothing (at least nothing from user mode) can escape their eyes.
The course provides the foundations for the most common software device drivers that are useful not just in cyber security, but also other scenarios, where monitoring and sometimes prevention of operations is required. Participants will write real device drivers with useful features they can then modify and adapt to their particular needs.
Syllabus
Module 1: Windows Internals quick overview
Processes
Virtual memory
Threads
System architecture
User / kernel transitions
Introduction to WinDbg
Windows APIs
Objects and handles
Summary
Module 2: The I/O System
I/O System overview
Device Drivers
The Windows Driver Model (WDM)
The Kernel Mode Driver Framework (KMDF)
Other device driver models
Driver types
Software drivers
Driver and device objects
I/O Processing and Data Flow
Accessing devices
Asynchronous I/O
Summary
Module 3: Kernel programming basics
Setting up for Kernel Development
Basic Kernel types and conventions
C++ in a kernel driver
Creating a driver project
Building and deploying
The kernel API
Strings
Linked Lists
The DriverEntry function
The Unload routine
Installation
Testing
Debugging
Summary
Lab: deploy a driver
Module 4: Building a simple driver
Creating a device object
Exporting a device name
Building a driver client
Driver dispatch routines
Introduction to I/O Request Packets (IRPs)
Completing IRPs
Handling DeviceIoControl calls
Testing the driver
Debugging the driver
Using WinDbg with a virtual machine
Summary
Lab: open a process for any access; zero driver; debug a driver
Module 5: Kernel mechanisms
Interrupt Request Levels (IRQLs)
Deferred Procedure Calls (DPCs)
Dispatcher objects
Low IRQL Synchronization
Spin locks
Work items
Summary
Module 6: Process and thread monitoring
Motivation
Process creation/destruction callback
Specifying process creation status
Thread creation/destruction callback
Notifying user mode
Writing a user mode client
Preventing potentially malicious processes from executing
Summary
Lab: monitoring process/thread activity; prevent specific processes from running; protecting processes
Module 7: Object and registry notifications
Process/thread object notifications
Pre and post callbacks
Registry notifications
Performance considerations
Reporting results to user mode
Summary
Lab: protect specific process from termination; simple registry monitor
Module 8: File system mini filters
File system model
Filters vs. mini filters
The Filter Manager
Filter registration
Pre and Post callbacks
File name information
Contexts
File system operations
Filter to user mode communication
Debugging mini-filters
Summary
Labs: protect a directory from file deletion; backup file before deletion
