As I am spending slices of my time refreshing some Malware Analysis theory, I thought was valuable (at least to my future amnesiac self) writing down a simple ‘custom base64 alphabet translator.’ This can/should be extended to support CLI/WebApp i ntegration. So, here is the skeleton:
UPDATE: added interactive mode below and also found this great tool which is already doing what I aimed for and much more.
import string import base64 # custom encoding function def EncodeCustomBase64(stringToBeEncoded,custom_b64): standard_b64 = 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/=' encoded = base64.
Hide and Seek (HNS) is a malicious worm which mainly infects Linux based IoT devices and routers. The malware spreads via bruteforcing SSH/Telnet credentials, as
Hot on the heels of the latest post, I have decided to port to linux another lab example from Intermediate x86 class.
This time I will talk about the RedPill paper by Joanna Rutkowska.
This test is expected to work on single-core CPUs only
The main purpose of this test code is to call the SIDT instruction and save its return value inside an array: this value will tell us whether we are running it inside a vm or not.
I was about to finish the Intermediate x86 class from OpenSecTraining, when I thought was worthwhile porting to Linux this interesting exercise about software breakpoints.
Whenever we send a software breakpoint on GDB or any other debugger, the debugger swaps the first instruction byte with the double C (0xCC) instruction and keeps track of each and every breakpoint/replaced-instruction via the breakpoint table and thus, leaving the code section of the executable altered.
Something that once was done with heavy and expensive serial cables, can now be achieved in a matter of seconds through virtual machines. I am of course speaking about kernel debugging, what else? Recently I have been following the exceptionally great Intermediate x86 training lead by Xeno Kovah where, in order to keep up with the labs, I had to setup a WinXP-to-WinXP kernel debugging setup. So after a few moments of bewilderment I reached a full working environment with the following steps.
Flare-on was a blast this year ! All challenges were great but I enjoyed solving the last one the most, although it was somewhat frustrating.
Due to my tight schedule, I won't go over all the details involved in solving the challenge. But I'll do my best to paint a complete picture of what's going on and how I approached the problem.
We start we a floppy disk image that you can download from here (PW : infected) :
When we boot the floppy using bochs, we see that it automatically executes infohelp.exe that asks us for a password.
Entering an arbitrary password, the message "Welcome to FLARE..." prints slowly on the screen and the password checks are performed.
What I did after this, is I mounted the floppy on my Ubuntu virtual machine and extracted the files in bold.
Both key.dat and message.dat contain nothing interesting. However, TMP.DAT appeared to contain the welcome message we see after typing the password and some funny strings like : "NICK RULES" and "BE SURE TO DRINK YOUR OVALTINE".
What I did next is I threw infohelp.exe into IDA to examine its behavior. To my surprise, I found that it does nothing but writes the supplied password to key.dat and then reads the contents of message.dat and prints them to the screen.
Here I thought that there should be some hooks involved that redirect execution to somewhere else when message.dat is opened or read. To confirm my suspicions, I executed the "type" command on message.dat; Lo and behold, the password check is performed.
Next, I opened TMP.DAT in IDA and found that it contains some code that seems to be our hook. So I attached IDA to bochs and started debugging.
To locate the hook within memory, I took advantage of the fact that the message is printed in a slow fashion so what I did is pause execution while stuff was still printing. I found myself in a loop implementing the subleq VM.
The caller supplies a pointer to the bytecode, its size, and the offset to the instruction where execution should start.
Each instruction is 6 bytes and has the following format :
struct inst {
WORD src_index;
WORD dest_index;
WORD branch_index;
};
The type of the subleq bytecode array is WORD*, so the VM views that its instruction size is 3 while it is 6 actually. This realization comes in handy when building an IDA processor module for the subleq.
As I did with last year's binary, I re-implemented the subleq VM with C to output each executed instruction to a file. However, I had an impossible-to-analyze file with over 1 GB. So what I did, is only print the subleq for the instructions that perform the password checks; That way I had a 30 MB-ish file that I could examine looking for patterns.
The way I had the emulated instructions printed was the following :
IP : sub [dest_index], [src_index] ; subtraction = result
The only thing that was visible on the fly is that the subleq starts by doing some initialization in the beginning and then enters a loop that keeps executing until the end. Here where suspicions of a second VM started to arise in my mind (OMG !).
I tried to locate the password characters inside the subleq and tried to figure out what operations were done on them but it was not clear at all.
I also did some text diffing between iterations and saw that the code was modifying itself. In these instances, the self-modification was done in order to dereference VM pointers and use their values as indexes in subsequent operations.
So, what I decided to do here is write a very minimal processor module that would allow me to view the subleq in a neat graph.
The file I extracted contains bytecode starting from IP : 0x5. So here's how you load it into IDA :
- Choose the subleq bytecode processor module and make sure to disable auto-analysis. It ruins everything when enabled.
- Change the ROM start address to 0xA and the input file loading address to the same : 0x5 * sizeof(WORD) == 0xA.
The bytecode will be loaded without being analyzed.
- Press 'P' at 0xA to convert into code and automatically into a function. You should have a beautiful graph as a result.
Well, it is not quite as beautiful as you might think, since we still have to deal with self-modifying code (knowing what exactly is modified) and also understanding the code as a whole.
It is quite hard to understand what subleq does by only reading "subleq" instructions, so the next thing that came to mind is to convert the subleq to MOV and ADD instructions without wasting too much time.
IDAPYTHON TO THE RESCUE !
I wrote a minimal script that looks for ADD and MOV patterns in the subleq and comments these instructions. First of all, the script intentionally skips the instructions that will be self-modified and doesn't comment the SUB since it's already there.
And the result was this :
More understandable ... still, not so much.
So what I did next is decompile this manually into C and then simplify the code.
So it is indeed another VM interpreted by the subleq. The nature of the VM was unknown to me until later when someone told me that it was RSSB. But I was able, however, to solve the challenge without needing that information.
Now, this RSSB VM executes bytecode that starts at offset 0xFEC from the start of the subleq or at offset 0x1250 of the TMP.DAT file.
If you dumped the bytecode from memory as I did, you'd find that the password you typed was written inside the RSSB VM at offset 0x21C (circled in red).
So far so good. I copied the whole RSSB bytecode and added it as an array and modified the C emulator code to print the "sub" instructions while executing the VM; the same way I did with the subleq.
The result looked like this :
IP : p[dest_index], ACCUM ; operation
Reading the code, I found out that a sum is calculated for the characters in the password. In addition to that, the number of characters in the password must be 64. I knew that by examining a variable that gets decremented from 64 at each iteration of the sum calculation.
For information, the sum is stored at : p[0b47].
So I patched the memory to store a 64 byte string and then I looked up where the first character of the input was read apart from where the sum was calculated. I performed a search for [010e] ( 0x21C / 2 == 0x010E).
65 in dec == 0x41 in hex
Long story short, the algorithm works in a loop, in each iteration two characters of the password are used to calculate a value. The sum of all characters is then added to that value as shown in the figure below (sum : in red, value : in green).
A few instructions later, a hardcoded value at [0a79] is subtracted from the resulting value of the previous operation.
We can see that the resulting value of the next two characters for example is compared against the next hardcoded value at [0a7a] and so on until the 30th character.
So, we have a password of 64 bytes but from which only the first 30 bytes are checked !
Let's leave that aside for now and ask : what makes a check correct ? Maybe the result of the subtraction must be zero ?
I quickly added a check onto my C emulator that did the following :
res = ptr[op_addr] - ptr[1];
if ( ptr[0] == 0x203d ) //IP of the check, see figure above { res = 0; }
This will simply patch the result to zero in the check at 0x203d. I ran the program and it showed that the password was correct, so I knew I was on the right path.
I also observed (based on a single test case) that in each iteration the calculated value depends on the position of the two characters. So even if we have the same two characters at different places the calculated value will be different.
Solution : Here I am going to describe how I solved the challenge during the CTF. Surely this is not the best solution out there, but I'd like to show my line of thought and how I came to solve this level.
We know that the same sum is used in all the operations, and that there can be only one sum (or a handful as we'll get to see later) that will satisfy all the checks.
We can run a bruteforce attack where we let the VM calculate the value for two given characters (by patching the characters at run-time) then run the check on all possible sums (by patching the sum at run-time too). The first check will give us a lot of sums that we'll use to bruteforce the second check. In its turn, this check will filter out invalid sums that we'll eliminate from the third check and so on until we do all 15 of them. (30 characters / 2 characters per check == 15 checks).
At the end, we'll get the valid sums from which we can easily deduce the characters that generated them in each check.
The problem I had with this approach was performance. For each combination of two characters, and for each sum, I was running the VM all over again which, if I left like that, would take a huge amount of time : printing the welcome message, calculating the sum for junk characters ... over and over again each time !
What I ended up doing is "snapshotting" the VM in multiple states.
Snapshot 1 : Where the first character of the two is read (depending on the iteration we're in).
Snapshot 2 : For each two characters, take a snapshot after the value that depends on both of them is calculated and right at the address where the sum is read and added to the calculated value (IP == 0x1ff7).
The optimization here is that we execute what we only need to execute and nothing more. We patch the two characters by the ones we're currently bruteforcing at Snapshot 1 and then after the value is calculated for those two we take Snapshot 2 and we only get to patch the sum. When each iteration is over, we re-initialize the snapshots to their original states.
Here's the overall algorithm involved in pseudo-code (this is not C nor anything close) :
sums [xxx] = {...};
new_sums [yyy] = {0};
for ( i = 0; i < 15; i++)
{
memcpy(initial_state_tmp, initial_state);
snapshot_1 = take_snapshot_1(initial_state_tmp, i); //i is passed to go to the corresponding check
//Execute the subtraction check and return a boolean if ( execute_check(snapshot_2_tmp) )
{
append(new_sums, sum); //valid sum, append it
}
}
}
}
sums = new_sums;
new_sums = [0];
}
print sums;
At the end we'll get the valid sums that resulted in the check being equal to 0.
Here's the full C script implementing this (a bit messy) :
After the 15 checks are done, the script gives us files containing the valid sums that passed each of the checks. We're only interested in the last file (4KB in size) highlighted below :
Contents of array_14
I actually forgot to include the characters that generated the sum for each check. And I had to do it separately.
This requires some modifications of the code above : we initialize the sums array with the contents of array_14 and for each sum bruteforce the two characters that pass each check. To optimize, I listed the first four characters (two first checks) for each one of these sums.
And one of them was particularly interesting. The sum 0xd15e resulted in these four characters "Av0c".
Running the same script for this single sum while bruteforcing all of the characters gives us the flag :
Flag : [email protected] Well in the end, this one really deserved being the 12th, it was time consuming, frustrating and above all FUN ! Thanks for bearing with me and until another time guys - take care :)
In this challenge we're given an x64 ELF binary. The program acts as a userspace host for KVM virtualization. Among other things, it sets up the VM's address space, initializes the necessary VM registers, copies the code from the ".payload" section to it, then finally runs it.
Additionally, the userspace host expects the VM to trap when executing the three illegal instructions : IN, OUT, and HLT as shown below. The host will do some processing and then fix the VM's state so it can graciously continue executing.
And here is an instance of a HLT instruction within the VM's code.
Let's now describe the behavior of the host for each illegal instruction.
IN (port 0xE9) : Reads a single character from STDIN and returns it to the VM (The first thing that the VM does is read user input from STDIN).
OUT (port 0xE9) : Outputs a single character to STDOUT.
HLT : Before the VM executes a HLT instruction, it moves a special value into EAX. After it traps, our host reads this value and uses it as a key in an array. Each key corresponds to a handler routine within the VM's address space.
What the host does then is set VM's RIP register to the corresponding handler.
And by examining the handlers, we see that they invoke each other using the HLT instruction.
Now, let's try to examine what the VM does and figure out what these handlers are used for.
Briefly, 0x2800 bytes are read from STDIN and for each of these bytes sub_1E0 is called. The first time it's called, this function takes the user-supplied character and the address 0x1300 which points to some data.
sub_1E0 initializes local variables and then branches to the handler at 0x32c.
This one examines the dereferenced value, if it is 0xFF it branches to the handler at 0x347, if not it branches to a handler that compares the dereferenced value with the user-supplied character.
Now, examining the handler at 0x347 and the handlers it invokes (see the screenshot below : renamed sub_1E0 to traverse_tree), we see that the address 0x1300 is a root node of a binary tree.
In the tree at 0x1300, parent nodes have 0xFF as a value and contain two pointers for the left & right children. A leaf node, contains an ASCII character as a value which we suspect constitutes our flag. Recall that when a leaf is encountered a comparison is made with the user-supplied character and a boolean value is returned (RET instruction).
In the screenshot below, we see that the tree is recursively traversed and when a leaf is encountered and the comparison is successful sub_172 is called as we return from the functions recursively called.
When we traverse a left node, sub_172 is called with 0 whereas when we traverse a right node 1 is passed.
What this function does is build an array of bits starting at 0x1320 in the following manner :
BYTE* bits = 0x1320; BYTE count = 0;
void sub_172( BYTE bit ) { *bits |= bit << count++; if ( count == 8 ) { count = 0; bits++; } } This way, the bit array will represent the path traversed from the leaves to the root for all characters.
When this is done for all input characters, the resulting bit array is compared against the bit array for the correct input at 0x580. So, what we have to do is this :
Extract the correct bit array from 0x580 as bytes.
Reverse the order of the bytes and then convert them to binary representation. We reverse the order because we want to traverse the tree from root to leaf, doing the opposite would be impossible since all bits are concatenated. Also, when doing this, we'll start by extracting the last character and so on until we reach the first.
Below is the IDA Python script that you should run on the extracted ".payload" section to get the flag :
As a result, we get the flag and we see that the VM was expecting a tar file as input:
flag.txt0000664000175000017500000000007113346340766011602 0ustar toshitoshiflag{who would win? 100 ctf teams or 1 obfuscat3d boi?}
Thank you for reading :)
You can follow me on Twitter : here
The purpose of this post is to highlight many of the frustrations I’ve had with Device Guard (rebranded as Windows Defender Application Control) and to discuss why I think it is not an ideal solution for most enterprise scenarios at scale. I’ve spent several years now at this point promoting its use, making it as approachable as possible for people to adopt but from my perspective, I’m not seeing it being openly embraced either within the greater community or by Microsoft (from a public evangelism perspective). Why is that? Hopefully, by calling out the negative experiences I’ve had with it, we might be able to shed a light on what improvements can be made, whether or not further investments should be made in Device Guard, or if application whitelisting is even really feasible in Windows (in its current architecture) for the majority of customer use cases.
In an attempt to prove that I’m not just here to complain for the sake of complaining, here is a non-exhaustive list of blog posts and conference presentations I’ve given promoting Device Guard as a solution:
For me, the appeal of Device Guard (and application whitelisting in general) was and remains as follows: Every… single… malware report I read whether its vanilla crimeware, red team/pentester tools, or nation-state malware has at least one component of their attack chain that would have been blocked and subsequently logged with a robust application whitelisting policy enforced. The idea that a technology could not only prevent, but also supply indications and warnings of well-funded nation-state attacks is extremely enticing. In practice however, at scale (and even on single systems to a lesser extent), both the implementation of Device Guard and the overall ability of the OS to enforce code integrity (particularly in user mode) begin to fall apart.
The Airing of Grievances
Based on my extensive experience working with Device Guard (which includes regularly subverting it), here is what I see as its shortcomings:
An application whitelisting solution that does not supply the ability to create temporary exemptions is unlikely to be a viable solution in the enterprise. This point becomes clear when you consider the following scenario: A new, prospective or current client asks you to join their teleconferencing solution with 30 minutes notice. Telling them that you cannot join because your enforced security solution won’t permit it is simply an unacceptable answer. Some 3rd party whitelisting solutions do permit temporary, quick exceptions to policy and audit accordingly. As a Device Guard expert myself, even if every component of a software package is consistently signed using the same code signing certificate (which is extremely rare), even I wouldn’t be able to build signer rules, update an existing policy, and deploy it in time for the client conference call.
Device Guard is not designed to be placed into audit mode for the purposes of supplementing your existing detections. I recently completed a draft blog post where I was going to highlight the benefits of using Device Guard as an extremely simple and effective means to supplement existing detections. After writing the post however, I discovered that it will only log the loading of an image that would have otherwise been blocked once per boot. This is unacceptable from a threat detection perspective because it would introduce a huge visibility gap. I can only assume that Device Guard in audit mode was only ever designed to facilitate the creation of an enforcement policy.
The only interface to the creation and maintenance of Device Guard code integrity policies is the ConfigCI PowerShell module which only works on Windows 10 Enterprise. As not only a PowerShell MVP and a Device Guard expert, I shamefully still struggle with using this very poorly designed module. If I still struggle with using the module, this doesn’t bode well for non-PowerShell and Device Guard experts.
Feel free to highlight precisely why I’m wrong with supporting evidence but I sense I’m one of the few people outside of Microsoft or even inside Microsoft who have supplied documentation on practical use cases for configuring and deploying Device Guard. The utter absence of others within Microsoft or the community embracing Device Guard at least supplies me with indirect evidence that it not a realistic preventative solution at scale. I’ll further note that I don’t feel that Device Guard was ever designed from the beginning as an enterprise security solution. It has the feel that it simply evolved as an extension of Secure Boot policy from the Windows RT era.
While the servicing efforts for PowerShell constrained language mode have been mostly phenomenal, the servicing of other Device Guard bypasses has been inconsistent at best. For example, this generic bypass still has yet to be fixed. There is an undocumented “Enabled:Dynamic Code Security” policy rule option that is designed to address that bypass (which is great that it's finally being address) but it suffers from a bug that prevents it from working as of Win 10 1803 (it fails to validate the trust of the emitted binary because it forgets to actually mark it as trusted). Additionally, Casey Smith’s “SquiblyTwo” bypass was never serviced, opening the door for additional XSL-based bypasses (which I can confirm exist but I can’t talk about at the time of this writing). Rather, it is just recommended that you blacklist wmic.exe. There is also no robust method to block script-based bypasses.
The strategy with maintaining AppLocker moving forward remains ambiguous. AppLocker still benefits to this day by its ability to apply rulesets to user and groups, unlike Device Guard. It also has a slightly better PowerShell module and a GUI.
Any new features in Device Guard are consistently not documented aside from me occasionally diffing code integrity policy schemas across Windows builds. For example, one of the biggest recent feature additions is the “Enabled:Intelligent Security Graph Authorization” policy rule option which is the feature that actually transformed Device Guard from a pure whitelisting solution to that of an application control solution, yet, it has only a single line mentioning the feature in the documentation.
As far as application whitelisting on Windows is concerned, from a user-mode enforcement perspective, staying on top of blocking new, non-PE based code execution vectors remains an intractable problem. Whether it’s the introduction of code execution vectors (e.g. Windows Subsystem for Linux) or old code execution techniques being rediscovered (e.g. the fact that you can embed arbitrary WSH scripts in XSL docs). People like myself, Casey Smith, Matt Nelson, and many others in the industry recognize the inability of vendors and those implementing application whitelisting solutions to keep pace with blocking/detecting signed applications that permit the execution of arbitrary, unsigned code which fundamentally subvert user mode code integrity (UMCI). This is precisely what motivates us to continue our research in identifying those target applications/scripts.
So what is Device Guard good for then?
What I still love about Device Guard is that it’s the only solution that allows you to apply policy to kernel images (even in the very early boot phase). Regardless of the application whitelisting solution, user mode policy configuration, deployment, and maintenance is really difficult. The appeal of driver enforcement is that Windows requires that all drivers be signed, meaning, the creation of signer rules is relatively straightforward and the set of required drivers is far smaller than the set of required user mode code.
Aside from that, I honestly see very little benefit in using Device Guard for user-mode enforcement or detection aside from using it on systems with extremely consistent hardware and software configurations - e.g. point of sale, ATMs, medical devices, etc.
For the record, I still use Device Guard to enforce kernel and user mode rules on my personal computers. I still cringe, however, any time I have to make updates to my policy, particularly, for software that isn’t signed or is inconsistently signed.
Are you admitting that you wasted the past few years dedicating much of your research time to Device Guard?
Absolutely not!!! I try my best to invest in new security technologies as a motivation to research new abuse and subversion opportunities and Device Guard was no exception. It motivated me to take a deep dive into code signing and signature enforcement which resulted in me learning about and abusing all the internals of subject interface packages and trust providers. It also motivated me to identify and report countless Device Guard and PowerShell Constrained Language Mode bypasses all of which not only bypass application whitelisting solutions but represent attacker tradecraft that subvert many AV/EDR solutions.
I also personally have a hard time blindly accepting the opinions of others (even those who are established, respected experts in their respective domains) without personally assessing the efficacy and limitations of a security solution myself. As a result of all my Device Guard research, I now have a very good sense of what does work and what doesn’t work in an application whitelisting solution. I am very grateful for the opportunity that Device Guard presented to motivate me to learn so much more about code signing validation.
What I’m hopeful for in the future
While I don’t see a lot of investment behind Device Guard compared to other security technologies (like Defender and Advanced Threat Protection), I sense that Microsoft is throwing a lot of their weight behind Windows Defender System Guard runtime attestation, some of the details of which are slowly starting to surface which I’m really excited about assuming the attestation rule engine is extended to 3rd parties. This tweet from Dave Weston I can only assume highlights System Guard in action blocking semi-legitimate signed drivers whereas a relatively simple Device Guard policy would have implicitly blocked those drivers.
Conclusion
My intent is certainly not to dissuade people from assessing the feasibility of Device Guard in your respective environment. Rather, I want to be as open and transparent about the issues I’ve encountered over the years working with it. My hope is to ignite an open and honest conversation about how application whitelisting in Windows can be improved or if it’s even a worthwhile investment in the first place.
As a final note, I want to encourage everyone to dive as deep as you can into technology you’re interested in. There are a lot (I can’t emphasize “a lot” enough) of curmudgeons and detractors who will tell you that you’re wasting your time. Don’t listen to them. Only you (and trusted mentors) should dictate the path of your curiosity! I may no longer be the zealous proponent of application whitelisting that I used to be but I could not be more grateful for the incredible technology Microsoft gave me the opportunity to dive into, upon which, I was able to draw my own conclusions.
A statement in the System Programming Guide of the Intel 64 and IA-32 Architectures Software Developer's Manual (SDM) was mishandled in the development of some or all operating-system kernels, resulting in unexpected behavior for #DB exceptions that are deferred by MOV SS or POP SS, as demonstrated by (for example) privilege escalation in Windows, macOS, some Xen configurations, or FreeBSD, or a Linux kernel crash. The MOV SS and POP SS instructions inhibit interrupts (including NMIs), data breakpoints, and single step trap exceptions until the instruction boundary following the next instruction (SDM Vol. 3A; section 6.8.3). Note that debug exceptions are not inhibited by the interrupt enable (EFLAGS.IF) system flag (SDM Vol. 3A; section 2.3). If the instruction following the MOV SS or POP SS instruction is an instruction like SYSCALL, SYSENTER, INT 3, etc. that transfers control to the operating system at CPL < 3, the debug exception is delivered after the transfer to CPL < 3 is complete. OS kernels may not expect this order of events and may therefore experience unexpected behavior when it occurs.
A detailed white paper describes this behavior here.
Sample code is provided on Github for the Windows Operating System to test if you're vulnerable to CVE-2018-8897. You are free to port it to any other operating systems. A precompiled binary (executable) is provided here for accessibility purposes.
One of the most common questions I’m asked is “what programming language(s) should I learn to get into malware analysis/reverse engineering”, to answer this question I’m going to write about the top 3 languages which I’ve personally found most useful. I’ll focus on native malware (malware which does not require …
In this blog post, I’ll be describing
two bugs I found inside the MalwareFox AntiMalware drivers (zam32.sys/zam64.sys)
that allow a non-privileged process to “authenticate” itself with the driver and issue special IOCTLs leading to privilege
escalation.
This process of registration or authentication is used by the driver to know
which processes to trust when receiving a device control request.Normally, these processes should be the
antimalware’s own processes.
A process that is authenticated by the driver can send special IOCTLs
that cannot be sent by other non-authenticated processes. These special IOCTLs can be used to:
-Enable/Disable real-time protection
-Read/Write to raw disk
-Create full access user-mode process handles
-…etc
Registered processes are stored in an array located in the data section
of the driver. In zam64.sys, each element of the array has 0x980 bytes and the
maximum number of elements is 100. An element contains information on the process
such as its PID, its session id and the name of the image file name from the
EPROCESS structure.
During the run-time of the anti-virus only a single process is
registered with the driver, and that is MalwareFox’s own process ZAM.exe, which runs within session
1. There’s also a ZAM.exe process running as
a Windows service but it doesn’t seem to be
registered.
Figure 1 – MalwareFox’s entry in the
registration array
So, by registering our process with the driver, we enable the god-mode
capability to send special IOCTLs and basically make use of them to escalate privileges
on the system.
I have found two ways to do so.
CVE-2018-6593: Register the process
by connecting to the mini-filter communication port:
As shown in the figure below, a default security descriptor is built for
the mini-filter communication port allowing access only to SYSTEM and the
administrators. But right after that, RtlSetDaclSecurityDescriptor
is called with a NULL DACL pointer. This leads to the DACL pointer, that was
setup by FltBuildDefaultSecurityDescriptor,
being overwritten with NULL. As a result, everyone has access to the object.
In addition, the maximum number of connections allowed to the port are
100 even though only a single connection appears to be needed by the
Antimalware.
Figure 2
And here’s what it looks like under Windbg.
Figure 3
The interesting thing here is that when a process connects to the port,
the driver automatically registers it as a trusted process in the array we saw
above. The figure below displays the first and second entries in the array when our process (exploit.exe) is connected to the port.
Figure 4
Our process is now registered and can send special IOCTLs as it pleases.
It turns out, the developers zeroed to DACL pointer of the port’s security descriptor because their own process (ZAM.exe) doesn’t run with administrator privileges and turns at medium IL. This of
course isn’t an excuse to disable all access
checks and from everything we saw until now this is probably an anti-virus you
don’t want on your machine; and this is not all!
CVE-2018-6606: Registering the process
by sending IOCTL 0x80002010
It turns out there’s a straightforward
way to register a process as trusted. Send IOCTL 0x80002010 with a process id
of your choice and voilà the process with
the PID you supplied is now registered and fully trusted by the driver!!!
What the driver fails to do here is check if the requestor process itself
is a registered process. It does check for this when a process sends a special
IOCTL, but it fails to do so if the process wants to register another process
as trusted; rendering all other checks useless.
Thus, all we need to do to register our process is send IOCTL 0x80002010
with our process’s PID.
Figure 5
We can now send any special IOCTLs we want, and we’ll be using IOCTL 0x8000204C to elevate privileges.
Getting SYSTEM
MalwareFox seems to need user-mode full access handles to processes. And
since we saw how its usermode process lacks the necessary privileges, it delegates this task of opening handles to its driver.
IOCTL 0x8000204C is a special IOCTL that must be sent by a registered
process to the driver. The requestor simply provides a PID as an input and gets
a full access handle as an output from kernel-mode; how cool is that for us ?
We use this opportunity to open a full access handle to winlogon.exe,
inject a cmd.exe shellcode and then create a remote thread.
Figure 6
CVE-2018-6593:
CVE-2018-6606:
A video demonstrating the first bug
(CVE-2018-6593):
After executing the binary it prints : "Input Key:" and waits for us to enter the flag. The routine printing the "Input Key:" message is executed at initialization alongside a sub-routine implementing the ptrace anti-debugging trick. Since we're going to debug the binary, I patched the anti-debugging sub-routine's address with nullsub_1.
After setting up remote debugging under IDA and supplying some random input to the binary we see a call to some code that was stored in executable memory.
IDA sometimes has trouble displaying memory under its debugger, so let's setup a manual memory region and set it as an executable 64-bit segment.
Now we should be able to view the entirety of the bytes copied to memory.
In the figure below, the code that is executed starts by checking if the 4th bit of input[0xC] is set. If it's not set, the message ":(" is printed and the process exits.
However, if the bit is set the code proceeds to decrypt the next block and also XOR the subsequent blocks that are still encrypted. (see figure below)
There's also a second test, implemented in some blocks, which involves the NOT instruction (figure below). This means that the 3rd bit of input[0x11] must not be set.
The amount of code executed is huge and doing this manually is impossible. So, we have two options :
1. Either dump the encrypted data, decrypt it statically and then build the flag by automatically reading the disassembly.
2. Automate the IDA debugger to save the index plus the bits that must be set and guarantee that everything will be executed by continually patching ECX in order not to take the JECXZ jump.
Even if the 2nd attempt would take longer to complete, I chose to implement it. If I ever do the first one, I'll be sharing it here too :)
So, what the script below does is create a dictionary where the key is the character's position and the value is an array of bits that must be set. I simply ignore the NOT case, since we only care about the bits that must be set.
For example if the character at index 2 needs to have bits : 0, 4 and 7 set, the dictionary entry would look like this : {2: [1, 16, 80]}
After the process terminates, we proceed to OR the numbers of each array which gives us the flag in the end.
Here's the script that must be executed under IDA to get the flag.
dont_panic - 100 pts + 15 bonus pts ( 19 solves ):
The file is a GO binary. After locating the main function, by stepping in the debugger, I found that the binary expects a command line argument of 42 characters.
For each character it calls a sub-routine sub_47B760 that returns a calculated byte corresponding to the one supplied. This byte is then compared to the one it corresponds to in a hardcoded array of bytes, which clearly represents the encrypted flag.
I didn't really look into how the values are calculated since GO binaries tend to be messy and I didn't really have to do it in order to solve the challenge. The thing is, the program branches to the block that displays the fail message ("Nope") as soon as it finds that one character is wrong. This opens room for brute-forcing since we can brute-force the flag character by character dynamically.
I used python with GDB to do so. Here's the solution script :
full script : here
Fibonacci - 100 pts + 6 bonus pts ( 45 solves ):
This binary is supposed to print the flag directly into the screen. However, it will take a very very long time to print the whole flag since the output is based on the calculation of fibonacci numbers recursively.
For each bit of the encoded flag (length = 33 stored at 00000000004007E0), the fibonacci number of that bit's position is calculated : this means that it will calculate fibonacci values for numbers from 0 to 263.
This is not all. Since the flag needs to be decoded, each call to the fibonacci sub-routine expects a pointer to a bit value which is XORed with a calculated bit from the resulting fibonacci number. Keep in mind that the fibonacci implementation is recursive, and thus we expects this boolean value to be XORed multiple times for greater numbers.
When the fibonacci sub-routine returns to the main function, the corresponding bit of the encrypted flag is XORed with the calculated bit value.
The solution that came in mind is to modify the fibonacci implementation so as to save both the calculated bit value and the resulting fibonacci number for a given number. So instead of recursing and re-calculating the fibonacci number of a previously calculated one (in a previous call for a previous bit of the flag), we simply load the result of the calculation and XOR the current output bit with the one we already saved.
The solution is implemented in the script below. All modifications done to the original function are commented.
Full script : here
We immediately get the flag when we run the program.
Although the majority of botnets still use a basic client-server model, with most relying on HTTP servers to receive commands, many prominent threats now use more advanced infrastructure to evade endpoint blacklisting and be resilient to take-down. In this article I will go through and explain my process of identifying …
As as a seventh and last assignment of the 32-bit Securitytube Linux Assembly Expert, I have been tasked to create a custom shellcode crypter.
The idea behind a crypter, is to encode the shellcode beforehand and decode it at runtime. This process will make the shellcode looks like random values, and thus aiming to bypass AV and IDS detection.
When it comes to cryptography, it is a well-known wise approach to not try to reinvent the wheel and instead use what is available and well tested: this is done to prevent any new weakness or bug to be introduced in a a freshly written crypto-algorithm.
As a sixth assignment of the 32-bit Securitytube Linux Assembly Expert, I had to create three different polymorphic version of shellcodes taken from ShellStorm.
Here is my selection:
Linux x86 execve(“/bin/sh”) - 28 bytes.
Linux x86 iptables flush - 43 bytes.
Linux x86 ASLR deactivation - 83 bytes.
Polymorphism means that we can mutate shellcode, so while keeping the same functionality the signature is different.
If you'd like to follow along, I'll be using system files from Windows x64 10.0.15063 (Creator's Update). All pseudo-source and disassembly is reconstructed from that specific release.
Don't have a kernel debugging environment set up? Don't fret. You can follow our tutorial on how to setup basic kernel debugging using WinDbg and VMware here.
Without further ado, let's begin.
What do these callbacks do?
These callbacks can be used by driver developers to gain notifications when certain events happen. For example, the basic process creation callback, nt!PsSetCreateProcessNotifyRoutine, registers a user-defined function pointer ("NotifyRoutine") that will be invoked by Windows each time a process is created or deleted. As part of the event notification, the supplied handler gets a wealth of information. In our example, this will include the parent process' (if one exists) PID, the actual process' PID, and a boolean value that will let us know if the process is being created or if it's terminating.
Security software leverages these callbacks to be able to carefully inspect code running on the machine.
Divin' deep
The documented APIs
Our investigation has to begin somewhere. What better place than at the start of a documented function? We turn to nt!PsSetCreateProcessNotifyRoutine. MSDN claims that this routine has been around since Windows 2000. Even our friends at ReactOS seem to have implemented this functionality a long time ago. We'll see exactly how (if at all) things have changed in the 17 years from Windows 2000 until now.
The only difference is in the second parameter being passed to nt!PspSetCreateProcessNotifyRoutine. These are effectively flags. In the base case (nt!PsSetCreateProcessNotifyRoutine), these flags can either be 1 or 0 depending on the state of the "Remove" parameter. If "Remove" is TRUE, Flags=1. If "Remove" is FALSE, Flags=0. In the extended case (nt!PsSetCreateProcessNotifyRoutineEx), the flags can take on the value 2 or 3:
Finally, for nt!PsSetCreateProcessNotifyRoutineEx2, these flags will take on the value 6 or 7:
Therefore, one can imply that the flags passed to nt!PspSetCreateProcessNotifyRoutine have this definition:
The undocumented world
nt!PspSetCreateProcessNotifyRoutine is slightly complicated. I've defined it below, but I strongly recommend opening it in another window and following the text to ease understanding.
Luckily for us, a lot of the internal data structures related to callback routines haven't changed since Windows 2000. The trailblazers at ReactOS have been spot-on with their structure definitions so we'll use them, when possible, to avoid duplicating work.
For each callback, there's a global array that can contain up to 64 entries. In our case, the start of this array for process creation callbacks is located at nt!PspCreateProcessNotifyRoutine. Each entry in this array is of type _EX_CALLBACK:
To avoid synchronization problems, nt!ExReferenceCallBackBlock is used which will safely acquire a reference to the underlying callback object, _EX_CALLBACK_ROUTINE_BLOCK (documented below). We can effectively reproduce the same behavior in a non-thread safe way via:
If we're deleting a callback object ("Remove" is TRUE), we need to make sure that we can find the appropriate _EX_CALLBACK_ROUTINE_BLOCK in the array. This is done by checking first if the target "NotifyRoutine" matches that of the current _EX_CALLBACK_ROUTINE with nt!ExGetCallBackBlockRoutine:
Then, we check to see if it's the right type (created with the correct version of (nt!PsSetCreateProcessNotifyRoutine/Ex/Ex2), by using nt!ExGetCallBackBlockContext:
At this point, we've found the entry in the array. We will erase it by setting the _EX_CALLBACK value to NULL via nt!ExCompareExchangeCallback, decrementing the appropriate global counter (nt!PspCreateProcessNotifyRoutineExCount or nt!PspCreateProcessNotifyRoutineCount), dereferencing the _EX_CALLBACK_ROUTINE_BLOCK with nt!ExDereferenceCallBackBlock, waiting for any other code using the _EX_CALLBACK (nt!ExWaitForCallBacks), and finally freeing memory (nt!ExFreePoolWithTag). As you can see, great care is taken by Microsoft to not free a callback object that is in use.
If we can't find the entry to remove in the nt!PspCreateProcessNotifyRoutine array after exhausting all 64 possibilities, the STATUS_PROCEDURE_NOT_FOUND error message is returned.
On the other hand, if we're adding a new entry into the callback array, things are a little easier. A sanity check is performed by nt!MmVerifyCallbackFunctionCheckFlags to ensure that the "NotifyRoutine" is present in a loaded module. This helps avoid unlinked drivers (or shellcode) from receiving callback events:
After we pass the sanity check, an _EX_CALLBACK_ROUTINE_BLOCK is allocated via nt!ExAllocateCallBack. This routine confirms the size and layout of the _EX_CALLBACK_ROUTINE_BLOCK structure:
To wrap up, the newly allocated _EX_CALLBACK_ROUTINE_BLOCK is added to a free (NULL) location in the nt!PspCreateProcessNotifyRoutine array using nt!ExCompareExchangeCallBack (ensuring that it doesn't overflow the 64 limit maximum). Finally, the appropriate global counter is incremented and a global flag is set in nt!PspNotifyEnableMask denoting that there are callbacks of the user-specified type registered on the system.
The other callbacks
Thankfully, thread and image creation callbacks are very similar to process callbacks. They utilize the same underlying data structures. The only difference is that thread creation/termination callbacks are stored in the nt!PspCreateThreadNotifyRoutine array and that image load notification callbacks are stored in nt!PspLoadImageNotifyRoutine.
The script
It's finally time to put what we know to good use. Using WinDbg, we can create a simple script to automagically enumerate process, thread, and image callback routines.
Instead of leveraging WinDbg's built-in scripting engine, I've elected to use something a little less disgusting. There's a great 3rd party extension for WinDbg called PyKd that enables Python scripting in WinDbg. Installing it is very straightforward. You'll need a copy of the appropriate bitness (e.g. 64-bit for 64-bit install of WinDbg) of Python for this to work.
The script should be easy to follow. I tried to document it as best I could. It should also be compatible, at a minimum, with all forms of Windows from XP and up (both 32-bit and 64-bit flavors).
After running the script using the "!py" command, you should see output similar to this:
Final thoughts
Knowing how the callback system functions in Windows allows us to do very interesting things. As seen above, we're able to programmatically iterate through each callback array and discover all registered callbacks. This is very useful for forensic purposes.
Furthermore, these underlying array lists aren't under the protection of PatchGuard. Since registering callbacks is more-or-less a requirement for anti-virus products in order to develop a useful driver that plays nicely with PatchGuard on x64 systems, malware could dynamically disable (or replace) these registered callbacks to thwart security protection solutions. The possibilities are endless.
Special thanks to the folks at ReactOS for their meticulous documentation. In particular, most of the structures I used were identified by Alex Ionescu for ReactOS a long time ago. Additionally, kudos to the folks that make PyKd. It's a much better alternative to the native scripting interface for WinDbg, in my opinion!
As always, if y'all have any questions or comments, please feel free to comment below. Suggestions are greatly appreciated too!
This blog post will go over an assumption made over a decade ago by Microsoft when dealing with software breakpoints that can be used to reveal the presence of most (all publicly available?) usermode and kernelmode debuggers.
The x86 architecture can potentially encode a particular assembly instruction in multiple ways. For example, adding two registers, eax and ebx, and storing the result in eax takes the following mnemonic form: add eax, ebx. This can be encoded as the byte sequence 0x03 0xC3or 0x01 0xD8. Fundamentally, the machine code represents the same assembly operation.
If you're just interested in the anti-debug trick (without any context on why it works the way it does), scroll to the bottom of this post. For the rest of you brave enough to read this article in its entirety... buckle up.
The "long form" of int 3
An int 3 can be encoded as either a single-byte 0xCC or via the more unconventional way as the multi-byte sequence 0xCD 0x03:
So, what happens when Windows encounters a multi-byte int 3? We create a simple C++ program to find out:
After you run this application, you should see output similar to this:
A single-byte int 3 (0xCC) works as expected. The start of the stub is located at 0x000001BE94B90000. When the stub is executed, the exception handler fires and we see that both the _EXCEPTION_RECORD.ExceptionAddress and _CONTEXT.Rip are located at 0x000001BE94B90000. This is the start of the int 3 instruction. Excellent!
The multi-byte int 3 (0xCD 0x03) is located at address 0x000001BE94B90002. When this stub executes, the exception handler proclaims that the _EXCEPTION_RECORD.ExceptionAddress and _CONTEXT.Rip are located at 0x000001BE94B90003. This is in the middle of the int 3 instruction. Why? What went wrong?
The assumption
NOTE: From this point on, all disassembly and pseudo-source is reconstructed from system files that are provided with Windows x64 10.0.15063 (Creator's Update). If you'd like to follow along, make sure you use the same version I'm using!
Microsoft assumes that all int 3's result from the single-byte variant.
This assumption occurs very early during interrupt processing. Namely, when any interrupt occurs, such as when an int 3 is executed by the processor, control is redirected by the CPU to a handler registered in the appropriate position of the IDT (Interrupt Descriptor Table). In Windows, the handler for software breakpoints can be found at the symbol nt!KiBreakpointTrap:
The first thing nt!KiBreakpointTrap does is generate a trap frame (_KTRAP_FRAME) on the stack that it passes to subsequent routines. A definition of this structure can be found below:
Parts of this structure are automatically filled by the CPU when the interrupt fires, in particular, the range from +0x160 (_KTRAP_FRAME.ErrorCode) to +0x188 (_KTRAP_FRAME.SegSs):
The _KTRAP_FRAME is essentially an extension of the elements saved on the stack by the CPU. It's purpose is to provide a place to store volatile registers which can be clobbered when calling into functions that are compiled in C.
A very important thing to note is that the instruction pointer (EIP) saved by the CPU on the stack (_KTRAP_FRAME.Rip) will be set to the instruction immediately following the one that caused entry into the handler. In our scenario, this means that the _KTRAP_FRAME.Rip member will be the instruction following our int 3, which will be ret (0xC3) in the example code above.
After the volatile registers have been saved off, nt!KiBreakpointTrap performs a quick check to see whether the interrupt fired from usermode (ring3) or kernelmode code (ring0). If execution is coming from ring3, a swapgs needs to occur as well as some other bookkeeping with debug registers.
Eventually, control flow will reconvene and the volatile floating point registers will also be stored off into the _KTRAP_FRAME. Before entering into more exception handling logic, the instruction pointer will be extracted from _KTRAP_FRAME.Rip (saved by the CPU upon entering nt!KiBreakpointTrap), decremented by one, and passed as an argument to nt!KiExceptionDispatch. Additionally, the exception code, EXCEPTION_BREAKPOINT (0x80000003), will also be passed in. The prototype for nt!KiExceptionDispatch:
It's important to note that nt!KiExceptionDispatch (like nt!KiBreakpointTrap) is written in hand-ASM. It assumes that ecx contains the exception code, edx is the number of exception parameters (up to 3), r8 contains the address of the exception, r9 is the first exception parameter (if one exists), r10 is the second exception parameter (if one exists), r11 is the third exception parameter (if one exists), and rbp points to a segment in the _KTRAP_FRAME structure (at offset +0x80).
Upon entry of nt!KiExceptionDispatch, the first thing that occurs is the generation of a _KEXCEPTION_FRAME. Whereas the _KTRAP_FRAME was used to store volatile registers, the _KEXCEPTION_FRAME provides a place to save all nonvolatile registers:
nt!KiExceptionDispatch also creates an _EXCEPTION_RECORD structure on the stack. If you've done any error handling in Windows (in either usermode or kernelmode), you'll be familiar with this data structure as it is contained as a child within the _EXCEPTION_POINTERS data structure. We use both of these structures in our example above.
Furthermore, this explains the first part of our mystery, namely, why the _EXCEPTION_RECORD.ExceptionAddress is incorrect. Recall that the _EXCEPTION_RECORD.ExceptionAddress is populated by the 3rd (r8) argument to nt!KiExceptionDispatch. This was passed in from nt!KiBreakpointTrap. This argument is a copy of the _KTRAP_FRAME.Rip member decremented by one.
To figure out where the _CONTEXT.Rip member is populated, we need to go deeper down the rabbit hole.
nt!KiExceptionDispatch will call into nt!KiDispatchException (yes, the ordering of the words are intentionally flipped) passing in the recently created _EXCEPTION_RECORD and _KEXCEPTION_FRAME:
This function will build a _CONTEXT out of the _KTRAP_FRAME and _KEXCEPTION_FRAME by invoking the helper routine KeContextFromKFrame. After the _CONTEXT is created, a check is made against the _EXCEPTION_RECORD.ExceptionCode (received as an argument from nt!KiExceptionDispatch) for STATUS_BREAKPOINT (0x80000003). If it's true, the _CONTEXT.Rip member will be decremented:
This solves the last part of the mystery and causes the value in _CONTEXT.Rip to be tainted.
The anti-debug trick
Knowing what we know about how Windows handles the different types of int 3s, is it possible to leverage this discrepancy in a useful way? The answer is yes.
Debuggers display the state of the program at the time of an exception. Since Windows will incorrectly assume that our int 3 exception was generated from the single-byte variant, it is possible to confuse the debugger into reading "extra" memory. We leverage this inconsistency to trip a "guard page" of sorts.
As we saw in our first example (at the start of the article), when a multi-byte int 3 occurs, the _EXCEPTION_RECORD.ExceptionAddress and _CONTEXT.Rip values will lie in the middle of our multi-byte instruction instead of at the start. This means that the debugger will incorrectly determine that the instruction which threw the software breakpoint begins with the opcode 0x03. Referring to the trusty Intel manual, we can see that this opcode represents a 2-byte add instruction:
What would happen if we positioned our multi-byte int 3 near the end of a page of memory?
When the operating system notifies our attached debugger of the breakpoint exception, the instruction pointer will point to memory that will be misinterpreted as the start of an add (0x03) instruction. This will cause the debugger to disassemble data on the adjacent page (since this instruction is 2 bytes long), and effectively read one byte past our "valid" memory range.
Our trick relies on the fact that Windows, as an optimization, will not commit virtual memory to physical RAM unless it absolutely needs it. That is to say that most memory, especially in usermode, is paged. When memory needs to be made available for use that is not currently in physical RAM, a page fault occurs. To learn more about memory management, check out the following articles on our site: Introduction to IA-32e hardware paging and Exploring Windows virtual memory management.
So, we can detect the memory read on this adjacent page by inspecting the corresponding PTE (Page Table Entry) using the QueryWorkingSetEx API. If the page is resident in our process' working set (e.g. mapped into our process by the debugger), the Valid bit in the _PSAPI_WORKING_SET_EX_BLOCK will be set.
PoC||GTFO
A full example can be found below:
As always, if you have any questions or comments, please feel free to send us a message below. Happy hacking 😎.
Constrained language mode is an extremely effective method of preventing arbitrary unsigned code execution in PowerShell. It’s most realistic enforcement scenarios are when Device Guard or AppLocker are in enforcement mode because any script or module that is not approved per policy will be placed in constrained language mode, severely limiting an attackers ability to execute unsigned code. Among the restrictions imposed by constrained language mode is the inability to call Add-Type. Restricting Add-Type makes sense considering it compiles and loads arbitrary C# code into your runspace. PowerShell code that is approved per policy, however, runs in “full language” mode and execution of Add-Type is permitted. It turns out that Microsoft-signed PowerShell code calls Add-Type quite regularly. Don’t believe me? Find out for yourself by running the following command:
Now, imagine if the following PowerShell module code (pretend it’s called “VulnModule”) were signed by Microsoft:
$Global:Source=@'
public class Test {
public static string PrintString(string inputString) {
return inputString;
}
}
'@
Add-Type-TypeDefinition$Global:Source
Any ideas on how you might influence the input to Add-Type from constrained language mode? Take a minute to think about it before reading on.
Alright, let’s think the process through together:
Add-Type is passed a global variable as its type definition. Because it’s global, its scope is accessible by anyone, including us, the attacker.
The issue though is that the signed code defines the global variable immediately prior to calling to Add-Type so even if we supplied our own malicious C# code, it would just be overwritten by the legitimate code.
Did you know that you can set read-only variables using the Set-Variable cmdlet? Do you know what I’m thinking now?
Weaponization
Okay, so to inject code into Add-Type from constrained language mode, an attacker needs to define their malicious code as a read-only variable, denying the signed code from setting the global “Source” variable. Here’s a weaponized proof of concept:
public static string ToString(string inputString) {
return inputString;
}
}
'@
Import-ModuleVulnModule
[Injected]::ToString('Injected!!!')
A quick note about weaponization strategies for Add-Type injection flaws. One of the restrictions of constrained language mode is that you cannot call .NET methods on non-whitelisted classes with two exceptions: properties (which is just a special “getter” method) and the ToString method. In the above weaponized PoC, I chose to implement a static ToString method because ToString permits me to pass arguments (a property getter does not). I also made my class static because the .NET class whitelist only applies when instantiating objects with New-Object.
So did the above vulnerable example sound contrived and unrealistic? You would think so but actually Microsoft.PowerShell.ODataAdapter.ps1 within the Microsoft.PowerShell.ODataUtils module was vulnerable to this exact issue. Microsoft fixed this issue in either CVE-2017-0215, CVE-2017-0216, or CVE-2017-0219. I can’t remember, to be honest. Matt Nelson and I reported a bunch of these injection bugs that were serviced by the awesome PowerShell team.
Prevention
The easiest way to prevent this class of injection attack is to supply a single-quoted here-string directly to -TypeDefinition in Add-Type. Single quoted string will not expand any embedded variables or expressions. Of course, this scenario assumes that you are compiling static code. If you must supply dynamically generated code to Add-Type, be exceptionally mindful of how an attacker might influence its input. To get a sense of a subset of ways to influence code execution in PowerShell watch my “Defensive Coding Strategies for a High-Security Environment” talk that I gave at PSConf.EU.
Mitigation
While Microsoft will certainly service these vulnerabilities moving forward, what is to prevent an attacker from bringing the vulnerable version along with them?
A surprisingly effective blacklist rule for UMCI bypass binaries is the FileName rule which will block execution based on the filename present in the OriginalFilename field within the “Version Info” resource in a PE. A PowerShell script is obviously not a PE file though - it’s a text file so the FileName rule won’t apply. Instead, you are forced to block the vulnerable script by its file hash using a Hash rule. Okay… what if there is more than a single vulnerable version of the same script? You’ve only blocked a single hash thus far. Are you starting to see the problem? In order to effectively block all previous vulnerable versions of the script, you must know all hashes of all vulnerable versions. Microsoft certainly recognizes that problem and has made a best effort (considering they are the ones with the resources) to scan all previous Windows releases for vulnerable scripts and collect the hashes and incorporate them into a blacklist here. Considering the challenges involved in blocking all versions of all vulnerable scripts by their hash, it is certainly possible that some might fall through the cracks. This is why it is still imperative to only permit execution of PowerShell version 5 and to enable scriptblock logging. Lee Holmes has an excellent post on how to effectively block older versions of PowerShell in his blog post here.
Another way in which a defender might get lucky regarding vulnerable PowerShell script blocking is due to the fact that most scripts and binaries on the system are catalog signed versus Authenticode signed. Catalog signed means that rather than the script having an embedded Authenticode signature, its hash is stored in a catalog file that is signed by Microsoft. So when Microsoft ships updates, eventually, hashes for old versions will fall out and no longer remain “signed.” Now, an attacker could presumably also bring an old, signed catalog file with them and insert it into the catalog store. You would have to be elevated to perform that action though and by that point, there are a multitude of other ways to bypass Device Guard UMCI. As a researcher seeking out such vulnerable scripts, it is ideal to first seek out potentially vulnerable scripts that have an embedded Authenticode signature as indicated by the presence of the following string - “SIG # Begin signature block”. Such bypass scripts exist. Just ask Matt Nelson.
Reporting
If you find a bypass, report it to [email protected] and earn yourself a CVE. The PowerShell team actively addresses injection flaws, but they are also taking making proactive steps to mitigate many of the primitives used to influence code execution in these classes of bug.
Conclusion
While constrained language mode remains an extremely effective means of preventing unsigned code execution, PowerShell and it’s library of signed modules/scripts remain to be a large attack surface. I encourage everyone to seek out more injection vulns, report them, earn credit via formal MSRC acknowledgements, and make the PowerShell ecosystem a more secure place. And hopefully, as a writer of PowerShell code, you’ll find yourself thinking more often about how an attacker might be able to influence the execution of your code.
Now, everything that I just explained is great but it turns out that any call to Add-Type remains vulnerable to injection due to a design issue that permits exploiting a race condition. I really hope that continuing to shed light on these issues, Microsoft will considering addressing this fundamental issue.
Attackers have been known to apply legitimate digital certificates to their malware, presumably, to evade basic signature validation utilities. This was the case with the Petya ransomware. As a reverse engineer or red team capability developer, it is important to know the methods in which legitimate signatures can be applied to otherwise unsigned, attacker-supplied code. This blog post will give some background on code signing mechanisms, digital signature binary formats, and finally, techniques describing the application of digital certificates to an unsigned PE file. Soon, you will also see why these techniques are even more relevant in research that I will be releasing next month.
Background
What does it mean for a PE file (exe, dll, sys, etc.) to be signed? The simple answer to many is to open up the file properties on a PE and if a “Digital Signatures” tab is present, it means it was signed. When you see that the “Digital Signatures” tab is present on a file, it actually means that the PE file was Authenticode signed, which means within the file itself there is a binary blob of data consisting of a certificate and a signed hash of the file (more specifically, the Authenticode hash which doesn’t consider certain parts of the PE header in the hash calculation). The format in which an Authenticode signature is stored is documented in the PE Authenticode specification.
Many files that one would expect to be signed, however, (for example, consider notepad.exe) do not have a “Digital Signatures” tab. Does this mean that the file isn’t signed and that Microsoft is actually shipping unsigned code? Well, it depends. While notepad.exe does not have an Authenticode signature embedded within itself, in reality, it was signed via another means - catalog signing. Windows contains a catalog store consisting of many catalog files that are basically just a list of Authenticode hashes. Each catalog file is then signed to attest that any files with matching hashes originated from the signer of the catalog file (which is Microsoft in almost all cases). So while the Explorer UI does not attempt to lookup catalog signatures, pretty much any other signature verification tool will perform catalog lookups - e.g. Get-AuthenticodeSignature in PowerShell and Sysinternals Sigcheck.
Note: The catalog file store is located in %windir%\System32\CatRoot\{F750E6C3-38EE-11D1-85E5-00C04FC295EE}
In the above screenshot, the SignatureType property indicates that notepad.exe is catalog signed. What is also worth noting is the IsOSBinary property. While the implementation is not documented, this will show “True” if a signature chains to one of several known, hashed Microsoft root certificates. Those interested in learning more about how this works should reverse the CertVerifyCertificateChainPolicy function.
Sigcheck with the “-i” switch will perform catalog certificate validation and also display the catalog file path that contains the matching Authenticode hash. The “-h” switch will also calculate and display the SHA1 and SHA256 Authenticode hashes of the PE file (PESHA1 and PE256, respectively):
Knowing the Authenticode hash allows you to look up the respective entry in the catalog file. You can double-click a catalog file to view its entries. I also wrote the CatalogTools PowerShell module to parse catalog files. The “hint” metadata field gives away that notepad.exe is indeed the corresponding entry:
Digital Signature Binary Format
Now that you have an understanding of the methods in which a PE file can be signed (Authenticode and catalog), it is useful to have some background on the binary format of signatures. Whether Authenticode signed or catalog signed, both signatures are stored as PKCS #7 signed data which is ASN.1 formatted binary data. ASN.1 is simply a standard that states how binary data of different data types should be stored. Before observing/parsing the bytes of a digital signature, you must first know how it is stored in the file. Catalog files are straightforward as the file itself consists of raw PKCS #7 data. There are online ASN.1 decoders that parse out ASN.1 data and present it in an intuitive fashion. For example, try loading the catalog file containing the hash for notepad.exe into the decoder and you will get a sense of the layout of the data. Here’s a snippet of the parsed output:
Each property within the ASN.1 encoded data begins with an object identifier (OID) - a unique numeric sequence that identifies the type of data that follows. The OIDs worth noting in the above snippet are the following:
1.2.840.113549.1.7.2 - This indicates that what follows is PKCS #7 signed data - the format expected for Authenticode and catalog-signed code.
1.3.6.1.4.1.311.12.1.1 - This indicates that what follows is catalog file hash data
It is worth spending time exploring all of the fields contained within a digital signature. All fields present are outside of the scope of this blog post, however. Additional crypto/signature-related OIDs are listed here.
Embedded PE Authenticode Signature Retrieval
The digital signature data in a PE file with an embedded Authenticode signature is appended to the end of the file (in a well-formatted PE file). The OS obviously needs a little bit more information than that though in order to retrieve the exact offset and size of the embedded signature. Let’s look at kernel32.dll in one of my favorite PE parsing/editing utilities: CFF Explorer.
The offset and size of the embedded digital signature is stored in the “security directory” offset within the “data directories” array within the optional header. The data directory contains offsets and size of various structures within the PE file - exports, imports, relocations, etc. All offsets within the data directory are relative virtual offsets (RVA) meaning they are the offset to the respective portion of the PE when loaded in memory. There is one exception though - the security directory which stores its offset as a file offset. The reason for this is because the Windows loader doesn’t actually load the content of the security directory in memory.
The binary data in the at the security directory file offset is a WIN_CERTIFICATE structure. Here’s what the structure for kernel32.dll looks like parsed out in 010 Editor (file offset 0x000A9600):
PE Authenticode signatures should always have a wRevision of WIN_CERT_TYPE_PKCS_SIGNED_DATA. The byte array that follows is the same PKCS #7, ASN.1 encoded signed data as was seen in the contents of a catalog file. The only difference is that you shouldn’t find the 1.3.6.1.4.1.311.12.1.1 OID, indicating the presence of catalog hashes.
Parsing out the raw bCertificate data in the online ASN.1 decoder confirms we’re dealing with proper PKCS #7 data:
Application of Digital Signatures to Unsigned PEs
Now that you have a basic idea of the binary format and storage locations of digital signatures, you can start applying existing signatures to your unsigned code.
Application of Embedded Authenticode Signatures
Applying an embedded Authenticode signature from a signed file to an unsigned PE file is quite straightforward. While the process can obviously be automated, I’m going to explain how to do it manually with a hex editor and CFF Explorer.
Step #1: Identify the Authenticode signature that you want to steal. In this example, I will use the one in kernel32.dll
Step #2: Identify the offset and size of the WIN_CERTIFICATE structure in the “security directory”
So the file offset in the above screenshot is 0x000A9600 and the size is 0x00003A68.
Step #3: Open kernel32.dll in a hex editor, select 0x3A68 bytes starting at offset 0xA9600, and then copy the bytes.
Step #4: Open your unsigned PE (HelloWorld.exe in this example) in a hex editor, scroll to the end, and paste the bytes copied from kernel32.dll. Take note of the file offset of the beginning of the signature (0x00000E00 in my case). Save the file after pasting in the signature.
Step #5: Open HelloWorld.exe in CFF Explorer and update the security directory to point to the digital signature that was applied: offset - 0x00000E00, size - 0x00003A68. Save the file after making the modifications. Ignore the “Invalid” warning. CFF Explorer doesn’t treat the security directory as a file offset and gets confused when it tries to reference what section the data resides in.
That’s it! Now, signature validation utilities will parse and display the signature properly. The only caveat is that they will report that the signature is invalid because the calculated Authenticode of the file does not match that of the signed hash stored in the certificate.
Now, if you were wondering why the SignerCertificate thumbprint values don’t match, then you are an astute reader. Considering we applied the identical signature, why doesn’t the certificate thumbprint match? That’s because Get-AuthenticodeSignature first attempts a catalog file lookup of kernel32.dll. In this case, it found a catalog entry for kernel32.dll and is displaying the signature information for the signer of the catalog file. kernel32.dll is also Authenticode signed though. To validate that the thumbprint values for the Authenticode hashes are identical, temporarily stop the CryptSvc service - the service responsible for performing catalog hash lookups. Now you will see that the thumbprint values match. This indicates that the catalog hash was signed with a different code signing certificate from the certificate used to sign kernel32.dll itself.
Application of a Catalog Signature to a PE File
Realistically, CryptSvc will always be running and catalog lookups will be performed. Suppose you want to be mindful of OPSEC and match the identical certificate used to sign your target binary. It turns out, you can actually apply the contents of a catalog file to an embedded PE signature by swapping out the contents of bCertificate in the WIN_CERTIFICATE structure and updating dwLength accordingly. Feel free to follow along as this is done. Note that our goal (in this case) is to apply an Authenticode signature to our unsigned binary that is identical to the one used to sign the containing catalog file: Certificate thumbprint AFDD80C4EBF2F61D3943F18BB566D6AA6F6E5033 in this case.
Step #1: Identify the catalog file containing the Authenticode hash of the target binary - kernel32.dll in this case. If a file is Authenticode signed, sigcheck will actually fail to resolve the catalog file. Signtool (included in the Windows SDK) will, however.
Step #2: Open the catalog file in in a hex editor and annotate the file size - 0x000137C7
Step #3: We’re going to manually craft a WIN_CERTIFICATE structure in a hex editor. Let’s go through each field we’ll supply:
dwLength: This is the total length of the WIN_CERTIFICATE structure - i.e. bCertificate bytes plus the size of the other fields = 4 (size of DWORD) + 2 (size of WORD) + 2 (size of WORD) + 0x000137C7 (bCertificate - the file size of the .cat file) = 0x000137CF.
wRevision: This will be 0x0200 to indicate WIN_CERT_REVISION_2_0.
wCertificateType: This will be 0x0002 to indicate WIN_CERT_TYPE_PKCS_SIGNED_DATA.
bCertificate: This will consist of the raw bytes of the catalog file.
When crafting the bytes in the hex editor, be mindful that the fields are stored in little-endian format.
Step #4: Copy all the bytes from the crafted WIN_CERTIFICATE, append them your unsigned PE, and update the security directory offset and size accordingly.
Now, assuming your calculations and alignments were proper, behold a thumbprint match with that of the catalog file!
Anomaly Detection Ideas
The techniques presented in this blog post have hopefully got some people thinking about how one might go about detecting the abuse of digital signatures. While I have not investigated signature heuristics thoroughly, let me just pose a series of questions that might motivate others to start investigating and writing detections for potential signature anomalies:
For a legitimately signed Microsoft PE, is there any correlation between the PE timestamp and the certificate validity period? Would the PE timestamp for attacker-supplied code deviate from the aforementioned correlation?
After reading this article, what is your level of trust in a “signed” file that has a hash mismatch?
How would you go about detecting a PE file that has an embedded Authenticode signature consisting of a catalog file? Hint: A specific OID mentioned earlier might be useful.
How might you go about validating the signature of a catalog-signed file on a different system?
What effect might a stopped/disabled CryptSvc service have on security products performing local signature validation? If that was to occur, then most system files, for all intents and purposes will cease to be signed.
Every legitimate PE I’ve seen is padded on a 0x10 byte boundary. The example I showed where I applied the catalog contents to an Authenticode signature is not 0x10 byte aligned.
How might you differentiate between a legitimate Microsoft digital signature and one where all the certificate attributes are applied to a self-signed certificate?
What if there is data appended beyond the digital signature? This has been abused in the past.
Threat intel professionals should find the Authenticode hash to be an interesting data point when investigating identical code with different certificates applied. VirusTotal supplies this as the "Authentihash" value: i.e. the hash value that was calculated with "sigcheck -h". If I were investigating variants of a sample that had more than one hit on a single Authentihash in VirusTotal, I would find that to be very interesting.
In a previous post, we discussed the IA-32e 64-bit paging structures, and how they can be used to turn virtual addresses into physical addresses. They're a simple but elegant way to manage virtual address mappings as well as page permissions with varying granularity of page sizes. All of which is provided by the architecture. But as one might expect, once you add an operating system like Windows into the mix, things get a little more interesting.
The problem of per-process memory
In Windows, a process is nothing more than a simple container of threads and metadata that represents a user-mode application. It has its own memory so that it can manage the different pieces of data and code that make the process do something useful. Let's consider, then, two processes that both try to read and write from the memory located at the virtual address 0x00000000`11223344. Based on what we know about paging, we expect that the virtual address is going to end up translating into the same physical address (let's say 0x00000001`ff003344 as an example) in both processes. There is, after all, only one CR3 register per processor, and the hardware dictates that the paging structure root is located in that register.
Figure 1: If the two process' virtual addresses would translate to the same physical address, then we expect that they would both see the same memory, right?
Of course, in reality we know that it can't work that way. If we use one process to write to a virtual memory address, and then use another process to read from that address, we shouldn't get the same value. That would be devastating from a security and stability standpoint. In fact, the same permissions may not even be applied to that virtual memory in both processes.
But how does Windows accomplish this separation? It's actually pretty straightforward: when switching threads in kernel-mode or user-mode (called a context switch), Windows stores off or loads information about the current thread including the state of all of the registers. Because of this, Windows is able to swap out the root of the paging structures when the thread context is switched by changing the value of CR3, effectively allowing it to manage an entirely separate set of paging structures for each process on the system. This gives each process a unique mapping of virtual memory to physical memory, while still using the same virtual address ranges as another process. The PML4 table pointer for each user-mode process is stored in the DirectoryTableBase member of an internal kernel structure called the EPROCESS, which also manages a great deal of other state and metadata about the process.
Figure 2: In reality, each process has its own set of paging structures, and Windows swaps out the value of the CR3 register when it executes within that process. This allows virtual addresses in each process to map to different physical addresses.
We can see the paging structure swap between processes for ourselves if we do a little bit of exploration using WinDbg. If you haven't already set up kernel debugging, you should check out this article to get yourself started. Then follow along below.
Let's first get a list of processes running on the target system. We can do that using the !process command. For more details on how to use this command, consider checking out the documentation using .hh !process. In our case, we pass parameters of zero to show all processes on the system.
We can use notepad.exe as our target process, but you should be able to follow along with virtually any process of your choice. The next thing we need to do is attach ourselves to this process - simply put, we need to be in this process' context. This lets us access the virtual memory of notepad.exe by remapping the paging structures. We can verify that the context switch is happening by watching what happens to the CR3 register. If the virtual memory we have access to is going to change, we expect that the value of CR3 will change to new paging structures that represent notepad.exe's virtual memory. Let's take a look at the value of CR3 before the context switch.
We know that this value should change to the DirectoryTableBase member of the EPROCESS structure that represents notepad.exe when we make the switch. As a matter of interest, we can take a look at that structure and see what it contains. The PROCESS fffffa8019218b10 line emitted by the debugger when we listed all processes is actually the virtual address of that process' EPROCESS structure.
The fully expanded EPROCESS structure is massive, so everything after what we're interested in has been omitted from the results above. We can see, though, that the DirectoryTableBase is a member at +0x028 of the process control block (KPROCESS) structure that's embedded as part of the larger EPROCESS structure.
According to this output, we should expect that CR3 will change to 0x00000006`52e89000 when we switch to this process' context in WinDbg.
To perform the context swap, we use the .process command and indicate that we want an invasive swap (/i) which will remap the virtual address space and allow us to do things like set breakpoints in user-mode memory. Also, in order for the process context swap to complete, we need to allow the process to execute once using the g command. The debugger will then break again, and we're officially in the context of notepad.exe.
Okay! Now that we're in the context we need to be in, let's check the CR3 register to verify that the paging structures have been changed to the DirectoryTableBase member we saw earlier.
Looks like it worked as we expected. We would find a unique set of paging structures at 0x00000006`52e89000 that represented the virtual to physical address mappings within notepad.exe. This is essentially the same kind of swap that occurs each time Windows switches to a thread in a different process.
Virtual address ranges
While each process gets its own view of virtual memory and can re-use the same virtual address range as another process, there are some consistent rules of thumb that Windows abides by when it comes to which virtual address ranges store certain kinds of information.
To start, each user-mode process is allowed a user-mode virtual address space ranging from 0x000`00000000 to 0x7ff`ffffffff, giving each process a theoretical maximum of 8TB of virtual memory that it can access. Then, each process also has a range of kernel-mode virtual memory that is split up into a number of different subsections. This much larger range gives the kernel a theoretical maximum of 248TB of virtual memory, ranging from 0xffff0800`00000000 to 0xffffffff`ffffffff. The remaining address space is not actually used by Windows, though, as we can see below.
Figure 3: All possible virtual memory, divided into the different ranges that Windows enforces. The virtual addresses for the kernel-mode regions may not be true on Windows 10, where these regions are subject to address space layout randomization (ASLR). Credits to Alex Ionescu for specific kernel space mappings.
Currently, there is an extremely large “no man's land” of virtual memory space between the user-mode and kernel-mode ranges of virtual memory. This range of memory isn't wasted, though, it's just not addressable due to the current architecture constraint of 48-bit virtual addresses, which we discussed in our previous article. If there existed a system with 16EB of physical memory - enough memory to address all possible 64-bit virtual memory - the extra physical memory would simply be used to hold the pages of other processes, so that many processes' memory ranges could be resident in physical memory at once.
As an aside, one other interesting property of the way Windows handles virtual address mapping is being able to quickly tell kernel pointers from user-mode pointers. Memory that is mapped as part of the kernel has the highest order bits of the address (the 16 bits we didn't use as part of the linear address translation) set to 1, while user-mode memory has them set to 0. This ensures that kernel-mode pointers begin with 0xFFFF and user-mode pointers begin with 0x0000.
A tree of virtual memory: the VAD
We can see that the kernel-mode virtual memory is nicely divided into different sections. But what about user-mode memory? How does the memory manager know which portions of virtual memory have been allocated, which haven't, and details about each of those ranges? How can it know if a virtual address within a process is valid or invalid? It could walk the process' paging structures to figure this out every time the information was needed, but there is another way: the virtual address descriptor (VAD) tree.
Each process has a VAD tree that can be located in the VadRoot member of the aforementioned EPROCESS structure. The tree is a balanced binary search tree, with each node representing a region of virtual memory within the process.
Figure 4: The VAD tree is balanced with lower virtual page numbers to the left, and each node providing some additional details about the memory range.
Each node gives details about the range of addresses, the memory protection of that region, and some other metadata depending on the state of the memory it is representing.
We can use our friend WinDbg to easily list all of the entries in the VAD tree of a particular process. Let's have a look at the VAD entries from notepad.exe using !vad.
The range of addresses supported by a given VAD entry are stored as virtual page numbers - similar to a PFN, but simply in virtual memory. This means that an entry representing a starting VPN of 0x7f and an ending VPN of 0x8f would actually be representing virtual memory from address 0x00000000`0007f000 to 0x00000000`0008ffff.
There are a number of complexities of the VAD tree that are outside the scope of this article. For example, each node in the tree can be one of three different types depending on the state of the memory being represented. In addition, a VAD entry may contain information about the backing PTEs for that region of memory if that memory is shared. We will touch more on that concept in a later section.
Let's get physical
So we now know that Windows maintains separate paging structures for each individual process, and some details about the different virtual memory ranges that are defined. But the operating system also needs a central mechanism to keep track of each individual page of physical memory. After all, it needs to know what's stored in each physical page, whether it can write that data out to a paging file on disk to free up memory, how many processes are using that page for the purposes of shared memory, and plenty of other details for proper memory management
That's where the page frame number (PFN) database comes in. A pointer to the base of this very large structure can be located at the symbol nt!MmPfnDatabase, but we know based on the kernel-mode memory ranges that it starts at the virtual address 0xfffffa80`00000000, except on Windows 10 where this is subject to ASLR. (As an aside, WinDbg has a neat extension for dealing with the kernel ASLR in Windows 10 - !vm 0x21 will get you the post-KASLR regions). For each physical page available on the system, there is an nt!_MMPFN structure allocated in the database to provide details about the page.
Figure 5: Each physical page in the system is represented by a PFN entry structure in this very large, contiguous data structure.
Though some of the bits of the nt!_MMPFN structure can vary depending on the state of the page, that structure generally looks something like this:
A page represented in the PFN database can be in a number of different states. The state of the page will determine what the memory manager does with the contents of that page.
We won't be focusing on the different states too much in this article, but there are a few of them: active, transition, modified, free, and bad, to name several. It is definitely worth mentioning that for efficiency reasons, Windows manages linked lists that are comprised of all of the nt!_MMPFN entries that are in a specific state. This makes it much easier to traverse all pages that are in a specific state, rather than having to walk the entire PFN database. For example, it can allow the memory manager to quickly locate all of the free pages when memory needs to be paged in from disk.
Figure 6: Different linked lists make it easier to walk the PFN database according to the state of the pages, e.g. walk all of the free pages contiguously.
Another purpose of the PFN database is to help facilitate the translation of physical addresses back to their corresponding virtual addresses. Windows uses the PFN database to accomplish this during calls such as nt!MmGetVirtualForPhysical. While it is technically possible to search all of the paging structures for every process on the system in order to work backwards up the paging structures to get the original virtual address, the fact that the nt!_MMPFN structure contains a reference to the backing PTE coupled with some clever allocation rules by Microsoft allow them to easily convert back to a virtual address using the PTE and some bit shifting.
For a little bit of practical experience exploring the PFN database, let's find a region of memory in notepad.exe that we can take a look at. One area of memory that could be of interest is the entry point of our application. We can use the !dh command to display the PE header information associated with a given module in order to track down the address of the entry point.
Because we've switched into a user-mode context in one of our previous examples, WinDbg will require us to reload our symbols so that it can make sense of everything again. We can do that using the .reload /f command. Then we can look at notepad.exe's headers:
Again, the output is quite verbose, so the section information at the bottom is omitted from the above snippet. We're interested in the address of entry point member of the optional header, which is listed as 0x3acc. That value is called a relative virtual address (RVA), and it's the number of bytes from the base address of the notepad.exe image. If we add that relative address to the base of notepad.exe, we should see the code located at our entry point.
And we do see that the address resolves to notepad!WinMainCRTStartup, like we expected. Now we have the address of our target process' entry point: 00000000`ffd53acc.
While the above steps were a handy exercise in digging through parts of a loaded image, they weren't actually necessary since we had symbols loaded. We could have simply used the ? qualifier in combination with the symbol notepad!WinMainCRTStartup, as demonstrated below, or gotten the value of a handy pseudo-register that represents the entry point with r $exentry.
In any case, we now have the address of our entry point, which from here on we'll refer to as our “target” or the “target page”. We can now start taking a look at the different paging structures that support our target, as well as the PFN database entry for it.
Let's first take a look at the PFN database. We know the virtual address where this structure is supposed to start, but let's look for it the long way, anyway. We can easily find the beginning of this structure by using the ? qualifier and poi on the symbol name. The poi command treats its parameter as a pointer and retrieves the value located at that pointer.
Knowing that the PFN database begins at 0xfffffa80`00000000, we should be able to index easily to the entry that represents our target page. First we need to figure out the page frame number in physical memory that the target's PTE refers to, and then we can index into the PFN database by that number.
Looking back on what we learned from the previous article, we can grab the PTE information about the target page very easily using the handy !pte command.
The above result would indicate that the backing page frame number for the target is 0x65207b. That should be the index into the PFN database that we'll need to use. Remember that we'll need to multiply that index by the size of an nt!_MMPFN structure, since we're essentially trying to skip that many PFN entries.
This looks like a valid PFN entry. We can verify that we've done everything correctly by first doing the manual calculation to figure out what the address of the PFN entry should be, and then comparing it to where WinDbg thinks it should be.
So based on the above, we know that the nt!_MMPFN entry for the page we're interested in it should be located at 0xfffffa80`12f61710, and we can use a nice shortcut to verify if we're correct. As always in WinDbg, there is an easier way to obtain information from the PFN database. This can be done by using the !pfn command with the page frame number.
Here we can see that WinDbg also indicates that the PFN entry is at 0xfffffa8012f61710, just like our calculation, so it looks like we did that correctly.
An interlude about working sets
Phew - we've done some digging around in the PFN database now, and we've seen how each entry in that database stores some information about the physical page itself. Let's take a step back for a moment, back into the world of virtual memory, and talk about working sets.
Each process has what's called a working set, which represents all of the process' virtual memory that is subject to paging and is accessible without incurring a page fault. Some parts of the process' memory may be paged to disk in order to free up RAM, or in a transition state, and therefore accessing those regions of memory will generate a page fault within that process. In layman's terms, a page fault is essentially the architecture indicating that it can't access the specified virtual memory, because the PTEs needed for translation weren't found inside the paging structures, or because the permissions on the PTEs restrict what the application is attempting to do. When a page fault occurs, the page fault handler must resolve it by adding the page back into the process' working set (meaning it also gets added back into the process' paging structures), mapping the page back into memory from disk and then adding it back to the working set, or indicating that the page being accessed is invalid.
Figure 7: An example working set of a process, where some rarely accessed pages were paged out to disk to free up physical memory.
It should be noted that other regions of virtual memory may be accessible to the process which do not appear in the working set, such as Address Windowing Extensions (AWE) mappings or large pages; however, for the purposes of this article we will be focusing on memory that is part of the working set.
Occasionally, Windows will trim the working set of a process in response to (or to avoid) memory pressure on the system, ensuring there is memory available for other processes.
If the working set of a process is trimmed, the pages being trimmed have their backing PTEs marked as “not valid” and are put into a transition state while they await being paged to disk or given away to another process. In the case of a “soft” page fault, the page described by the PTE is actually still resident in physical memory, and the page fault handler can simply mark the PTE as valid again and resolve the fault efficiently. Otherwise, in the case of a “hard” page fault, the page fault handler needs to fetch the contents of the page from the paging file on disk before marking the PTE as valid again. If this kind of fault occurs, the page fault handler will likely also have to alter the page frame number that the PTE refers to, since the page isn't likely to be loaded back into the same location in physical memory that it previously resided in.
Sharing is caring
It's important to remember that while two processes do have different paging structures that map their virtual memory to different parts of physical memory, there can be portions of their virtual memory which map to the same physical memory. This concept is called shared memory, and it's actually quite common within Windows. In fact, even in our previous example with notepad.exe's entry point, the page of memory we looked at was shared. Examples of regions in memory that are shared are system modules, shared libraries, and files that are mapped into memory with CreateFileMapping() and MapViewOfFile().
In addition, the kernel-mode portion of a process' memory will also point to the same shared physical memory as other processes, because a shared view of the kernel is typically mapped into every process. Despite the fact that a view of the kernel is mapped into their memory, user-mode applications will not be able to access pages of kernel-mode memory as Windows sets the UserSupervisor bit in the kernel-mode PTEs. The hardware uses this bit to enforce ring0-only access to those pages.
Figure 8: Two processes may have different views of their user space virtual memory, but they get a shared view of the kernel space virtual memory.
In the case of memory that is not shared between processes, the PFN database entry for that page of memory will point to the appropriate PTE in the process that owns that memory.
Figure 9: When not sharing memory, each process will have PTE for a given page, and that PTE will point to a unique member of the PFN database.
When dealing with memory that is shareable, Windows creates a kind of global PTE - known as a prototype PTE - for each page of the shared memory. This prototype always represents the real state of the physical memory for the shared page. If marked as Valid, this prototype PTE can act as a hardware PTE just as in any other case. If marked as Not Valid, the prototype will indicate to the page fault handler that the memory needs to be paged back in from disk. When a prototype PTE exists for a given page of memory, the PFN database entry for that page will always point to the prototype PTE.
Figure 10: Even though both processes still have a valid PTE pointing to their shared memory, Windows has created a prototype PTE which points to the PFN entry, and the PFN entry now points to the prototype PTE instead of a specific process.
Why would Windows create this special PTE for shared memory? Well, imagine for a moment that in one of the processes, the PTE that describes a shared memory location is stripped out of the process' working set. If the process then tries to access that memory, the page fault handler sees that the PTE has been marked as Not Valid, but it has no idea whether that shared page is still resident in physical memory or not.
For this, it uses the prototype PTE. When the PTE for the shared page within the process is marked as Not Valid, the Prototype bit is also set and the page frame number is set to the location of the prototype PTE for that page.
Figure 11: One of the processes no longer has a valid PTE for the shared memory, so Windows instead uses the prototype PTE to ascertain the true state of the physical page.
This way, the page fault handler is able to examine the prototype PTE to see if the physical page is still valid and resident or not. If it is still resident, then the page fault handler can simply mark the process' version of the PTE as valid again, resolving the soft fault. If the prototype PTE indicates it is Not Valid, then the page fault handler must fetch the page from disk.
We can continue our adventures in WinDbg to explore this further, as it can be a tricky concept. Based on what we know about shared memory, that should mean that the PTE referenced by the PFN entry for the entry point of notepad.exe is a prototype PTE. We can already see that it's a different address (0xfffff8a0`09e25a00) than the PTE that we were expecting from the !pte command (0xfffff680007fea98). Let's look at the fully expanded nt!_MMPTE structure that's being referenced in the PFN entry.
We can compare that with the nt!_MMPTE entry that was referenced when we did the !pte command on notepad.exe's entry point.
It looks like the Prototype bit is not set on either of them, and they're both valid. This makes perfect sense. The shared page still belongs to notepad.exe's working set, so the PTE in the process' paging structures is still valid; however, the operating system has proactively allocated a prototype PTE for it because the memory may be shared at some point and the state of the page will need to be tracked with the prototype PTE. The notepad.exe paging structures also point to a valid hardware PTE, just not the same one as the PFN database entry.
The same isn't true for a region of memory that can't be shared. For example, if we choose another memory location that was allocated as MEM_PRIVATE, we will not see the same results. We can use the !vad command to give us all of the virtual address regions (listed by virtual page frame) that are mapped by the current process.
We can take a look at a MEM_PRIVATE page, such as 0x1cf0, and see if the PTE from the process' paging structures matches the PTE from the PFN database.
As we can see, it does match, with both addresses referring to 0xfffff680`0000e780. Because this memory is not shareable, the process' paging structures are able to manage the hardware PTE directly. In the case of shareable pages allocated with MEM_MAPPED, though, the PFN database maintains its own copy of the PTE.
It's worth exploring different regions of memory this way, just to see how the paging structures and PFN entries are set up in different cases. As mentioned above, the VAD tree is another important consideration when dealing with user-mode memory as in many cases, it will actually be a VAD node which indicates where the prototype PTE for a given shared memory region resides. In these cases, the page fault handler will need to refer to the process' VAD tree and walk the tree until it finds the node responsible for the shared memory region.
Figure 12: If the invalid PTE points to the process' VAD tree, a VAD walk must be performed to locate the appropriate _MMVAD node that represents the given virtual memory.
The FirstPrototypePte member of the VAD node will indicate the starting virtual address of a region of memory that contains prototype PTEs for each shared page in the region. The list of prototype PTEs is terminated with the LastContiguousPte member of the VAD node. The page fault handler must then walk this list of prototype PTEs to find the PTE that backs the specific page that has faulted.
Figure 13: The FirstPrototypePte member of the VAD node points to a region of memory that has a contiguous block of prototype PTEs that represent shared memory within that virtual address range.
One more example to bring it all together
It would be helpful to walk through each of these scenarios with a program that we control, and that we can change, if needed. That's precisely what we're going to do with the memdemo project. You can follow along by compiling the application yourself, or you can simply take a look at the code snippets that will be posted throughout this example.
To start off, we'll load our memdemo.exe and then attach the kernel debugger. We then need to get a list of processes that are currently running on the system.
Let's quickly switch back to the application so that we can let it create our initial buffer. To do this, we're simply allocating some memory and then accessing it to make sure it's resident.
Upon running the code, we see that the application has created a buffer for us (in the current example) at 0x000001fe`151c0000. Your buffer may differ.
We should hop back into our debugger now and check out that memory address. As mentioned before, it's important to remember to switch back into the process context of memdemo.exe when we break back in with the debugger. We have no idea what context we could have been in when we interrupted execution, so it's important to always do this step.
When we wrote memdemo.exe, we could have used the __debugbreak() compiler intrinsic to avoid having to constantly switch back to our process' context. It would ensure that when the breakpoint was hit, we were already in the correct context. For the purposes of this article, though, it's best to practice swapping back into the correct process context, as during most live analysis we would not have the liberty of throwing int3 exceptions during the program's execution.
We can now check out the memory at 0x000001fe`151c0000 using the db command.
Looks like that was a success - we can even see the 0xff byte that we wrote to it. Let's have a look at the backing PTE for this page using the !pte command.
That's good news. It seems like the Valid (V) bit is set, which is what we expect. The memory is Writeable (W), as well, which makes sense based on our PAGE_READWRITE permissions. Let's look at the PFN database entry using !pfn for page 0xa1dd0.
We can see that the PFN entry points to the same PTE structure we were just looking at. We can go to the address of the PTE at 0xffffed00ff0a8e00 and cast it as an nt!_MMPTE.
We see that it's Valid, Dirty, Accessed, and Writeable, which are all things that we expect. The Accessed bit is set by the hardware when the page table entry is used for translation. If that bit is set, it means that at some point the memory has been accessed because the PTE was used as part of an address translation. Software can reset this value in order to track accesses to certain memory. Similarly, the Dirty bit shows that the memory has been written to, and is also set by the hardware. We see that it's set for us because we wrote our 0xff byte to the page.
Now let's let the application execute using the g command. We're going to let the program page out the memory that we were just looking at, using the following code:
Once that's complete, don't forget to switch back to the process context again. We need to do that every time we go back into the debugger! Now let's check out the PTE with the !pte command after the page has been supposedly trimmed from our working set.
We see now that the PTE is no longer valid, because the page has been trimmed from our working set; however, it has not been paged out of RAM yet. This means it is in a transition state, as shown by WinDbg. We can verify this for ourselves by looking at the actual PTE structure again.
In the _MMPTE_TRANSITION version of the structure, the Transition bit is set. So because the memory hasn't yet been paged out, if our program were to access that memory, it would cause a soft page fault that would then simply mark the PTE as valid again. If we examine the PFN entry with !pfn, we can see that the page is still resident in physical memory for now, and still points to our original PTE.
Now let's press g again and let the app continue. It'll create a shared section of memory for us. In order to do so, we need to create a file mapping and then map a view of that file into our process.
Let's take a look at the shared memory (at 0x000001fe`151d0000 in this example) using db. Don't forget to change back to our process context when you switch back into the debugger.
And look! There's the 0xff that we wrote to this memory region as well. We're going to follow the same steps that we did with the previous allocation, but first let's take a quick look at our process' VAD tree with the !vad command.
You can see the first allocation we did, starting at virtual page number 0x1fe151c0. It's a Private region that has the PAGE_READWRITE permissions applied to it. You can also see the shared section allocated at VPN 0x1fe151d0. It has the same permissions as the non-shared region; however, you can see that it's Mapped rather than Private.
Let's take a look at the PTE information that's backing our shared memory.
This region, too, is Valid and Writeable, just like we'd expect. Now let's take a look at the !pfn.
We see that the Share Count now actually shows us how many times the page has been shared, and the page also has the Shared property. In addition, we see that the PTE address referenced by the PFN entry is not the same as the PTE that we got from the !pte command. That's because the PFN database entry is referencing a prototype PTE, while the PTE within our process is acting as a hardware PTE because the memory is still valid and mapped in.
Let's take a look at the PTE structure that's in our process' paging structures, that was originally found with the !pte command.
We can see that it's Valid, so it will be used by the hardware for address translation. Let's see what we find when we take a look at the prototype PTE being referenced by the PFN entry.
This PTE is also valid, because it's representing the true state of the physical page. Something interesting to note, though, is that you can see that the Dirty bit is not set. Because this bit is only set by the hardware in the context of whatever process is doing the writing, you can theoretically use this bit to actually detect which process on a system wrote to a shared memory region.
Now let's run the app more and let it page out the shared memory using the same technique we used with the private memory. Here's what the code looks like:
Let's take a look at the memory with db now.
We see now that it's no longer visible in our process. If we do !pte on it, let's see what we get.
The PTE that's backing our page is no longer valid. We still get an indication of what the page permissions were, but the PTE now tells us to refer to the process' VAD tree in order to get access to the prototype PTE that contains the real state. If you recall from when we used the !vad command earlier in our example, the address of the VAD node for our shared memory is 0xffffa50d`d2313a20. Let's take a look at that memory location as an nt!_MMVAD structure.
The FirstPrototypePte member contains a pointer to a location in virtual memory that stores contiguous prototype PTEs for the region of memory represented by this VAD node. Since we only allocated (and subsequently paged out) one page, there's only one prototype PTE in this list. The LastContiguousPte member shows that our prototype PTE is both the first and last element in the list. Let's take a look at this prototype PTE as an nt!_MMPTE structure.
We can see that the prototype indicates that the memory is no longer valid. So what can we do to force this page back into memory? We access it, of course. Let's let the app run one more step so that it can try to access this memory again.
Remember to switch back into the context of the process after the application has executed the next step, and then take a look at the PTE from the PFN entry again.
Looks like it's back, just like we expected!
Exhausted yet? Compared to the 64-bit paging scheme we talked about in our last article, Windows memory management is significantly more complex and involves a lot of moving parts. But at it's core, it's not too daunting. Hopefully, now with a much stronger grasp of how things work under the hood, we can put our memory management knowledge to use in something practical in a future article.
If you're interested in getting your hands on the code used in this article, you can check it out on GitHub and experiment on your own with it.
Microsoft has a great track record of maintaining support for legacy software running under Windows. There is an entire compatibility layer baked into the OS that is dedicated to fixing issues with decades old software running on modern iterations of Windows. To learn more about this application compatibility infrastructure, I'd recommend swinging over to Alex Ionescu's blog. He has a great set of posts describing the technical details on how user (even kernel) mode shimming is implemented.
With all of that said, it's an understatement to say that Microsoft takes backwards compatibility seriously. Occasionally, the humans at Microsoft make mistakes. Usually, though, they're very quick to address these problems.
This blog post will go over an unnoticed bug that was introduced in Windows 8 with a documented Win32 API. At the time of this post, this bug is still present in Windows 10 (Creator's Update) and has been around for over 5 years.
Forgotten Win32 APIs
There is a set of Win32 APIs that were introduced in Windows XP to monitor the working set of a process. A process' working set is a collection of pages, chunks of memory, that are currently in RAM (physical memory) and are accessible to that process without inducing a page fault. In particular, the APIs of interest for us are InitializeProcessForWsWatch and GetWsChanges/GetWsChangeEx.
After reading the MSDN documentation, it's easy to discover what the intended use for these APIs were. These APIs profile the number of page faults that occur within a process' address space.
What's a page fault? A quick recap.
There are 3 general categories of page faults.
A hard page fault occurs when memory is accessed that's not currently in RAM (physical). In situations like this, the OS will need to retrieve the memory from disk (e.g. pagefile.sys) and make it accessible to the faulting process.
A soft page fault occurs when memory is in RAM (physical), but not currently accessible to the process that induced the fault. This memory might be shared amongst multiple processes and the process that caused the page fault might not have it mapped into its working set. These types of page faults are much more performant than hard page faults as there is no disk I/O conducted.
The last and final type of page fault is known formally as an invalid fault. These can also be referred to as access violations. This can be caused when a program, for example, tries to access unallocated memory or tries to write to memory that's marked read-only.
Paging is necessary to make modern operating systems work. You probably have many processes running on your system, but not nearly enough RAM to hold all the possible contents of each process into physical memory. To learn more about paging, I strongly recommend this article posted by my colleague.
Demo
The best way to illustrate what's broken is through an example. I created two simple programs.
The first application, WorkingSetWatch.exe, implements the InitializeProcessForWsWatch and GetWsChangeEx APIs. This application logs when a specific memory region is paged into our process' working set:
The second application, ReadProcessMemory.exe, implements reading of an arbitrary memory blob from another target process' memory space:
The basic idea is to use ReadProcessMemory.exe to read from the monitored memory address inside of WorkingSetWatch.exe. This will induce a page fault.
Windows 7: Build 7601 (SP1)
The WorkingSetWatch.exe application works as expected. We're able to read any (valid) sized buffer using ReadProcessMemory.exe and log it.
Windows 10: Build 15063 (Creator's Update)
Unfortunately, WorkingSetWatch.exe does not seem to log the page fault that occurs when our remote application, ReadProcessMemory.exe, reads a buffer greater than or equal to 512 bytes; however, it does seem to work as expected when a read occurs that's less than 512 bytes.
This renders these working set APIs useless for profiling reasons on Windows 8+.
What went wrong?
To determine what went wrong, we'll need to reverse engineer parts of Windows and see exactly how the implementation changed in Windows 8+ from Windows 7.
All disassembly and pseudo-source is reconstructed from system files that are provided with Windows x64 10.0.15063 (Creator's Update).
Enabling process working set logging
To enable working set logging for a process, we need to call InitializeProcessForWsWatch. From the MSDN documentation, we're told that on newer versions of Windows this API is exported as K32InitializeProcessForWsWatch within kernel32.dll. Our analysis begins there:
This function is very simple. It invokes an import from another library. In this case, it executes a function of the same name (K32InitializeProcessForWsWatch), but contained within a different library, api-ms-win-core-psapi-l1-1-0.dll. This library doesn't exist on disk, but rather resolves to an API Set mapping corresponding to kernelbase.dll (which does exist on disk) for this version of Windows. A look into kernelbase.dll's implementation shows that a call to NtSetInformationProcess is performed without any parameter marshalling:
Our next target is NtSetInformationProcess within ntdll.dll:
This is just a simplistic syscall stub that will eventually make its way into the implementation contained within ntoskrnl.exe, the Windows kernel. nt!NtSetInformationProcess is a massive function that contains a huge switch statement that supports all the different PROCESSINFOCLASS that can be passed to it.
We're interested in the PROCESSINFOCLASS for ProcessWorkingSetWatch. This is case 15 (0xF). A snippet of the relevant parts (with the cleaned-up disassembly):
It's interesting to note that you're able to start monitoring on a process' working set with either a class of ProcessWorkingSetWatch (15) or ProcessWorkingSetWatchEx (42). This can be achieved by invoking nt!NtSetInformationProcess directly instead of going through the documented route with kernel32!InitializeProcessForWsWatch. The latter utilizes only the ProcessWorkingSetWatch class.
The actual logic of nt!NtSetInformationProcess is pretty trivial to understand. A blob of memory is allocated per process that we're monitoring. This blob of memory is a _PAGEFAULT_HISTORY structure and contains up to 1024 _PROCESS_WS_WATCH_INFORMATION structures internally. Each _PROCESS_WS_WATCH_INFORMATION structure is an entry that describes a page fault. These entries will be cycled through as the array fills up. Recall from the MSDN documentation (the "Remarks" section) that you must call GetWsChanges/Ex with enough frequency to avoid record loss. This makes perfect sense because we can see that there are a fixed number of these records (1024) allocated. I took the liberty of documenting these structures:
The union at the beginning of the _PAGEFAULT_HISTORY structure may be a little confusing, but it'll be explained later.
On successful execution of this routine, the monitored process object will have an internal member (_EPROCESS.WorkingSetWatch) updated to include this recently allocated _PAGEFAULT_HISTORY pointer. Additionally, the PsWatchEnabled global will be set. This value informs the system to track page faults for processes. It will remain set until the system reboots (even if there are no processes running that have working sets tracked). There are only 2 references to PsWatchEnabled and we've already inspected the one in nt!NtSetInformationProcess.
Our investigation leads us to nt!KiPageFault.
Logging a page fault
When a page fault occurs, the CPU transfers execution to nt!KiPageFault:
If the PsWatchEnabled global is set, that means we've enabled working set logging for processes on the system and execution is passed to nt!PsWatchWorkingSet. This function is documented below:
As I mentioned above, there are 3 types of page faults. Access violations are not logged to our process' working set due to an early out by nt!MmAccessFault in nt!KiPageFault. Since this function is executed for the other 2 types of page faults (hard and soft) on the system, it will be accessed heavily by the operating system. Luckily, one of the first things the routine does is check whether or not a working set watch was enabled on the process where the page fault occurred. If there is no working set watch on the process, the routine completes.
As per the documentation, nt!PsWatchWorkingSet will not function while records are being processed (EntrySelector.Busy). We'll describe this part in depth at a later time. Since higher priority interrupts can preempt our working set monitor, most of the logic in this routine needs to have adequate sanity (safety) checks and complete as atomically (Interlocked*** operations) as possible. The first part of the function will safely select a free index in the _PAGEFAULT_HISTORY.WatchInfo array that it can use for logging purposes. If the array is full (there can be at most 1024 entries), a "miss" is recorded (_PAGEFAULT_HISTORY.MissingRecords) and the routine completes. If everything is successful, a page fault event is recorded in a free slot in the _PAGEFAULT_HISTORY.WatchInfo array. An interesting (and undocumented) feature changes the entry's _PROCESS_WS_WATCH_INFORMATION.FaultingVa least significant bit to 0 if a hard page fault occurred and 1 if a soft page fault occurred.
Ultimately, there doesn't seem to be any apparent bugs with this code. Additionally, this code matches very closely to the Windows 7 version which we know works. Our investigation leads us to the working set watch retrieval functions: GetWsChanges/Ex.
Querying working set logging
For article brevity, I'll give a quick summary of the call-flow of kernel32!GetWsChanges (kernel32!K32GetWsChanges) and kernel32!GetWsChangesEx (kernel32!K32GetWsChangesEx). These functions will call into their kernelbase.dll variants. From there, they will branch into kernelbase!GetWsChangesInternal which will invoke ntdll!NtQueryInformationProcess with the appropriate PROCESSINFOCLASS. In particular, the ProcessWorkingSetWatch class will be used for the GetWsChanges family of functions and ProcessWorkingSetWatchEx will be used for the others. From ntdll!NtQueryInformationProcess, a syscall will be made. This makes it to the implementation of NtQueryInformationProcess within the kernel. A massive switch statement awaits:
The part that interests us resides one level deeper within nt!PspQueryWorkingSetWatch:
There's some input validation (e.g. alignment checks) and a safety check (nt!ExIsRestrictedCaller) to avoid kernel pointer leaks in low integrity processes. After that, the process object is retrieved from the supplied process handle. The operating system checks to see that the _EPROCESS.WorkingSetWatch member is set. Just like the documentation states, at most one query can access a process' working set buffer at a time (EntrySelector.Busy). Additionally, while the buffer is being accessed, logging (by nt!PsWatchWorkingSet in nt!KiPageFault) will produce misses.
As long as there's enough space in the user supplied buffer, the operating system will copy over the entry array to the user supplied buffer. The data will be structured in the appropriate way for the appropriate PROCESSINFOCLASS. The last entry in the user supplied buffer (PSAPI_WS_WATCH_INFORMATION/EX) will be terminated with a FaultingPc member of NULL. Additionally, the number of "misses" will be recorded in the FaultingVa member of the last entry.
Finally, the _PAGEFAULT_HISTORY.WatchInfo array of the _EPROCESS.WorkingSetWatch will be reset after a successful call.
/rant.
The InitializeProcessForWsWatch and GetWsChanges/Ex APIs are surprisingly very finicky. There are many weird restrictions and caveats which make it surprisingly difficult for developers to retrieve information regarding the complete set of page faults that occurred within a process.
There is a very good chance that you will run into situations where records will wind up missing especially in a multi-processor and multi-threaded environment. For example, if a thread is querying the working set of a process, but a page fault occurs on another thread within that same process, a miss could be recorded since the _PAGEFAULT_HISTORY.Busy member will be acquired by nt!PspQueryWorkingSetWatch. This will prevent the page fault logging logic in nt!PsWatchWorkingSet. Functionally, this weakens the usability of the API for profiling purposes. To compound this problem, only 1024 entries can be stored in the array between calls of GetWsChanges/Ex. That's at most 4 MB (1024*PAGE_SIZE) of page fault history. This really isn't enough for modern applications which can be very complex.
In our specific situation, we ran our tests on a VM that had 1 processor allocated to it. Furthermore, our application was simple enough that it had 1 thread. This mitigates the chance of page fault "misses". Additionally, after a thorough investigation of the working set APIs, we've concluded that we've still not discovered where the bug is. In particular, why does the buffer size play a role in the success of these APIs? In our demo, we were unable to log page faults on Windows 10 when the buffer size was greater than or equal to 512 bytes. Is it possible that the bug is not within WorkingSetWatch.exe, but rather ReadProcessMemory.exe?
To continue our investigation, we need to turn to ReadProcessMemory.exe.
Reading memory
The ReadProcessMemory.exe application is simple enough to understand. We know that we're not logging a page fault when we're reading a buffer that is greater than or equal to 512 bytes. Since there is no apparent bug in the working set APIs, the problem most likely resides in kernel32!ReadProcessMemory.
I'll step past the irrelevant details, but the same strategy is applied as was in the previous parts. In particular, kernel32!ReadProcessMemory calls into kernelbase!ReadProcessMemory. These functions do nothing special and more-or-less directly issue a system call by invoking ntdll!NtReadVirtualMemory. This takes us to the implementation of nt!ReadVirtualMemory in the kernel:
This function just invokes nt!MiReadWriteVirtualMemory. On some versions of ntoskrnl, this routine may just be inlined into the caller's body.
Aside from a check that prevents reading and writing to protected processes (ProcessObject->Pcb.SecurePid), this function is nearly identical to the one in the Windows 7 kernel. We need to go deeper. We traverse into nt!MmCopyVirtualMemory.
This function is massive. It contains many subfunctions that have been inlined. For article brevity, the important parts of nt!MmCopyVirtualMemory will be highlighted. One of the first things that this routine does is search for VAD entries that corresponds to the input addresses (FromAddress and ToAddress). The idea is to leverage the "region size" information for memory, but this isn't really relevant to our bug. We'll leave the discussion of the VAD (Virtual Address Descriptor) to another time.
nt!MmCopyVirtualMemory's next task is to determine the input buffer's length. In particular, there are a couple checks against the buffer length and the value 512. This is significant to us because we know the bug only seems to manifest when the buffer size is greater than or equal to 512 bytes.
Basically, it seems that if the buffer is greater than or equal to 512 bytes, nt!MmCopyVirtualMemory will utilize nt!MmProbeAndLockPages and nt!MmMapLockedPagesSpecifyCache followed by a memcpy to clone over memory.
If the buffer is less than 512 bytes, nt!MmCopyVirtualMemory will just leverage memcpy directly by using a buffer on the stack or a buffer allocated in dynamic memory (based on buffer size) via nt!ExAllocatePoolWithTag.
This is probably done for performance reasons. Larger memory copies probably benefit from direct mapping instead of memory pool copying. If we do leverage memory pool copying (buffers that are less than 512 bytes in size), we trigger a page fault and the event is logged by our WorkingSetWatch.exe application. On the other hand, if we leverage a direct mapping to copy memory, we do not trigger a page fault.
One incorrect assumption is to believe that on Windows 7 this optimization did not exist. On the contrary, there is very similar logic inside of the older version of nt!MmCopyVirtualMemory. However, something did change, otherwise we would not have any discrepancies with our WorkingSetWatch program. Our investigation leads us into nt!MmProbeAndLockPages.
The bug: an optimization in nt!MmProbeAndLockPages
The implementation of nt!MmProbeAndLockPages underwent drastic changes between Windows 7 to now. If you looked at these two functions side-by-side, you'd quickly notice that the Windows 7 implementation was in some ways much simpler.
The purpose of nt!MmProbeAndLockPages (per the documentation) is to ensure that the specified virtual pages (in the argument contained within MemoryDescriptorList) are backed by physical memory. Additionally, there is a series of permission checks to ensure that the virtual pages permit the user-specified access rights. In Windows 7, to perform this access check, the routine actually "probed" the memory by directly accessing it. This would induce a page fault in the context of the correct process and therefore we'd be able to log it using our WorkingSetWatch.exe application.
On Windows 10, this process was optimized. Instead of accessing the memory directly, a PTE (Page Table Entry) walk is performed to ensure that the correct permissions exist. This change makes the process more efficient especially since the PTEs are leveraged to lock the memory into physical pages anyway.
OS development isn't easy
One seemingly inconspicuous change can break functionality in an entirely unrelated part of the operating system. In our case, an optimization in the underlying logic of how nt!MmProbeAndLockPages functioned broke backwards compatibility of the working set APIs. This bug seems to be entirely unnoticed, but it unfortunately renders the performance profiling nature of the GetWsChanges/Ex APIs useless.
A potential fix for Microsoft is to simply just throw a page fault for "invalid" pages if the PsWatchEnabled global is set or, more granularly, if a process' _EPROCESS.WorkingSetWatch is set.
This post will describe a Device Guarduser mode code integrity (UMCI) bypass (or any other application whitelisting solution for that matter) that takes advantage of the fact the code integrity checks are not performed on any code that compiles C# dynamically with csc.exe. This issue was reported to Microsoft on November 14, 2016. Despite all other Device Guard bypasses being serviced, a decision was made to not service this bypass. This bypass can be mitigated by blocking csc.exe but that may not be realistic in your environment considering the frequency in which legitimate code makes use of these methods - e.g. msbuild.exe and many PowerShell modules that call Add-Type.
Introduction
When Device Guard enforces user mode code integrity (UMCI), aside from blocking non-whitelisted binaries, it also only permits the execution of signed scripts (PowerShell and WSH) approved per policy. The UMCI enforcement mechanism in PowerShell is constrained language mode. One of the features of constrained language mode is that unsigned/unapproved scripts are prevented from calling Add-Type as this would permit arbitrary code execution via the compilation and loading of supplied C#. Scripts that are approved per Device Guard code integrity (CI) policy, however, are under no such restrictions, execute in full language mode, and are permitted to call Add-Type. While investigating Device Guard bypasses, I considered targeting legitimate, approved calls to Add-Type. I knew that the act of calling Add-Type caused csc.exe – the C# compiler to drop a .cs file to %TEMP%, compile it, and load it. A procmon trace of PowerShell calling Add-Type confirms this:
Upon seeing these files created, I asked myself the following questions:
Considering an approved (i.e. whitelisted per policy) PowerShell function is permitted to call Add-Type (as many Microsoft-signed module functions do), could I possibly replace the dropped .cs file with my own? Could I do so quickly enough to win that race?
How is the .DLL that’s created loaded? Is it subject to code integrity (CI) checks?
Research methodology
Let’s start with the second question since exploitation would be impossible if CI would prevent the loading of a hijacked, unsigned DLL. To answer this question, I needed to determine what .NET methods were called upon Add-Type being called. This determination was relatively easy by tracing method calls in dnSpy. I quickly traced execution of the following .NET methods:
Once the Microsoft.CSharp.CSharpCodeGenerator.Compile method is called, this is where csc.exe is ultimately invoked. After the Compile method returns, FromFileBatch takes the compiled artifacts, reads them in as a byte array, and then loads them using System.Reflection.Assembly.Load(byte[], byte[], Evidence). This is the same method called by msbuild.exe when compiling inline tasks – a known Device Guard UMCI bypassed discovered by Casey Smith. Knowing this, I gained the confidence that if I could hijack the dropped .cs file, I would end up having a constrained language mode bypass, allowing arbitrary unsigned code execution. What we’re referring to here is known as a “time of check time of use” (TOCTOU) attack. If I could manage to replace the dropped .cs file with my own prior to csc.exe consuming it, then I would win that race and perform the bypass. The only constraints imposed on me, however, would be that I would need to write a hijack payload within the constraints of constrained language mode. As it turns out, I was successful.
Exploitation
I wrote a function called Add-TypeRaceCondition that will accept attacker-supplied C# and get an allowed call to Add-Type to compile it and load it within the constraints of constrained language mode. The weaponized bypass is roughly broken down as follows:
Spawn a child process of PowerShell that constantly tries to drop the malicious .cs file to %TEMP%.
Maximize the process priority of the child PowerShell process to increase the likelihood of winning the race.
In the parent PowerShell process, import a Microsoft-signed PowerShell module that calls Add-Type – I chose the PSDiagnostics process for this.
Kill the child PowerShell process.
At this point, you will have likely won the race and your type will be loaded in place of the legitimate one expected by PSDiagnostics.
In reality, the payload wins the race a little more than 50% of the time. If Add-TypeRaceCondition doesn’t work on the first try, it will almost always work on the second try.
Do note that while I weaponized this bypass for PowerShell, this can be weaponized using anything that would allow you to overwrite the dropped .cs file quickly enough. I've weaponized the bypass using a batch script, VBScript, and with WMI. I'll leave it up to the reader to implement a bypass using their language of choice.
Operational Considerations
It's worth noting that while an application whitelisting bypass is just that, it also serves as a method of code execution that is likely to evade defenses. In this bypass, an attacker need only drop a C# file to disk which results in the temporary creation of a DLL on disk which is quickly deleted. Depending upon the payload used, some anti-virus solutions with real-time scanning enabled could potentially have the ability to quarantine the dropped DLL before it's consumed by System.Reflection.Assembly.Load.
Prevention
Let me first emphasize that this is a .NET issue, not a PowerShell issue. PowerShell was simply chosen as a convenient means to weaponize the bypass. As I’ve already stated, this issue doesn’t just apply to when PowerShell calls Add-Type, but when any application calls any of the CodeDomProvider.CompileAssemblyFrom methods. Researchers will continue to target signed applications that make such method calls until this issue is mitigated.
A possible user mitigation for this bypass would be to block csc.exe with a Device Guard rule. I would personally advise against this, however, since there are many legitimate Add-Type calls in PowerShell and presumably in other legitimate applications. I’ve provided a sample Device Guard CI rule that you can merge into your policy if you like though. I created the rule with the following code:
# Copy csc.exe into the
following directory
# csc.exe should be the
only file in this directory.
Unfortunately, detection using free, off-the-shelf tools will be difficult due to the fact that the disk artifacts are created and subsequently deleted and by the nature of System.Reflection.Assembly.Load(byte[]) not generating a traditional module load event that something like Sysmon would be able to detect.
Vendors with the ability to hash files on the spot should consider assessing the prevalence of DLLs created by csc.exe. Files with low prevalence should be treated as suspicious. Also, unfortunately, since dynamically created DLLs by their nature will not be signed, there will be no code signing heuristics to key off of.
It's worth noting that I intentionally didn't mention PowerShell v5 ScriptBlock logging as a detection option since PowerShell isn't actually required to achieve this bypass.
Conclusion
I remain optimistic of Device Guard’s ability to enforce user mode code integrity. It is a difficult problem to tackle, however, and there is plenty of attack surface. In most cases, Device Guard UMCI bypasses can be mitigated by a user in the form of CI blacklist rules. Unfortunately, in my opinion, no realistic user mitigation of this particular bypass is possible. Microsoft not servicing such a bypass is the exception and not the norm. Please don’t let this discourage you from reporting any bypasses that you may find to [email protected]. It is my hope that by releasing this bypass that it will eventually be addressed and it will provide other vendors with the opportunity to mitigate.
This article assumes you've read the first part of the series. In particular, at this point you should have successfully setup VMware's GDB stub and IDA Pro's GDB debugger. You should now be in a connected state and broken into IDA Pro's debugger GUI.
Furthermore, the focus of this post is going to be exclusively on loading kernel symbols for 64-bit editions of Windows (AMD64). Different operating systems (and different architectures of Windows) require slight modifications to the article's logic.
Where's Waldo ntoskrnl?
The end goal
The first and most important thing is to discover where the NT Kernel (ntoskrnl.exe) is loaded in memory since it's not at any fixed (static) address thanks to address space layout randomization (ASLR).
We are then able to force IDA Pro to load symbol data (PDBs) at ntoskrnl's base address to have useful debugging information. From there, we can enumerate the linked list, nt!PsLoadedModuleList, to figure out where other kernel mode components are located. However, this isn't trivial. When you break in to IDA Pro's GDB debugger, it's difficult to know what state you'll be in on any given processor. You might be executing code in a usermode process, or you might be busy servicing a system call. Additionally, you're further restricted to the functionality the GDB stub exposes.
Enter the _KPCR
On all architectures and versions of Windows, each processor maintains a control structure dubbed as the _KPCR (Kernel Processor Control Region). This structure is massive and it can be used to infer exactly what the processor is doing. On Windows 10 (15063.0.amd64fre.rs2_release.170317-1834), the _KPCR is 0x6bc0 bytes large. It contains many kernel pointers that we can leverage to figure out exactly where the base of ntoskrnl is in memory. A link detailing the members of the _KPCR can be found here.
This structure can be accessed through its virtual address or through the fs segment on x86 and the gs segment on x64. In fact, if you've done any reverse engineering of the Windows kernel, you should have seen many examples of Windows itself accessing members of the _KPCR through the segment selector.
For example, when an int 3 (a software breakpoint; 0xCC) is executed by the processor, control is redirected by the CPU to a handler registered in the appropriate position of the IDT (Interrupt Descriptor Table). We'll touch more on this process later. In Windows, the handler for software breakpoints is nt!KiBreakpointTrap. Here is a snippet of the assembly code of the handler under AMD64:
In particular, at address 0x00000001401749FD we see a swapgs instruction. Since the gs selector means different things in user-mode (_TEB) and the kernel (_KPCR), this instruction is utilized to ensure that we're operating on the kernel-mode construct (_KPCR). Immediately following that instruction at address 0x0000000140174A00, we have an access of the gs segment with a mov r10, gs:188h. The astute reader will realize that upon execution of this instruction, r10 will contain the pointer from the _KPCR.Prcb.CurrentThread. This is discerned from the definition of the structure's members posted above. A breakdown of this process can be illustrated below:
We don't know the _KPCR's exact linear address (it too isn't allocated at a fixed location), but we should be able to access it through the segment selector, though, just like the Windows kernel does. This approach might seem like the ideal one, but, unfortunately, we're further restricted by the functionality of the GDB stub. Let's see what the GDB stub exposes by issuing help:
There are only three major commands available: help, r, and linuxoffsets. We've just executed help, and linuxoffsets isn't relevant to us since we're debugging a Windows kernel. The only other command is r. At first, r looks very useful to us. However, on closer examination, we can see that the GDB stub is unable to read arbitrary offsets off of the gs selector, e.g. the _KPCR.Prcb.CurrentThread from gs:188h by executing r gs:188h.
At least executing r gs without an offset produces data:
This command should get us the base of the gs selector. We then should be able to define a _KPCR structure at that location using IDA Pro. According to the GDB stub, though, our base is 0. If we go to that memory location in the "IDA View - RIP" tab by pressing 'G' and entering 0 in the "Jump to address" window, we don't see anything there:
What changed from x86 to x64?
If you ran this test on a VM running on an x86 (32-bit) version of Windows and substituted fs for gs, the base of the fs selector would not be 0. It would be a valid memory location. You would then have the address of the _KPCR and could continue on your merry way.
Unfortunately, you're a sucker for pain and are following this tutorial down to the T. In 64-bit (long) mode on x64, the cs, ss, ds, and es segment selectors have a zero-forced base address. gs and fs are the exceptions and have a non-zero base address. So, how is it possible that the base of the gs selector is 0 when Windows itself uses the segment selector to retrieve processor state?
The answer is in the model-specific registers, MSRs. MSRs are per-processor registers that can be read via rdmsr and written via wrmsr instructions. On x64, the IA32_GS_BASE (0xC0000101) and IA32_KERNEL_GS_BASE (0xC0000102) MSRs are used for storage of the base address of the gs selector. swapgs was introduced to exchange the address of the current gs base register with the value contained in the IA32_KERNEL_GS_BASE MSR.
This means that we could, theoretically, read the IA32_GS_BASE MSR if we're executing code in CPL0 (ring0/kernel-mode). This would get us the base address of the gs segment. However, that's not directly possible through the VMware GDB stub. There is no support for reading or writing to MSRs directly.
A shimmer in the shadows
Nevertheless, through persistence, we come up with an approach that plays nicely given our constraints. There are multiple ways to skin a cat and this approach may not be the most elegant solution, but it should work nicely for all x64 Windows kernels.
The basic idea is to leverage the IDT, the interrupt descriptor table, to find a symbol that's in the address space of ntoskrnl. We can access the idtr, a register that houses the IDT, through the GDB stub:
Once we have the base of the IDT, in our case 0xfffff802c4850000, we can access the first entry of the IDT. This should resolve to a symbol within ntoskrnl (nt!KiDivideErrorFault):
From there, we can walk kernel memory backwards until we get to a valid PE header. Since the symbol is contained within ntoskrnl's address space, the first valid PE header should belong to ntoskrnl:
Figure 1: Layout of kernel memory.
Writing an IDA script using IDAPython
It'd be nice to programmatically implement the algorithm described above so we don't need to manually go through it each time we're trying to discover the base address of ntoskrnl. We'll do this by writing a script for IDA Pro to run. I chose to do this with IDAPython instead of IDC (IDA's C-like bindings) because of the niceties that Python provides (like string manipulation).
The basics
We'll start by switching the input from "GDB" to "Python" in the "Output window". If your "Output window" is missing, you can restore it by selecting "Windows" and then "Output window" from the menu bar:
We can see all the functionality exposed by IDAPython by executing the Python command dir() in the text box. If you try to do this, you'll see lots of output. It's easy to feel overwhelmed. Luckily, there exists amble documentation on the Hex-Rays website that can help us navigate these murky waters.
I try to find useful things by searching for it first in the dir() listing. You can position your cursor in the "Output window" and press Alt+T to search for a keyword. To find the next occurrence, you can hit Ctrl+T. If this fails, I move on to the documentation.
Sending a command to the GDB stub
Our first task is to figure out how to send a command to the GDB stub. If you search for the "command" keyword in the "Output window" you'll find something labeled "SendDbgCommand". Let's see what this function does by executing help(SendDbgCommand):
It seems very relevant to us. Let's give it a try:
Looks like it's working. This is the same output we received from the GDB stub when we issued the help command.
Parsing the response from the GDB stub
Now that we know how to send a command to the GDB stub, we need to issue a command to retrieve the contents of the idtr. We then parse and extract the base address from the resulting string.
It's important to tell Python that we're working with an integer object by "casting" the string to an integer-type:
Easy!
Getting the first IDT entry's handler
We have the base of the IDT in idt_base. Our next task is to retrieve the first entry in the IDT. The IDT is effectively an array that contains 256 IDT entries (0-0xFF) on x64. The format of the IDT is dictated by the architecture of the processor (e.g. Intel x64). Each IDT entry on x64 takes the following form:
To get to the handler (e.g. where the processor moves control to when an interrupt occurs), the target address is built from the OffsetHigh, OffsetMiddle, and OffsetLow fields of this structure using the following algorithm: HandlerAddress = ((OffsetHigh << 32) + (OffsetMiddle << 16) + OffsetLow).
We'll leverage the Dbg* commands to read virtual memory from IDAPython. Since we're extracting the first IDT entry, we can just read directly from the start of our idt_base:
This shows us that the handler for the first IDT entry (nt!KiDivideErrorFault) is loaded at 0xfffff802c27f4300. If we wanted to read the N'th IDT entry, we'd have to index into the array by adding 0x10, the size of a _KIDTENTRY64, times the location in the array (in this case N). So, to index into the 3rd IDT entry, we'd apply the following math: idt_entry = idt_base + (0x10 * 2).
Finding the base address from a symbol within ntoskrnl
First, we'll define a simple helper function that will align addresses to their page boundaries. This will help speed up our lookup because we know that the base address of ntoskrnl will be on a page boundary:
We'll then create a very simple loop to walk memory backwards (on a page-aligned boundary) searching for the magical value 0x5A4D, commonly known as 'MZ' (IMAGE_DOS_SIGNATURE). This value signifies the start of the IMAGE_DOS_HEADER which is also the base address of an image:
Voila! The base address of ntoskrnl is discovered at 0xfffff802c2680000.
Creating the final version of the script
After some refactoring and code tidying (including error checking), we produce a much better version of the script. This does the same thing as the commands we inserted in the IDAPython "Output window":
Save a copy of the script to your local drive. We are then able to run it at any time by going to "File" and then "Script file..." in the IDA Pro GUI. A sample of the output is listed below:
The important line appears on the bottom; the base address of ntoskrnl is displayed. It checks out with the work we did by hand too.
Loading ntoskrnl at its base address
We mustn't forget the final objective: loading kernel symbols. We're almost at the finish line. Let's tell IDA to load ntoskrnl at the base address our script found.
First, we'll need to grab a copy of ntoskrnl on the VM. Don't use the version on your host as this may not match with what's on the VM. This'll be found in your guest's system directory:
You might need to resume your VM if you're currently active in IDA's GDB debugger by selecting "Debugger" and then "Continue process" (or by hitting F9) from the menu bar.
After you've pulled ntoskrnl from your VM, break into IDA's GDB debugger by selecting "Suspend". Now, we must load it by selecting "File" then "Load file" and finally "PDB file..."
Find where you copied ntoskrnl to on your host and use the address that the script found:
It'll take IDA at least a couple of minutes to fully finish the loading process. You can see IDA's progress in the bottom left corner:
You'll know IDA's finished when the status changes to "AU: idle".
Quick validation
We should make sure that the symbols are loaded correctly. Navigate to "Jump" and then "Jump to address" (or press "G"). Enter PsLoadedModuleList (case sensitive) and hit "OK".
From there, double click the address immediately to the right of the PsLoadedModuleList symbol. This takes you to the first entry in the list.
Each entry in this list is of type _LDR_DATA_TABLE_ENTRY. You might be familiar with this structure from usermode programming. It's also used in the kernel.
We'll need to add the definition of the _LDR_DATA_TABLE_ENTRY to IDA's structures. Luckily, we have symbols loaded and this is a pretty straightforward process.
After the structure was added, you'll see a window similar to this.
Go back to the "Debug View". Impose the _LDR_DATA_TABLE_ENTRY structure on that memory region:
Let's follow the FullName.Buffer field:
And now let's convert this to a readable string:
You should see the characters \SystemRoot\system32\ntoskrnl.exe. We did it!
Final thoughts
Now that symbols are loaded for ntoskrnl, it would be wise to iterate through the PsLoadedModuleList and load symbols for all the other kernel mode components. This can be scripted using IDAPython too, however, it's beyond the scope of this article.
Sometimes you'll run into a situation that you can't analyze with a traditional kernel debugger like WinDbg. An example of such is trying to troubleshoot the runtime logic of PatchGuard (Microsoft's Kernel Patch Protection). In situations like this, you need to bust out the heavy tools. VMware has built in support for remote debugging of virtual machines running inside it through a GDB stub. IDA Pro, the defacto disassembler that most reverse engineers have, includes a GDB debugger. Together these make for a very powerful combo.
This article goes over how to setup VMware's GDB stub and how to connect to it using IDA Pro's GDB debugger.
Requirements
A copy of VMware Workstation (free 30-day trial). I'll be using VMware Workstation 12.5.7 (build-5813279).
Unfortunately, VMware Player (entirely free for non-commercial use) does not expose the GDB stub interface.
You can use either the Linux or Windows build of VMware. I'll be using the 64-bit Windows build.
A Windows operating system installed on your host and guest (VM). These do not have to be the same versions of Windows. My host and guest OS are both running Windows x64 10.0.15063 (Version 1703).
This can be any OS supported by VMware such as Ubuntu.
The second part of this tutorial (loading kernel symbols) assumes you're running a Windows 64-bit VM (AMD64).
Enabling the GDB stub within VMware
Select the VM you wish to enable GDB stub debugging on within VMware.
VMs should be listed in the "Library" pane on the left of the GUI. If the "Library" pane is missing, you can restore it by selecting "View" then "Customize" and choosing "Library" (or hit F9).
Ensure that the VM is currently not running. If it's currently active, power it off via the menu bar: "VM" then "Power" then "Shut Down Guest" (or Ctrl+E).
Select "Edit virtual machine settings". Ensure that you are on the "Options" tab.
Find the "Working directory" text field and copy the string to your clipboard. 'Cancel' out of the prompt.
Go to the working directory.
Right-click on the *.vmx file and "Open with" your favorite text editor. I'll be using Notepad++.
Add one of the following lines to the end of the file, based on preference.
If your VM is 32-bit and you want to debug locally: debugStub.listen.guest32 = "TRUE"
If your VM is 64-bit and you want to debug locally: debugStub.listen.guest64 = "TRUE"
If your VM is 32-bit and you want to debug remotely: debugStub.listen.guest32.remote = "TRUE"
If your VM is 64-bit and you want to debug remotely: debugStub.listen.guest64.remote = "TRUE"
The default port for the GDB stub is 8864 for 64-bit guests and 8832 for 32-bit guests. If you'd like to change what port the VMware GDB stub listens on (e.g. 55555), add one of the following lines to the file:
If your VM is 32-bit: debugStub.port.guest32 = "55555"
If your VM is 64-bit: debugStub.port.guest64 = "55555"
If you want to start debugging immediately on BIOS load add one of the following lines to your file:
If your VM is 32-bit: monitor.debugOnStartGuest32 = "TRUE"
If your VM is 64-bit: monitor.debugOnStartGuest64 = "TRUE"
To make it difficult to detect breakpoints that you've set using GDB, it's strongly recommended to add the following option too:
debugStub.hideBreakpoints = "TRUE"
An important thing to note is that this option is restricted by the number of hardware breakpoints available to the processor (usually 4).
Save the *.vmx file via "File" and then "Save" from the menu bar (or hit Control+S). Here's a copy of the contents of my *.vmx file:
Close the file.
Run the VM corresponding to the *.vmx file you just edited. Validate that the GDB stub is currently running by opening the vmware.log file in the same directory as the *.vmx file:
If you see a message from "Debug stub" that tells you VMware is "listening" for a debug connection on a certain port number, you're in a good state.
If you are missing that log line or have an error, ensure that your *.vmx file has the appropriate settings. Remember: you must edit the *.vmx file when the Virtual Machine is off or your changes may be lost.
Configuring the GDB debugger within IDA Pro
Launch the 64-bit version of IDA Pro if you're debugging a 64-bit VM and the 32-bit version of IDA Pro if you're debugging a 32-bit VM.
Skip the "Welcome" dialog (by hitting "Go") and go to the main disassembler window. Choose "Debugger" and then "Attach" and finally "Remote GDB debugger" from the menu bar.
Enter the appropriate "Hostname" and "Port". These were set up by you during steps 7 and 8 of the "Enabling the GDB stub within VMware" section. Furthermore, these can be validated in the vmware.log file (this was done in step 12 of the same section). Then hit "Debug options".
In the "Debugger setup" window, select "Set specific options".
Ensure that the right "Processor" is set in the drop down box. If you're debugging a 64-bit edition of Windows (AMD64), select "Intel x64". If you're debugging a 32-bit edition of Windows (X86), select "Intel x86".
Select 'OK' in the "GDB configuration" window. And then select 'OK' in the "Debugger setup" window. Finally, select 'OK' in the "Debug application setup" window.
The VM will become suspended and a green "play" button will appear. At this point, IDA should bring up a window with the title "Choose process to attach to".
Select "<attach to the process started on target>" and hit 'OK'.
If you see this window, that means you're almost done.
Select "Debugger" and then "Manual memory regions" from the menu bar.
Inside of the "Manual memory regions" tab, right click and select "Insert" (or just press "Insert" on your keyboard).
A new window will pop up. Enter in the "Start address" as 0 and the "End address" as -2. Make sure the right "segment" is selected (e.g. 64-bit segment for 64-bit VM debugging) and hit 'OK'.
This essentially maps all virtual memory from 0 to 0xFFFFFFFFFFFFFFFE (on x64).
"-1" is not an acceptable boundary for IDA Pro as the "End address".
Find the "General Registers" window and find the IP register (RIP on x64, EIP on x86).
If the "General Registers" window is gone, select "Debugger" and then "Debugger windows" and finally "General Registers" from the menu bar.
Right click on the IP register and select "Jump" from the context menu that appears.
Your memory view will become synched to the IP register. If there are raw bytes listed and not code, don't panic. Place your cursor on the address of the IP and hit "C".
Congratulations. At this point you've successfully set up VMware's GDB stub and IDA Pro's GDB debugger. You are now able to debug the VM and apply breakpoints through the IDA Pro GUI just as you would normally through a kernel debugger. Most of the functionality of the GDB debugger can be accessed through the "Debugger" menu bar.
This type of debugging is transparent to the kernel and therefore "debugger" checks like "KdDebuggerEnabled" and "KdDebuggerNotPresent" will not trigger. Furthermore, if the debugStub.hideBreakpoints option was enabled, breakpoints (up until the hardware maximum) will not make any inline code edits!
Final thoughts
Ultimately, the GDB debugger is not very useful without kernel symbols being loaded. One option, albeit a naive one, is to attach WinDbg as a kernel debugger while running IDA's GDB stub in the background. A tutorial on how to setup kernel debugging using WinDbg and VMware can be found here. You are then able to use the symbolic data that is provided from WinDbg to power debugging in IDA's GDB debugger. This is very cumbersome and has many disadvantages such as not being able to avoid kernel debugger checks.
Luckily, there is a better way. In the second part of this series, we'll discover how to load kernel symbols in IDA Pro's GDB debugger.
In this article, we explore the complexities and concepts behind Intel's 64-bit paging scheme, why we need paging in the first place, and some practical analysis of paging structures.
Why do we need paging?
In any application, whether it's a student's first program or a complicated operating system, instructions executed by the computer that involve memory use a virtual address. In fact, even when the CPU fetches the next instruction to execute, it uses a virtual address. A virtual address represents a specific location in the application's view of memory; however, it does not represent a location within physical RAM. Paging, or linear address translation, is the mechanism that converts a linear address accessible by the CPU to a physical address that the memory management unit (MMU) can use to access physical memory.
Technically, a linear address and a virtual address are not the same. For the purposes of this article, though, we will consider them to be the same, since we do not need to consider segmentation. Older architectures would first need to convert a virtual address to a linear address using segmentation.
Figure 1: An application with different parts of virtual memory mapping to different parts of physical memory.
Paging modes
In this article, we will focus on IA-32e 4-level paging (64-bit paging) on Intel architectures. It is worth noting, though, that there are other paging modes supported by Intel.
There are three mechanisms which control paging and the currently enabled paging mode. The first is the PG flag (bit 31) in control register 0 (CR0). If this bit is set, paging is enabled on the processor. If this bit is not set, no paging is enabled. In the latter case, the virtual address and physical address are considered equivalent and no translation is necessary.
If paging is enabled on the processor, then control register 4 (CR4) is checked for the Physical Address Extension (PAE) bit (bit 5) being set. If it is not, then 32-bit paging is used. If it is set, then the final condition that is checked is the Extended Feature Enable Register, or IA32_EFER MSR. If the Long Mode Enable (LME) bit (bit 8) of this register is not set, the processor is in PAE 36-bit paging mode. If the LME bit is set, the processor is in 4-level paging mode, which is the 64-bit mode that we plan to explore in this article. This mode translates 48-bit virtual addresses into 52-bit physical addresses, though because the virtual addresses are limited to 48-bits, the maximum addressable space is limited to 256TB.
Paging structures
Regardless of which paging mode is enabled, a series of paging structures are used to facilitate the translation from a virtual address to a physical address. The format and depth of these paging structures will depend on the paging mode chosen. Generally speaking, each entry in the paging structure is the size of a pointer and contains a series of control bits, as well as a page frame number.
In our case, 64-bit mode structures are 4,096 bytes in size (the size of the smallest architecture page - we will touch more on that later), containing 512 entries each. Every entry is 8 bytes.
Figure 2: A paging structure containing 512 pointer-size entries in 64-bit mode.
The first paging structure is always located at the physical address specified in control register 3 (CR3). As an aside, this is also the only place that stores the fully qualified physical address to a paging structure - in all other cases, we need to multiply a page frame number by the size of a page to get the real physical address. Each entry within the paging structure will contain a page frame number which either references a child paging structure for that region of memory, or maps directly to a page of physical memory that the original virtual address translates to. Again, in both cases, the page frame number is simply an index of a physical page in memory, and needs to be multiplied by the size of a page to get a meaningful physical address. Each paging structure entry also describes the the different memory access protections that are applied to the memory region they describe - whether the code is writable, executable, etc - as well as some more interesting properties such as whether or not that specific structure has previously been used for a translation.
While the nested paging structures are being walked, the translation can be considered complete either by identifying a page frame at the lowest level of paging structure or by an early termination caused by the configuration of a paging structure. For example, if a paging structure is marked as not present (bit 0 of the structure is not set) or if a reserved bit is set, the translation fails and the virtual address is considered invalid. Additionally, a paging structure can set its Page Size bit to indicate that it is the lowest paging structure for that region of memory, which we will touch more on later.
Figure 3: Some paging structures may not map to a physical page because the virtual address range they represent is invalid.
Anatomy of a virtual address
Information is encoded in a virtual address that makes the translation to a physical address possible. In 64-bit mode, we use 4-level paging, which means that any given virtual address can be divided into 6 sections with 4 of them associated with the different paging structures.
The different paging structures are as follows: a PML4 table (located in CR3), a Page Directory Pointer Table (PDPT), a Page Directory (PD), and a Page Table (PT). The figure below illustrates which bits of a given virtual address map to these different paging structures.
A single entry in the PML4 table (a PML4E) can address up to 512GB of memory, while an entry in the PDPT (a PDPTE) can address 1GB (parent granularity divided by 512) of memory, and so on. This is how we get the granularity of the paging structures down to 4KB at the lowest level.
Figure 4: The anatomy of a virtual address in 64-bit mode.
In the example above, we see that the highest bits (bits 63-48) are reserved. We will talk more about these bits in a future article, but for the purposes of address translation they are not used.
The next 9 bits (bits 47-39) are used to identify the index into the PML4 table that contains the entry (PML4E) that's next in our paging structure walk. For example, if these 9 bits evaluate to the number 16, then the 16th entry in the table (PML4[15]) is selected to be used for the address translation.
Once we have the PML4E entry from the given index, we can use that entry to provide us the address of the start of the next paging structure to walk to. Here is an example of what a PML4E structure would look like in C++.
Using the page frame number (PFN) member of the structure (in this case, it actually refers to the page frame where the next structure is located), we can now walk to the next structure in the hierarchy by multiplying that number by the size of a page (0x1000). The result of that multiplication is the physical address where the next paging structure is located. The PML4E points to a Page Directory Pointer Table (PDPT). We use the next 9 bits of our original virtual address (bits 38-30) to determine the index in the PDPT that we want to look at. At that index, we will find a PDPTE structure, like the one defined below.
It's worth noting at this point that paging structures other than those in the PML4 table contain a Page Size (PS) bit (bit 7). If this bit is set, then the current entry represents the physical page. This means that page sizes as large as 1GB can be supported, if the associated PDPTE indicates that it is a 1GB page by setting the PS bit. Otherwise, 2MB pages can be supported if the PS bit is set in the PDE structure. Not all processors support the PS bit being set in a PDPTE; therefore, not all processors will support 1GB pages.
Moving along in our example, we can assume that the PS bit is not set in the PDPTE that we just referenced. So, we will look at the page frame number of this structure and multiply by the page size again to get the physical address of the next paging structure root.
Figure 5: Our walk so far, from the CR3 register, through the PML4 and PDPT structures.
Using the PFN stored in the PDPTE structure, we're able to locate the Page Directory paging structure, which is next in the hierarchy. As before, we use the next 9 bits (bits 29-21) of the original virtual address to get the index into this structure where our entry of interest (a PDE, in this case) resides. The PDE structure is defined similarly to the previous structures, as shown below.
Again, we can use the PFN member of this structure multiplied by the size of a page to locate the next, and final, paging structure that facilitates the translation - the Page Table. The next 9 bits (bits 20-12) of our original virtual address are the index into the Page Table where the associated entry (PTE) is located. This PTE structure is defined below, and once again has similar characteristics to its predecessors.
The PFN member of this structure indicates the real page frame of the backing physical memory. Because our example went the full depth of the paging structures, the size of a page frame is 4KB, or 0x1000. Thus, in order to get the location in physical memory where the backing page begins, we multiply the page frame number from the PTE by 0x1000 as we had been doing previously. The remaining 12 bits (bits 11-0) of the original virtual address are the offset into the physical page where the actual data resides. Had our example not used the full depth of the paging structures, and had instead used 2MB page sizes (stopping at the Page Directory level), that PDE would have contained the page frame number of interest, and we would have multiplied that number by the size of a page frame, which in that case would be 2MB or 0x20000. We would then add the offset into the page, which would be the remaining bits (bits 20-0) of the original virtual address since we did not need to use the usual 9 bits for indexing into a Page Table structure.
Figure 6: Here we have a full traversal of the paging structures from CR3 all the way to the final PTE. We use the PFN from the PTE to calculate the backing physical page.
Practical exploration with WinDbg
We can use WinDbg to explore what this structure hierarchy looks like in practice. Windows does some things differently (such as per-process CR3 to keep the virtual address spaces of processes separate) and there are certain complexities that we will cover in a future article, but we will choose a simple example that demonstrates what we've just learned.
Attach an instance of WinDbg as a kernel debugger to the virtual machine or physical box of your choice to get started. Check out this article for instructions on how to do so.
Once we've broken in, use the lm command to list the modules that have been loaded by the current process.
We'll use the image base of ntdll.dll as our example. It's located at 0x00000000`771d0000. We can view the memory at that virtual address by using db (or dX, where X is your desired format specifier).
Here we can see the signature 'MZ' as we would expect from a DOS header. But where are these bytes located in physical memory? There are two ways we can find out.
The first way is the hard way - we can get the value stored in CR3 which gives us the beginning of our PML4 paging structure, and begin our manual walk like we described above.
This means that the start of our PML4 table is located at physical address 0x187000. We can take a look at the physical memory at that location using !dq (or !dX, again where X is the format specifier you want to use). We're aligning on a quad-word because the size of each entry in any paging structure in 64-bit mode is 8 bytes.
Here we see that we have one PML4E structure, with 0x00700007`ddc82867 as the value. For a paging structure entry, we know that bits 47-12 represent the page frame number of the next paging structure. So we extract those bits to get 0x7ddc82, then multiply it by the size of a page frame on this architecture (4KB) to get a physical address of 0x00000007`ddc82000.
If we navigate to that physical address, let's see what we get.
Sure enough, there are two PDPTE entries (or potentially more, off-screen, since there can be up to 512 listed) here in this PDPT that we've walked to. In order to figure out which PDPTE we need to reference, we'd need to refer to the 9 bits in the original virtual address that map to the PDPT (bits 39-30), which in the case of our example works out to 0x1. That means we want the second entry of the PDPT structure, at index 1.
We can extract the page frame number from that PDPTE entry using the same bits we used in the last example (bits 47-12), resulting in 0x7d96b8. Let's multiply that number by 4KB, and see what we've got at that physical address.
You may be wondering at this point: what's going on? Why is there nothing in the PD structure that was referenced by our PDPTE? Remember, not all memory is valid and mapped, so the fact that we are seeing a bunch of zero-value PDE entries isn't a surprise. It just means that those regions of virtual memory aren't currently mapped to a physical page. In order to get to the PDE we care about, we need to take the next 9 bits of the original virtual address as we did before, this time getting a value of 0x1b8 after extracting the bits. That will get us the index into the PD structure where our PDE of interest is located. We can navigate to that memory location now, remembering to multiply the index by the size of a paging structure entry, which is 8 bytes.
That gets us 0x67e00007`d96b9867 as our PDE value. Once again, we extract the bits that are relevant to the page frame number, and we come up with 0x7d96b9.
We can repeat the steps we've taken previously to multiply that page frame number by 4KB, add the PT index using the next 9 bits of the original virtual address (0x1d0 in this case), then navigate to the correct physical address.
We've gotten the value 0xe7d00007`d9cc0025 for our PTE entry. We're almost there! We just need to do the same steps we've been doing one more time - extract the PFN from that value (0x7d9cc0), multiply by the size of a page (0x1000), but this time, we need to add the page offset (bits 11-0) from our original virtual address to the result. This should get us to 0x00000007`d9cc0000 since our page offset in this example was actually zero. Let's look at the memory!
There's the header, just like we expected. That's a cumbersome amount of work, though, and we don't want to have to be doing that manually every time we try to translate an address. Luckily, there's an easier way.
WinDbg provides the !pte command to illustrate the entire walk down the paging structures and what each entry contains. It is important to note, though, that the addresses of the paging structures are converted to virtual addresses before being displayed, so they will look different from the physical addresses we extrapolated on our own, but they point to the same memory.
You can see that WinDbg gives us the address of the paging structure used, what it contained, and the page frame number for each. You can verify that the PFN on the PXE (PML4E) entry matches up with what we calculated, too. The most important part of all of this information is the PFN that's within the lowermost entry, the PTE. In our case it's 0x7d9cc0.
So, we can multiply that page frame number by 0x1000 to get 0x00000007`d9cc0000, and that should be the physical address of the DOS header of ntdll.dll! This checks out based on the manual calculations we did previously, but let's take a look again to make sure.
And there it is! We can test this by editing the DOS header in WinDbg and seeing if those changes are reflected on the physical page.
Let's check it out using the virtual address...
...and the physical address...
And there you have it! We now know how to successfully walk the IA-32e paging structures to convert a virtual address into a physical address.
Setting up WinDbg for kernel-mode debugging is a fairly trivial process, however, it's easy to miss (or incorrectly configure) a step causing you to waste precious time. In this post, I have written a tutorial that goes through the entire process of setting up WinDbg (and configuring symbol lookup) for kernel-mode debugging with VMware using a named pipe and a virtual serial connection.
Serial port debugging was chosen for compatibility reasons. Other debugging modes like ethernet/network, while quicker, require special hardware (e.g. certain network interface cards are compatible and many are not) and are only supported on newer versions of Windows.
Requirements
A copy of either VMware Workstation (free 30-day trial) or VMware Player (entirely free for non-commercial use) for Windows. I'll be using VMware Workstation 12.5.7 (build-5813279).
A Windows operating system installed on your host and guest (VM). These do not have to be the same versions of Windows, but should be running at least Windows XP or later. My host and guest OS are both running Windows x64 10.0.15063 (Version 1703).
A free copy of Windows 10 can be found here as long as the tool is run on a machine that has a valid Windows license (of any version). Follow the steps to create an ISO file. Use the ISO file to install the OS on the Virtual Machine (helpful documentation can be found on the VMware website and WikiHow).
WinDbg.
The latest and greatest version can be downloaded from this page (direct link). This requires installation through the Windows SDK, however, you can unselect all components except "Debugging Tools for Windows" if you do not plan on doing any software development. I'll be using WinDbg x64 10.0.15063.400.
Setting up symbols on your host
Microsoft provides stripped ("redacted") PDBs (commonly referred to as "symbols") for most of their software releases. This includes the kernel components that power the operating system. In order to leverage this very useful information, we'll need to setup WinDbg so it can access these resources.
Locate your WinDbg installation.
For most people, this will be located in the following directory: C:\Program Files (x86)\Windows Kits\10\Debuggers\x64
Right-click on the windbg.exe file and select "Create shortcut". This shortcut should be placed on the desktop or another convenient place.
Right-click on the shortcut that you just created. Select "Properties". In the "Shortcut" tab, you'll see a window similar to this:
Select the "Target:" text field and append a string of the following format: -y "srv*c:\symbols*https://msdl.microsoft.com/download/symbols"
The syntax of this command string is: srv*[local cache]*[private symbol server]*https://msdl.microsoft.com/download/symbols
This will download all available symbols, as necessary, from the Microsoft Symbol Server to your local symbol directory at c:\symbols. If you prefer to place your downloaded symbols somewhere else, choose another local path instead.
This command supports multiple symbol servers. For example, if you wish to pull symbols from a remote share, you can append to this path, e.g:
srv*c:\symbols*\\mainserver\symbols*https://msdl.microsoft.com/download/symbols
Example of a fully qualified "Target:" text field:
"C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\windbg.exe" -y "srv*c:\symbols*https://msdl.microsoft.com/download/symbols"
Save your shortcut changes by pushing 'OK'. Now that we've instructed WinDbg to pull symbol information from the Microsoft Symbol Server, let's test it to ensure that everything is working.
We pass the symbol path via a command line parameter to WinDbg for reliability reasons. We could have, alternatively, configured an environment variable, _NT_SYMBOL_PATH, to achieve the same functionality, but it's a less elegant solution.
Run the shortcut and a copy of the pre-installed application Notepad.
Select "File" then "Attach to a Process..." (or hit F6) in WinDbg. Scroll down all the way and select "notepad.exe" from your process list. Then hit 'OK'.
A "Command" window will appear. We'll use this to issue commands to WinDbg.
I like to expand my "Command" window so it takes up the full view in the debugger. You can do this by right clicking on the "Command" window title and selecting "Dock":
Let's try to load symbols for all the running modules (executable and DLLs). First, let's list what modules are currently loaded in our process by using the lm (list modules) command in the text box immediately to the right of ">". This is in the bottom left corner of the "Command" window:
If your list looks different from mine, don't worry. Different versions of Windows and different versions of Notepad will have different modules loaded.
Next, let's force a symbol load of all modules within our process by executing .reload /f:
Pro-tip: WinDbg has a great manual. To access it, you can type the command .hh within the debugger (or select "Help" and then "Search" from the menu bar). Typing .hh search terms go here automatically runs a search for the user supplied argument.
.hh .reload documents the .reload command. In particular, it explains why the /f argument is supplied.
It may take WinDbg a few moments to load all symbol information. You can see the status of WinDbg in the bottom left corner (next to where commands are inserted). After WinDbg has loaded symbols, run the lm command again.
Pro-tip: If WinDbg stays "BUSY" for a long time, you can force it to stop its current task by pushing Ctrl+Break on your keyboard or by selecting "Debug" and then "Break" from the menu bar.
As you can see, most modules now have a local symbol path listed to the right of their module name. It's very possible that there may be some modules that still do not have symbols loaded. These modules are most likely not distributed by Microsoft (e.g. 3rd party antivirus vendors).
For validation, go to the directory that you've setup for your local symbol cache, e.g. C:\symbols. If the folder contains data, you're set and can skip the troubleshooting step.
Troubleshooting
Verbose output
The easiest way to troubleshoot problems with symbol loading is to enable verbose output with the !sym noisy command:
Next, issue the .reload /f command.
In my example, it's easy to see that I mistyped the URL to the Microsoft Symbol Server in my shortcut's "Target:" field. After applying the right URL to my "Target:" field, I can restart WinDbg and try again.
The lazy fix
WinDbg may be able to fix the problem for you automagically if you issue .symfix and then .reload /f. In this case, WinDbg will alter your symbol path to the Microsoft Symbol Server. Your downloaded symbols will be stored, locally, in WinDbg's current working directory (C:\Program Files (x86)\Windows Kits\10\Debuggers\x64) or C:\ProgramData\dbg.
Setting up VMware on your host
Select the VM you wish to enable kernel-mode debugging on within VMware.
VMs should be listed in the "Library" pane on the left of the GUI. If the "Library" pane is missing, you can restore it by selecting "View" then "Customize" and choosing "Library" (or hit F9).
If your VM is not listed in the "Library" pane, you can manually navigate to it's .vmx file via "File" and then "Open..." (or Control+O).
Ensure that the VM is currently not running. If it's currently active, power it off via the menu bar: "VM" then "Power" then "Shut Down Guest" (or Ctrl+E).
Select "Edit virtual machine settings". Ensure that you are on the "Hardware" tab.
Select the "Add" button and choose "Serial Port" from the "Add Hardware Wizard". Hit "Next >".
Ensure that the "Serial port" checkbox is targeting "Output to named pipe" and then hit "Next >".
On the final screen, you should see similar settings to this. Make a note of the "Named pipe" field and then hit "Finish".
Ensure that your settings match those above. In particular, output to a "Named pipe" at \\.\pipe\com_1 and ensure that the first drop down box has "This end is the server" selected and the last drop down box has "The other end is a virtual machine" selected. Finally, make sure that you've selected "Connect at power on".
The com_1 substring can be changed to something else (e.g. kdebug), but it needs to be remembered and the exact name should be used within WinDbg too.
The "Add Hardware Wizard" will now close and a new "Serial port" will be added to your "Hardware" tab. Ensure the "Yield CPU on poll" checkbox is selected in "Virtual Machine Settings". Make a note of the number to the right of "Serial Port" (if there is no number, it's assumed to be 1).
In my example, my serial port is number 2.
The 'Printer' is using "Serial Port 1".
In the guest (Virtual Machine) context
For guests (VMs) running Windows Vista and later.
Start the VM.
After Windows is finished loading, run "Command Prompt" (Start+R > cmd.exe) as an Administrator.
In Windows 10, you can right-click on the Windows logo in the taskbar (bottom-left) and select "Command Prompt (Admin)".
Input the following commands in this elevated prompt:
bcdedit /debug on
bcdedit /dbgsettings serial debugport:2 baudrate:115200
Make sure your debugport argument matches your serial port number from step 7 in the "Setting up VMware" section. My serial port number is 2 because my VM has a printer that is using serial port number 1.
Pro-tip: You can add the /noumex switch to the the dbgsettings command, e.g. bcdedit /dbgsettings serial debugport:2 baudrate:115200 /noumex. This avoids user mode exceptions from causing the system to break into the kernel debugger.
Now validate that the settings have been successfully applied:
bcdedit /dbgsettings
bcdedit
You should see similar command prompt output to this:
Finally, shutdown Windows cleanly. You can do this via the traditional route (the start menu) or by executing the shutdown -s -t 0 command in command prompt.
For guests (VMs) running Windows XP.
Start the VM.
bcdedit does not exist on Windows XP. To enable kernel debugging, you must alter the boot.ini file. The easiest way to do this is by clicking on Start and then Run (Start+R). Enter C:\boot.ini as the argument and hit 'OK'.
You might have to change the drive letter (from C:\) if your operating system is installed on a different drive.
This file is hidden (and considered a protected operating system file). Therefore, it won't be displayed in Windows Explorer by default.
Append the string /debug /debugport=COM2 /baudrate=115200 to the end of the first entry in the [operating systems] section.
Make sure your debugport argument matches your serial port number from step 7 in the "Setting up VMware" section. My serial port number is 2 (hence COM2) because my VM has a printer that is using serial port number 1.
Save the boot.ini via "File" and then "Save" from the menu bar (or hit Control+S). Close the file.
Finally, shutdown Windows cleanly via the traditional route (the start menu).
Finalizing WinDbg on your host
Open the shortcut to your WinDbg that you created in step 2 in the "Setting up symbols on your host" section.
Click on "File" and then "Kernel Debug..." (or press Ctrl+K). Select the "COM" tab and use your settings from the previous sections. If you've been following the tutorial verbatim, you can just use these settings:
Finally, hit 'OK' and launch your Virtual Machine. WinDbg should automatically establish a connection to VMware when Windows begins loading.
Break into the debugger by pressing Ctrl+Break or by selecting "Debug" and then "Break" from the menu bar. At this point, the Virtual Machine will be in a suspended state (e.g. Windows will stop loading).
Load your kernel symbols with a .reload /f command. Then list the loaded modules via lm. If you're having troubles loading symbols, review the "Setting up symbols on your host" section above and work through the "Troubleshooting" tips if all else fails.
Congratulations. At this point you've successfully set up kernel debugging using WinDbg and VMware over a virtual serial connection.
Extra special bonus stage
Modifying the shortcut to start kernel debugging immediately
Having to manually configure WinDbg each time for kernel debugging is a real pain. Luckily, there is a better way.
Right-click on the shortcut that you created for WinDbg. Select "Properties". In the "Shortcut" tab, you'll see a window similar to this:
Append the following string to the "Target:" textbox: -k com:pipe,port=\\.\pipe\com_1,resets=0,reconnect
You might have to change the pipe name from com_1 to whatever you selected in step 6 in the "Setting up VMware on your host" section.
The final "Target:" argument should look similar to this: "C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\windbg.exe" -y "srv*c:\symbols*https://msdl.microsoft.com/download/symbols" -k com:pipe,port=\\.\pipe\com_1,resets=0,reconnect
Hit 'OK' and you should be all set. Now when you run this shortcut of WinDbg, it will correctly configure your symbol path (without having to use yucky environment variables) and will automatically start kernel debugging the first active named pipe.
This blog post is about my personal attempt to superficially list VAD types under Windows 10. It all started when I was wondering, out of sheer curiosity, if there's any way to determine the VAD type (MMVAD_SHORT or MMVAD) other than by looking at the pool tag preceding the structure. In order to do that, I had to list all VAD types, do some reverse engineering, and then draw a table describing what I've been able to find. You can view the full document by clickinghere
From the table above it is possible to deduce the VAD structure type from both the VadType and PrivateMemory flags.
VadType flag
PrivateMemory flag
Type
0
0
MMVAD
0
1
MMVAD_SHORT
1
1
MMVAD
2
0
MMVAD
3
1
MMVAD_ENCLAVE
To test it out, I wrote a kernel driver that prints the deduced VAD type for each node of calc.exe. It also prints the pool tag so we can check the result.
And that's all for this article.
You can follow me on Twitter : here
