In this post, I have written a tutorial that goes through the entire process of setting up WinDbg (and configuring symbol lookup) for kernel-mode debugging with VMware using a named pipe and a virtual serial connection.
Serial port debugging was chosen for compatibility reasons. Other debugging modes like ethernet/network, while quicker, require special hardware (e.g. certain network interface cards are compatible and many are not) and are only supported on newer versions of Windows.
- A copy of either VMware Workstation (free 30-day trial) or VMware Player (entirely free for non-commercial use) for Windows. I'll be using VMware Workstation 12.5.7 (build-5813279).
- A Windows operating system installed on your host and guest (VM). These do not have to be the same versions of Windows, but should be running at least Windows XP or later. My host and guest OS are both running Windows x64 10.0.15063 (Version 1703).
- A free copy of Windows 10 can be found here as long as the tool is run on a machine that has a valid Windows license (of any version). Follow the steps to create an ISO file. Use the ISO file to install the OS on the Virtual Machine (helpful documentation can be found on the VMware website and WikiHow).
- The latest and greatest version can be downloaded from this page (direct link). This requires installation through the Windows SDK, however, you can unselect all components except "Debugging Tools for Windows" if you do not plan on doing any software development. I'll be using WinDbg x64 10.0.15063.400.
Setting up symbols on your host
Microsoft provides stripped ("redacted") PDBs (commonly referred to as "symbols") for most of their software releases. This includes the kernel components that power the operating system. In order to leverage this very useful information, we'll need to setup WinDbg so it can access these resources.
- Locate your WinDbg installation.
- For most people, this will be located in the following directory:
C:\Program Files (x86)\Windows Kits\10\Debuggers\x64
- The syntax of this command string is:
srv*[local cache]*[private symbol server]*https://msdl.microsoft.com/download/symbols
- This will download all available symbols, as necessary, from the Microsoft Symbol Server to your local symbol directory at c:\symbols. If you prefer to place your downloaded symbols somewhere else, choose another local path instead.
- This command supports multiple symbol servers. For example, if you wish to pull symbols from a remote share, you can append to this path, e.g:
- Example of a fully qualified "Target:" text field:
"C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\windbg.exe" -y "srv*c:\symbols*https://msdl.microsoft.com/download/symbols"
- We pass the symbol path via a command line parameter to WinDbg for reliability reasons. We could have, alternatively, configured an environment variable, _NT_SYMBOL_PATH, to achieve the same functionality, but it's a less elegant solution.
- I like to expand my "Command" window so it takes up the full view in the debugger. You can do this by right clicking on the "Command" window title and selecting "Dock":
- Pro-tip: WinDbg has a great manual. To access it, you can type the command .hh within the debugger (or select "Help" and then "Search" from the menu bar). Typing .hh search terms go here automatically runs a search for the user supplied argument.
- .hh .reload documents the .reload command. In particular, it explains why the /f argument is supplied.
Pro-tip: If WinDbg stays "BUSY" for a long time, you can force it to stop its current task by pushing Ctrl+Break on your keyboard or by selecting "Debug" and then "Break" from the menu bar.
The easiest way to troubleshoot problems with symbol loading is to enable verbose output with the !sym noisy command:
Now validate that the settings have been successfully applied:
Finally, shutdown Windows cleanly. You can do this via the traditional route (the start menu) or by executing the shutdown -s -t 0 command in command prompt.
Next, issue the .reload /f command.
The lazy fix
WinDbg may be able to fix the problem for you automagically if you issue .symfix and then .reload /f. In this case, WinDbg will alter your symbol path to the Microsoft Symbol Server. Your downloaded symbols will be stored, locally, in WinDbg's current working directory (C:\Program Files (x86)\Windows Kits\10\Debuggers\x64) or C:\ProgramData\dbg.
Setting up VMware on your host
- Select the VM you wish to enable kernel-mode debugging on within VMware.
- VMs should be listed in the "Library" pane on the left of the GUI. If the "Library" pane is missing, you can restore it by selecting "View" then "Customize" and choosing "Library" (or hit F9).
- If your VM is not listed in the "Library" pane, you can manually navigate to it's .vmx file via "File" and then "Open..." (or Control+O).
- Ensure that the VM is currently not running. If it's currently active, power it off via the menu bar: "VM" then "Power" then "Shut Down Guest" (or Ctrl+E).
- Select "Edit virtual machine settings". Ensure that you are on the "Hardware" tab.
- Select the "Add" button and choose "Serial Port" from the "Add Hardware Wizard". Hit "Next >".
- Ensure that the "Serial port" checkbox is targeting "Output to named pipe" and then hit "Next >".
- On the final screen, you should see similar settings to this. Make a note of the "Named pipe" field and then hit "Finish".
- Ensure that your settings match those above. In particular, output to a "Named pipe" at \\.\pipe\com_1 and ensure that the first drop down box has "This end is the server" selected and the last drop down box has "The other end is a virtual machine" selected. Finally, make sure that you've selected "Connect at power on".
- The com_1 substring can be changed to something else (e.g. kdebug), but it needs to be remembered and the exact name should be used within WinDbg too.
- The "Add Hardware Wizard" will now close and a new "Serial port" will be added to your "Hardware" tab. Ensure the "Yield CPU on poll" checkbox is selected in "Virtual Machine Settings". Make a note of the number to the right of "Serial Port" (if there is no number, it's assumed to be 1).
In my example, my serial port is number 2.
- The 'Printer' is using "Serial Port 1".
In the guest (Virtual Machine) context
For guests (VMs) running Windows Vista and later.
- Start the VM.
- After Windows is finished loading, run "Command Prompt" (Start+R > cmd.exe) as an Administrator.
- In Windows 10, you can right-click on the Windows logo in the taskbar (bottom-left) and select "Command Prompt (Admin)".
- Input the following commands in this elevated prompt:
- bcdedit /debug on
- bcdedit /dbgsettings serial debugport:2 baudrate:115200
- Make sure your debugport argument matches your serial port number from step 7 in the "Setting up VMware" section. My serial port number is 2 because my VM has a printer that is using serial port number 1.
- Pro-tip: You can add the /noumex switch to the the dbgsettings command, e.g. bcdedit /dbgsettings serial debugport:2 baudrate:115200 /noumex. This avoids user mode exceptions from causing the system to break into the kernel debugger.
- bcdedit /dbgsettings
For guests (VMs) running Windows XP.
- Start the VM.
bcdedit does not exist on Windows XP. To enable kernel debugging, you must alter the boot.ini file. The easiest way to do this is by clicking on Start and then Run (Start+R). Enter C:\boot.ini as the argument and hit 'OK'.
- You might have to change the drive letter (from C:\) if your operating system is installed on a different drive.
- This file is hidden (and considered a protected operating system file). Therefore, it won't be displayed in Windows Explorer by default.
- Append the string /debug /debugport=COM2 /baudrate=115200 to the end of the first entry in the [operating systems] section.
- Make sure your debugport argument matches your serial port number from step 7 in the "Setting up VMware" section. My serial port number is 2 (hence COM2) because my VM has a printer that is using serial port number 1.
- Save the boot.ini via "File" and then "Save" from the menu bar (or hit Control+S). Close the file.
- Finally, shutdown Windows cleanly via the traditional route (the start menu).
Finalizing WinDbg on your host
- Open the shortcut to your WinDbg that you created in step 2 in the "Setting up symbols on your host" section.
- Click on "File" and then "Kernel Debug..." (or press Ctrl+K). Select the "COM" tab and use your settings from the previous sections. If you've been following the tutorial verbatim, you can just use these settings:
- Finally, hit 'OK' and launch your Virtual Machine. WinDbg should automatically establish a connection to VMware when Windows begins loading.
- Break into the debugger by pressing Ctrl+Break or by selecting "Debug" and then "Break" from the menu bar. At this point, the Virtual Machine will be in a suspended state (e.g. Windows will stop loading).
- Load your kernel symbols with a .reload /f command. Then list the loaded modules via lm. If you're having troubles loading symbols, review the "Setting up symbols on your host" section above and work through the "Troubleshooting" tips if all else fails.
- Congratulations. At this point you've successfully set up kernel debugging using WinDbg and VMware over a virtual serial connection.
Extra special bonus stage
Modifying the shortcut to start kernel debugging immediately
Having to manually configure WinDbg each time for kernel debugging is a real pain. Luckily, there is a better way.
- Right-click on the shortcut that you created for WinDbg. Select "Properties". In the "Shortcut" tab, you'll see a window similar to this:
- Append the following string to the "Target:" textbox:
- You might have to change the pipe name from com_1 to whatever you selected in step 6 in the "Setting up VMware on your host" section.
- The final "Target:" argument should look similar to this:
"C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\windbg.exe" -y "srv*c:\symbols*https://msdl.microsoft.com/download/symbols" -k com:pipe,port=\\.\pipe\com_1,resets=0,reconnect
- Hit 'OK' and you should be all set. Now when you run this shortcut of WinDbg, it will correctly configure your symbol path (without having to use yucky environment variables) and will automatically start kernel debugging the first active named pipe.