Introduction to Kernel Debugging with Windbg

Introduction

You've probably heard about Windbg before, right? It's a Windows debugger written by Microsoft that's used by developers and hackers for debugging the OS. We won't go into the process of installing the Windbg debugger, since it's fairly easy to do. When we install and start Windbg, it will look like this:

Become a certified reverse engineer!

Get live, hands-on malware analysis training from anywhere, and become a Certified Reverse Engineering Analyst.

Start Learning

In the previous article, accessible here, I was talking about kernel debugging in general and explained why we might need it. I pointed out that when debugging, all the DLLs, like kernel32.dll, ntdll.dll and others, are loaded in user mode and provide a gateway to the kernel mode. Actually, all the code from 0x00000000-0x7FFFFFFF address memory is located in user mode, while the rest of the addresses at 0x80000000-0xFFFFFFFF are used for kernel mode. In this article, we'll use the same example as we already did in the previous one, but just for completeness, we're presenting it below:

[cpp]

#include "stdafx.h"

#include <stdio.h>

#include <windows.h>

#include <Winternl.h>

int _tmain(int argc, _TCHAR* argv[]) {

__asm { int 3 }

typedef long NTSTATUS;

#define STATUS_SUCCESS ((NTSTATUS)0L)

HANDLE hProcess = GetCurrentProcess();

typedef struct _SYSTEM_KERNEL_DEBUGGER_INFORMATION {

BOOLEAN DebuggerEnabled;

BOOLEAN DebuggerNotPresent;

} SYSTEM_KERNEL_DEBUGGER_INFORMATION, *PSYSTEM_KERNEL_DEBUGGER_INFORMATION;

enum SYSTEM_INFORMATION_CLASS { SystemKernelDebuggerInformation = 35 };

typedef NTSTATUS (__stdcall *ZW_QUERY_SYSTEM_INFORMATION)(IN SYSTEM_INFORMATION_CLASS SystemInformationClass, IN OUT PVOID SystemInformation, IN ULONG SystemInformationLength, OUT PULONG ReturnLength);

ZW_QUERY_SYSTEM_INFORMATION ZwQuerySystemInformation;

SYSTEM_KERNEL_DEBUGGER_INFORMATION Info;

/* load the ntdll.dll */

HMODULE hModule = LoadLibrary(_T("ntdll.dll"));

ZwQuerySystemInformation = (ZW_QUERY_SYSTEM_INFORMATION)GetProcAddress(hModule, "ZwQuerySystemInformation");

if(ZwQuerySystemInformation == NULL) {

printf("Error: could not find the function ZwQuerySystemInformation in library ntdll.dll.");

exit(-1);

}

printf("ZwQuerySystemInformation is located at 0x%08x in ntdll.dll.n", (unsigned int)ZwQuerySystemInformation);

if (STATUS_SUCCESS == ZwQuerySystemInformation(SystemKernelDebuggerInformation, &Info, sizeof(Info), NULL)) {

if (Info.DebuggerEnabled && !Info.DebuggerNotPresent) {

printf("System debugger is present.");

}

else {

printf("System debugger is not present.");

}

}

/* wait */
getchar();

return 0;

}

[/cpp]

We won't explain the program in details, since we've already done so in one of the previous articles. But let's just say that we're loading the ntdll.dll into the user space and calling the ZwQuerySystemInformation function to determine whether the system debugger is present or not, which we're displaying on the screen.

Configuring the Debugging Environment Manually

To do kernel debugging with Windbg, we will need two machines to do so. Remember that when using SoftICE, we could get away with one machine; this isn't the case here. Here we're going to use two Windows XP virtual machines: the first VM will be used as a debuggee and the other will be used to debug the first VM.

First, we must edit the C:boot.init configuration file on the first virtual machine and add "/debug /debugport=COM1 /baudrate=115200" to it as follows:

[plain]

[boot loader]

timeout=30

default=multi(0)disk(0)rdisk(0)partition(1)WINDOWS

[operating systems]

multi(0)disk(0)rdisk(0)partition(1)WINDOWS="Microsoft Windows XP Professional" /noexecute=optin /fastdetect

multi(0)disk(0)rdisk(0)partition(1)WINDOWS="Microsoft Windows XP Professional" /noexecute=optin /fastdetect /debug /debugport=COM1 /baudrate=115200

[/plain]

After successfully changing the boot.ini file, we must shut down the VM and configure the serial port. Go to the settings of the first virtual machine and click on "Serial Ports," then enable the serial port and provide the port number, port mode and port/file path as can be seen below:

Once we've set up the serial port correctly, we can start the virtual machine, and these two options will be shown:

Notice that the path for "Port/File Path" is /tmp/debugpipe; this is because we're using a Linux system as a host system. If we're running VirtualBox on Windows OS, we can specify a different pipe, like ".pipedebugpipe". Also keep in mind that when configuring the debuggee, we shouldn't enable the "Create Pipe" option, because the other VM should create the named pipe on the system.

After that, we must also configure the second virtual machine with the settings presented on the picture below:

Notice that we've used the same settings as with the first virtual machine, except that we've also enabled "/tmp/debugpipe" which should create the named pipe on the Linux operating system under the /tmp directory. Since the second VM creates the named pipe and the first VM just connects to it, we must first start the second VM and wait for it to boot. After that, we need to start Windbg and select File – Kernel Debugging, which will open a new window as presented below:

In the new window, we must input all the details as we inputted them in the virtual machine. The most important option is the "Port" command, which is com1 in this case. This directly corresponds to the COM1 serial port specified in the VirtualBox settings. Also don't enable the "Pipe" and "Reconnect" options, because they are not important for now. Once we've done that, Windbg will open the .com1 named pipe and start listening for connections:

Now is the time to start the first VM, which will first ask us which operating system to boot into. This correlates directly to the boot.ini configuration file where we've specified two operating systems: one with debugging disabled and one with debugging enabled. We must of course select the one with the enabled debugging, so that the OS will be able to connect to the Windbg program we started in the second VM. The choice about booting the operating system can be seen on the picture below:

Soon after selecting the "Microsoft Windows XP Professional [debugger enabled]" option, we'll see that the first virtual machine will successfully connect to the Windbg program in the second virtual machine, and we'll then be able to debug the OS. This can be seen on the picture below:

Okay, there's some text being displayed on the debugger window, so something must be happening, but there are also some errors about symbols not being loaded. We must also load these symbols, which are kind of a necessity, because they will ease our work dramatically. If we don't have the symbols loaded, we'll have to operate with direct virtual addresses, but with symbols, we'll only need to operate with symbolic names for corresponding virtual addresses, so it's much easier to work with. The symbols can be loaded by selecting File – Symbol Search Path and inputting the following into the message box:

[plain]

SRV*your local folder for symbols*http://msdl.microsoft.com/download/symbols

[/plain]

Also remember to enable the "Reload" option, so that the symbols will be loaded into the current session. This can be seen on the picture below:

After clicking on the OK button, the symbols will be downloaded and a .reload command will be executed:

If you don't want to do all of the above work manually, you can also use the VirtualKD tool to set up everything automatically. Instructions on how to use VirtualKD can be found on the following link: http://www.codeproject.com/Articles/130016/Setting-Up-Kernel-Mode-Debugger-in-Windows.

VirtualKD

In the previous example, we've set-up all we needed to connect two Windows machines together, where one was used as a debuggee and the other as a debugger. You may be wondering, is there a better way to do this? If you're a freak who wants to know how a program does everything that it does, like me, then you're probably not very excited about some automatic way of doing things. But once we've learned how to do something in depth, a program that does it for you is a great way of automating tasks, as long as you know what's going on behind the curtains.

Here, we'll be using VirtualKD to set up the debugging automatically. Another great feature of VirtualKD besides automation is the performance boost. VirtualKD promises up to 45 times faster Windows kernel debugging, which is well worth the time to invest in doing it.

Since I don't run a Windows host machine, I won't actually do this; VirtualKD can only be done on Windows host machines and Linux is not supported. However, you can find really good steps to do so here: http://virtualkd.sysprogs.org/tutorials/install/.

I just want to say to all of you using Windows who are a little discouraged about debugging the whole operating system: don't be - use VirtualKD and be done with it. Of course, if you want to know the details, first try to set up the debuging session manually, and then use VirtualKD after that.

Running the Example

The example program we provided at the beginning of this article consequently loads the ntdll.dll into the current user-space memory and finds the ZwQuerySystemInformation function that must be present in the ntdll.dll library. Upon locating the function, it calls it to get a structure, which can be used to figure out whether a system debugger is present or not.

Let's now compile the example we provided above to see what will happen. Upon compiling the program, we need to run it in the debuggee Windows box. When we do that the following window will open:

You can see that nothing was printed on the screen and that Windows has halted. This happened because we manually inputted the "int 3" interrupt instruction directly in our program's code (take a look at the C++ code of the program). Since we have a kernel debugger attached to the system, it must have caught the exception, paused the program and displayed some information about a breakpoint being hit. If we take a look at the debugger Windows XP virtual machine, we can see that this is exactly so.

On the picture below, we can see that a breakpoint was hit, because of the "int 3" instruction that we've inputted in our source code:

If we input the g command into the kd> prompt, the program will continue with the execution until some other exception occurs either in the program itself or in the Windows system. When we execute the go command, the program will display a message about a system debugger being present as can be seen below:

This is a clear indication that the program was able to discover whether a system debugger is currently attached to it or not. But more importantly, we've seen how we can use the Windbg kernel debugger to interact with the Windows operating system.

Conclusion

In this tutorial, we've seen how we can approach kernel debugging if we need it. Most of the time, developers don't need to, but there are times when the need arises. Most often, this is used to debug some misbehaving kernel driver, but the technique is often used by hackers and malware writers or researchers to try to understand what the malware does. However, we must not forget rootkits, where using the technique above is mandatory and without which we won't be able to go very far.

You can even debug kernels for fun. Let's say some blue screen error keeps happening when you do the same action in Windows, but you have no idea why. In such cases, you're probably thinking: hell, I'll have to backup my data, reinstall the whole Windows operating system, install all the drivers, all the programs, and resynchronize the data back. But instead of doing all that cumbersome and time-consuming work, you can play it cool, attach a kernel debugger to the system and try to figure out why the blue screen keeps happening. When you figure it out, you can boot to Windows and fix the problem. Imagine your friends' mouths wide open when you tell them what you did :).

 

References:

[1] Setting Up Kernel-Mode Debugging of a Virtual Machine Manually, https://msdn.microsoft.com/en-us/library/windows/hardware/ff538143%28v=vs.85%29.aspx

[2] Live Kernel-Mode Debugging Using WinDbg, https://msdn.microsoft.com/en-us/library/windows/hardware/hh451166(v=vs.85).aspx.

[3] Getting Started with WinDbg (Kernel-Mode), https://msdn.microsoft.com/en-us/library/windows/hardware/dn745912(v=vs.85).aspx.

[4] Local Kernel-Mode Debugging, https://msdn.microsoft.com/en-us/library/windows/hardware/ff553382(v=vs.85).aspx.