Debugging a Kernel Extension with GDB
In this tutorial, you learn how to debug a kext. You set up a two-machine debugging environment and use GDB to perform remote debugging. If you have not yet created a kext, complete Creating a Generic Kernel Extension with Xcode or Creating a Device Driver with Xcode before completing this tutorial. If you are unfamiliar with GDB, see Debugging with GDB.
Although this tutorial is written with a device driver as the example, the steps for debugging are similar for debugging any type of kext.
You need two machines for remote debugging: a target machine and a development machine. You load and run the kext on the target machine and debug the kext on the development machine. It is important to keep the two machines clear in your mind as you work through this tutorial, because you will be moving back and forth between them many times. It may help if you take a piece of paper, tear it in half, and write “Development” on one piece and “Target” on the other. Then place the pieces of paper next to the two machines.
These are the major steps you will follow:
Start GDB and Connect the Two Machines
Debug the Kernel Extension
Prepare the Machines
Prepare the two machines by doing the following:
Ensure that the machines are running the same version of OS X.
Ensure that the machines are connected to the same network with their built-in Ethernet ports.
Ensure that you are logged in as an administrator on both machines, which is necessary for using the
Mount the Kernel Debug Kit disk image on the development machine. Download the Kernel Debug Kit from the Apple Developer website under the OS X download category. Make sure the Kernel Debug Kit you download matches the version of OS X installed on your target machine.
For more information on the Kernel Debug Kit, see the Read Me file included in the disk image.
If your target machine is running OS X Server, disable the OS X Server watchdog timer.
$ sudo killall -TERM watchdogtimerd
For more information, see the manual page for
(Development Machine) Sabotage the Kernel Extension
To better simulate a real-world kext debugging scenario, you need your kext to produce a kernel panic. The easiest way to do this is to dereference a null pointer.
In Xcode, add the following code to your driver’s
startmethod (if you are debugging a generic kext, add it to the kext’s
char *kernel_panic = NULL;
char message = *kernel_panic;
Rebuild your kext. In the Terminal application, create a copy of the kext as root:
$ sudo cp -R MyDriver.kext /tmp
Copy your kext’s
dSYMfile (in the Xcode project’s build folder with your kext) to the same location you copy your kext.
Transfer the copy of your kext from the development machine to the target machine.
If the transferred copy of your kext has an incorrect owner or group, correct it with the following command:
$ sudo chown -R root:wheel MyDriver.kext
(Target Machine) Enable Kernel Debugging
Before you can debug your kext, you must first enable kernel debugging. On the target machine, do the following:
Start the Terminal application.
Set the kernel debug flags.
To enable kernel debugging, you must set an NVRAM (nonvolatile random access memory) variable:
$ sudo nvram boot-args="debug=0x144 -v"
For more information on debugging flags, see Building and Debugging Kernels in Kernel Programming Guide.
Restart the computer for the debugging flags to take effect.
Start GDB and Connect the Two Machines
(Target Machine) Get the Target Machine’s IP Address
To connect to the target machine from the development machine, you need the target machine’s IP address. If you don’t already know it, you can find it in the Network pane of the System Preferences application.
(Development Machine) Start GDB
Start GDB with the following command, indicating the target machine’s architecture and the location of your debug kernel:
$ gdb -arch i386 /Volumes/KernelDebugKit/mach_kernel
Add kernel-specific macros from the Kernel Debug Kit to your GDB session.
(gdb) source /Volumes/KernelDebugKit/kgmacros
Inform GDB that you are going to be debugging a kext remotely.
(gdb) target remote-kdp
(Target Machine) Load the Kernel Extension
You are ready to load your kext and cause a kernel panic. Do so with the following command:
$ sudo kextutil MyDriver.kext
The kernel panic should occur immediately. Interactivity ceases and debugging text appears on the screen, including the text
Awaiting debugger connection.
(Development Machine) Attach to the Target Machine
Now you can tell GDB to attach to the target machine. On the development machine, do the following:
Attach to the target machine. At the GDB prompt, use the
kdp-reattachmacro with the target machine’s name or IP address:
(gdb) kdp-reattach target.apple.com
The target machine prints:
Connected to remote debugger.
(Development Machine) Get the Load Address of the Kernel Extension
You need your kext’s load address in order to generate a symbol file for it. Enter the following at the GDB prompt:
A list appears displaying information about every kext running on the target machine. Find your kext in the list and write down the value in the address column. Note that the values in the kmod and size columns look similar to the address value, so make sure you have the correct value.
(Development Machine) Create and Load the Symbol File
You can create a symbol file for your kext on the development machine with the
kextutil command. Once again, make sure that the version of the Kernel Debug Kit you provide matches the version of OS X on the target machine.
Open a second Terminal window.
Create the symbol file.
Adjust the path to the Kernel Debug Kit, the path to your kext, and the architecture of the kernel as appropriate. The path after the
-soption specifies the output directory where the symbol file is written. This should be the directory you copied your kext and
dSYMfile to. The
-noption prevents the command from loading the kext into the kernel.
sudo kextutil -s /tmp -n -arch i386 -k /Volumes/KernelDebugKit/mach_kernel -e -r /Volumes/KernelDebugKit /tmp/MyDriver.kext
kextutiltool prompts you for the load address of your kext. Provide the load address you obtained in the previous step.
When you finish, the symbol file is in the output directory you specified. The filename is the bundle identifier of the kext with the
At the GDB prompt, specify the location of the symbol file.
Again, make sure the symbol file is in the same folder as the copy of your kext and
(gdb) set kext-symbol-file-path /tmp
At the GDB prompt, add your kext to the debug environment with the following macro:
(gdb) add-kext /tmp/MyDriver.kext
GDB asks you if you want to add the kext’s symbol file. When you confirm, it loads the symbols.
Debug the Kernel Extension
(Development Machine) Debug with GDB
Now you are ready to begin debugging! Request a backtrace from GDB to locate the source of the panic:
A list of stack frames appears. Your kext’s stack frame that caused the panic should be easily identifiable as about the fifth frame from the top. When you are debugging your own kernel panics and don’t know the cause, you can now enter the offending stack frame and figure out what exactly caused the panic.
(Development Machine) Stop the Debugger
When you've finished debugging, stop the debugger by quitting GDB.
The debugging session ends. Because the target machine is still panicked, you need to reboot it. When you log back into the target machine, it displays the following message:
Where to Go Next
Congratulations! You've learned how to set up a two-machine debugging environment to debug a kext with GDB. To learn how to package your kext for installation by your customers, read Packaging a Kernel Extension for Distribution and Installation.