.Dd December 30, 2003 .Dt GDB 4 .Os .Sh NAME .Nm gdb .Nd external kernel debugger .Sh SYNOPSIS .Cd makeoptions DEBUG=-g .Cd options DDB .Cd options GDB_REMOTE_CHAT
p To prevent activation of the debugger on kernel .Xr panic 9 : .Cd options DDB_UNATTENDED .Sh DESCRIPTION The .Nm kernel debugger is a variation of .Xr gdb 1 which understands some aspects of the .Fx kernel environment. It can be used in a number of ways:
p l -bullet -offset indent -compact t It can be used to debug another system interactively via a serial or firewire link. In this mode, the processor can be stopped and single stepped. t It can be used to examine the memory of the processor on which it runs. t It can be used to analyse a processor dump after a panic. .El
p When used for remote debugging, .Nm requires the presence of the .Xr ddb 4 kernel debugger. Commands exist to switch between .Nm and .Xr ddb 4 . .Sh PREPARING FOR DEBUGGING When debugging kernels, it is practically essential to have built a kernel with debugging symbols .Cd ( makeoptions DEBUG=-g ). It's easiest to perform operations from the kernel build directory, by default
p First, ensure you have a copy of the debug macros in the directory: d -literal -offset 5m # \f(CBmake gdbinit .Ed
p This command performs some transformations on the macros installed in
a /usr/src/tools/debugscripts to adapt them to the local environment. .Ss Debugging a local machine To look at and change the contents of the memory of the system you're running on, d -literal -offset 5m # \f(CBgdb -k -wcore kernel.debug /dev/mem .Ed
p In this mode, you need the .Fl k flag to indicate to .Nm gdb that the ``dump file''
a /dev/mem is a kernel data file. You can look at live data, and if you include the .Fl wcore option, you can change it at your peril. The system does not stop (obviously), so a number of things will not work. You can set breakpoints, but you can't ``continue'' execution, so they won't work. .Ss Debugging a crash dump By default, crash dumps are stored in the directory
a /var/crash . Investigate them from the kernel build directory with: d -literal -offset 5m # \f(CBgdb -k kernel.debug /var/crash/vmcore.29 .Ed
p In this mode, the system is obviously stopped, so you can only look at it. .Ss Debugging a live system with a remote link To debug a live system with a remote link, the kernel must be compiled with the options .Cd options DDB and .Cd options GDB_REMOTE_CHAT . The option .Cd options BREAK_TO_DEBUGGER enables the debugging machine stop the debugged machine once a connection has been established by pressing .Li ^C. .Ss Debugging a live system with a remote serial link When using a serial port for the remote link on the i386 platform the serial port must be identified by setting the flag bit .Li 0x80 for the specified interface. Generally this port will also be used as a serial console (flag bit .Li 0x10a), so the entry in
a /boot/device.hints should be: d -literal -offset 5m hint.sio.0.flags="0x90" .Ed .Ss Debugging a live system with a remote firewire link As with serial debugging, to debug a live system with a firewire link, the kernel must be compiled with the options .Cd options DDB and .Cd options GDB_REMOTE_CHAT .
p A number of steps must be performed to set up a firewire link:
p l -bullet -offset indent -compact t First, ensure that the kernels of both systems include firewire support. If it isn't compiled into the kernel, load the klds: d -literal -offset 5m # \f(CBkldload dcons # \f(CBkldload dcons_crom .Ed
p You should see something like this in the .Nm dmesg output:
p d -literal -offset 5m fwohci0: BUS reset fwohci0: node_id=0xc000ffc1, gen=3, CYCLEMASTER mode firewire0: 2 nodes, maxhop <= 1, cable IRM = 1 (me) firewire0: bus manager 1 (me) firewire0: New S400 device ID:000199000003622b dcons_crom0: <dcons configuration ROM> on firewire0 dcons_crom0: bus_addr 0xf93000 .Ed
p The corresponding output on the other machine looks like this:
p d -literal -offset 5m fwohci0: BUS reset fwohci0: node_id=0x8800ffc0, gen=2, non CYCLEMASTER mode firewire0: 2 nodes, maxhop <= 1, cable IRM = 1 firewire0: bus manager 1 firewire0: New S400 device ID:00c04f3226e88061 dcons_crom0: <dcons configuration ROM> on firewire0 dcons_crom0: bus_addr 0x22a000 .Ed
p It's a good idea to load these modules at boot time with the following entry in
a /boot/loader.conf :
p d -literal -offset 5m dcons_enable=YES .Ed
p t Next, use .Nm fwcontrol to find the firewire node corresponding to the remote machine:
p d -literal -offset 5m # \f(CBfwcontrol 2 devices (info_len=2) node EUI64 status 1 0x00c04f3226e88061 0 0 0x000199000003622b 1 .Ed
p The first node is always the local system, so in this case, node 0 is the remote system. If there are more than two systems, check from the other end to find which node corresponds to the remote system. On the remote machine, it looks like this:
p d -literal -offset 5m # \f(CBfwcontrol 2 devices (info_len=2) node EUI64 status 0 0x000199000003622b 0 1 0x00c04f3226e88061 1 .Ed
p t Next, establish a firewire connection with .Nm fwchat :
p d -literal -offset 5m # \f(CBfwchat -b -t \f[CBI]target-address .Ed
p .Ar target-address is the EUI64 address of the remote node, in this case .Li 0x000199000003622b . When started in this manner, .Nm fwchat establishes two local tunnel connections: .Ar localhost:5555 is a connection to a .Nm getty process, which is not of interest here, and .Ar localhost:5556 is a connection to the debugger. The port numbers can be changed with the .Fl p flag to .Nm fwcontrol .
p .Nm fwcontrol does not return control to the user. You can start it in the background, but sometimes it will produce error messages, so it's a good idea to start it in its own window.
p t Finally, establish connection: d -literal -offset 5m # \f(CBgdb kernel.debug GNU gdb 5.2.1 (FreeBSD) \fI(politcal statements omitted)\/ Ready to go. Enter 'tr' to connect to the remote target with /dev/cuaa0, 'tr /dev/cuaa1' to connect to a different port or 'trf portno' to connect to the remote target with the firewire interface. portno defaults to 5556. Type 'getsyms' after connection to load kld symbols. If you're debugging a local system, you can use 'kldsyms' instead to load the kld symbols. That's a less obnoxious interface. (gdb) \f(CBtrf 0xc21bd378 in ?? () .Ed
p t It's currently possible for the two ends of the firewire link to get out of sync after a reboot. On the .Nm fwchat screen you see: d -literal -offset 5m fwchat: get crom faild .Ed
p In this case, stop .Nm fwchat and perform a firewire bus reset:
p d -literal -offset 5m # \f(CBfwcontrol -r # \f(CBfwchat -b -t 0x000199000003622b .El .Sh COMMANDS The user interface to .Nm is via .Xr gdb 1 , so .Xr gdb 1 commands also work. This section discusses only the extensions for kernel debugging that get installed in the kernel build directory. .Ss "Debugging Environment" The following macros manipulate the debugging environment: l -ohang -offset 3m t Cm ddb Switch back to .Nm ddb . This command is only meaningful when performing remote debugging. t Cm getsyms Display .Nm kldstat information for the target machine and invite user to paste it back in. This is required because .Nm gdb does not allow data to be passed to shell scripts. It's necessary for remote debugging and crash dumps; for local memory debugging use .Nm kldsyms instead. t Cm kldsyms Read in the symbol tables for the debugging machine. This doesn't work for remote debugging and crash dumps; use .Nm getsyms instead. t Cm tr Ar interface Debug a remote system via the specified serial or firewire interface. t Cm tr0 Debug a remote system via serial interface
a /dev/cuaa0 . t Cm tr1 Debug a remote system via serial interface
a /dev/cuaa1 . t Cm trf Debug a remote system via firewire interface at default port 5556. .El
p The commands .Nm tr0 , .Nm tr1 and .Nm trf are convenience commands which invoke .Nm tr . .Ss "The current process environment" The following macros are convenience functions intended to make things easier than the standard .Nm gdb commands. l -ohang -offset 3m t Cm f0 Select stack frame 0 and show assembler-level details. t Cm f1 Select stack frame 1 and show assembler-level details. t Cm f2 Select stack frame 2 and show assembler-level details. t Cm f3 Select stack frame 3 and show assembler-level details. t Cm f4 Select stack frame 4 and show assembler-level details. t Cm f5 Select stack frame 5 and show assembler-level details. t Cm xb Show 12 words in hex, starting at current .Va ebp value. t Cm xi List the next 10 instructions from the current .Va eip value. t Cm xp Show the register contents and the first four parameters of the current stack frame. t Cm xp0 Show the first parameter of current stack frame in various formats. t Cm xp1 Show the second parameter of current stack frame in various formats. t Cm xp2 Show the third parameter of current stack frame in various formats. t Cm xp3 Show the fourth parameter of current stack frame in various formats. t Cm xp4 Show the fifth parameter of current stack frame in various formats. t Cm xs Show the last 12 words on stack in hexadecimal. t Cm xxp Show the register contents and the first ten parameters. t Cm z Single step 1 instruction (over calls) and show next instruction. t Cm zs Single step 1 instruction (through calls) and show next instruction. .El .Ss "Examining other processes" The following macros access other processes. .Nm gdb does not understand the concept of multiple processes, so they effectively bypass the entire .Nm gdb environment. l -ohang -offset 3m t Cm btp Ar pid Show a backtrace for the process .Va pid . t Cm btpa Show backtraces for all processes in the system. t Cm btpp Show a backtrace for the process previously selected with .Nm defproc . t Cm btr Ar ebp Show a backtrace from the .Va ebp address specified t Cm defproc Ar pid Specify the PID of the process for some other commands in this section. t Cm fr Ar frame Show frame .Va frame of the stack of the process previously selected with .Nm defproc . t Cm pcb Ar proc Show some pcb contents of process .Ar proc . .El .Ss "Examining data structures" You can use standard .Nm gdb commands to look at most data structures. The macros in this section are convenience functions which typically display the data in a more readable format, or which omit less interesting parts of the structure. l -ohang -offset 3m t Cm bp Show information about the buffer header pointed to by the variable .Va bp in the current frame. t Cm bpd Show the contents .Vt (char*) of .Va bp->data in the current frame. t Cm bpl Show detailed information about the buffer header .Vt (struct bp) pointed at by the local variable .Va bp . t Cm bpp bp Show summary information about the buffer header .Vt (struct bp) pointed at by the parameter .Va bp . t Cm bx Print a number of fields from the buffer header pointed at in by the pointer .Va bp in the current environment. t Cm vdev Show some information of the vnode pointed to by the local variable .Va vp . .El .Ss "Miscellaneous macros" l -ohang -offset 3m t Cm checkmem Check unallocated memory for modifications. This assumes that the kernel has been compiled with .Cd options DIAGNOSTIC This causes the contents of free memory to be set to .Li 0xdeadc0de . t Cm dmesg Print the system message buffer. This corresponds to the .Nm dmesg command. It can take a very long time over a serial line, and it's even slow via firewire or local memory due to inefficiencies in .Nm gdb . This macro used to be called .Nm msgbuf . t Cm kldstat Equivalent of the kldstat(8) command without options t Cm pname Print the command name of the current process. t Cm ps Show process status. This corresponds in concept, but not in appearance, to the .Nm ps command. t Cm y Kludge for writing macros. When writing macros, it's convenient to paste them back into the .Nm gdb window. Unfortunately, if the macro is already defined, .Nm gdb insists on asking d -literal -offset 5m Redefine foo? .Ed
p
It won't give up until you answer
.Li y .
This command is that answer. It does nothing else except to print a warning
message to remind you to remove it again.
.El
.Sh AUTHORS
This man page was written by
.An "Greg Lehey" Aq grog@FreeBSD.org
.Sh SEE ALSO
.Xr ddb 4 ,
.Xr fwchat 8 ,
.Xr fwcontrol 8 ,
.Xr gdb 1 ,
.Xr vinumdebug 4 .
.Sh HISTORY
.Sh BUGS
l -bullet -offset indent -compact t .Nm
was never designed to debug kernels, and it's not a very good match.
Many problems exist.
t The debugging macros ``just growed''.
In general, the person who wrote them did so while looking for a specific
problem, so they may not be general enough, and they may behave badly when used
in ways for which they were not intended, even if those ways make sense.
t Serial debugging is very slow, and race conditions can make it difficult to run
the link at more than 9600 bps. Firewire connections do not have this problem.
t Many of these commands only work on the ia32 architecture.
.El