debugging methods:

https://github.com/nanovms/nanos/wiki/using-the-qemu-remote-gdb-interface

printfs 32, 64, and process environments support rprintf, which doesn't attempt to hew very closely to the standard printf, but does support % formats and varargs, and allows modules to register their own single character formatters. '%v' is interesting in that its supposed to print any value which is dynamically tagged (i.e. tuples and their associated types). '%p' prints a pointer of the native type.

fault reports. if you get past initializing the interrupt system, a fault should dump the current register file and the tail of the stack

gdb stub. if you get up into user space, you can enable the remote gdb stub.

to run gdb on a non-recoverable fault, add fault:t to the top level of the manifest

on a fault, after the stack dump, you should see 'starting gdb'

the network stack should have come up and you should see 'assigned 192.168.42.48' (or whatever)

if both of those things are true, enter gdb, type 'target remote 192.168.42.48:1234'

to load the kernel symbols, use 'symbol-file stage3/stage3'. you can also do something like 'add-symbol-file /lib64/ld-linux-x86-64.so.2 0x0500000000' if you know where a library has been loaded.

bt, stepi, breakpoints and such should work

set breakpoints and watch points directly. x86_64/breakpoint.c can be called directly from C to generate a trap when an instruction is fetched, a read, or write is performed.

• qemu based debugging

  • '-d int -D foo' will generate a report all of interrupts into the file 'foo'. the format of the register file gives you a good hint what mode the processor is in. this report isn't generated when kvm is enabled

  • '-s -S' turns on the qemu gdb stub (at :1234). the processor starts halted before the bios. set a breakpoint at the generated address for stage2:run64, single step until it complains about the returned context being too large (g packet reply too long). 'disconnect', 'set architecture i386:x86-64:intel', 'target remote :1234', and you can continue execution up into stage3. this also isn't supported when running with kvm.

  • instrumenting qemu. if you're at a loss as to why the virtio driver isn't responding, you can check out a copy of qemu, add printfs, and go to town. qemu has alot of internal strata, so expect to spend a while tracking down the control flow, but introspecting the 'hardware' can be super helpful.

Common Debugging Problems/Solutions:

Q: If when running with '-d' and you don't see any output and you don't ever get to the debug message of "Read program complete:" then:

A: Try bumping up your memory in config.json:

  "RunConfig": {
    "Memory": "4G"
  },