Kernel Debugging with QEMU

Setup

Clone the repository

$ git clone https://github.com/torvalds/linux.git

For kernel dependencies, create a shell.nix file in the cloned repo

{ pkgs ? import <nixpkgs> {} }:

(pkgs.buildFHSUserEnv {
  name = "linux-kernel-build";
  targetPkgs = pkgs: (with pkgs;
  [
    getopt
    flex
    bison
    libelf
    ncurses.dev
    openssl.dev
    gcc
    gnumake
    bc

  ]);
  runScript = "bash";
}).env

Generate a config for KVM

If on make you get asked some questions, just press enter till you are done, this will select the default answer.

$ cd linux
$ nix-shell shell.nix
$ make x86_64_defconfig
$ make kvmconfig
$ scripts/config --set-val DEBUG_INFO y # For gdb debug symbols
$ scripts/config --set-val GDB_SCRIPTS y
$ make -j <number-cpu-cores>

Create a bootable Debian image with replaceable kernel

 $ nix-shell -p debootstrap qemu
 $ qemu-img create qemu-image.img
 $ mkfs.ext2 qemu-image.img
 $ mkdir mount-point.dir
 $ sudo mount -o loop qemu-image.img mount-point.dir
 $ sudo debootstrap --arch amd64 buster mount-point.dir
 $ sudo chroot mount-point.dir /bin/bash -i
 $ export PATH=$PATH:/bin
 $ passwd # Set root password
 $ exit
 $ sudo umount mount-point.dir
 $ rmdir mount-point.dir

Launch qemu

Discard the -enable-kvm flag if virtualisation.libvirtd.enable is not set in your configuration.nix. The nokaslr kernel flag is important to be able to set breakpoints in kernel memory.

 $ qemu-system-x86_64 -s -S \
    -kernel ../arch/x86/boot/bzImage \
    -hda qemu-img.img \
    -append "root=/dev/sda console=ttyS0 nokaslr" \
    -enable-kvm \
    -nographic

Connect with gdb

 $ echo "add-auto-load-safe-path `pwd`/scripts/gdb/vmlinux-gdb.py" >> ~/.gdbinit
 $ gdb ./vmlinux
 (gdb) target remote :1234
 (gdb) continue

Installing tools to the image

The filesystem is mounted read only so to add tools like lspci, mount and chroot then use apt to install the needed binaries.

 $ sudo  mount -o loop qemu-image.img mount-point.dir
 $ sudo chroot mount-point.dir /bin/bash -i
 $ export PATH=$PATH:/bin
 $ apt install pciutils tree