NixOS Wiki

Kernel Debugging with QEMU: Difference between revisions

← Older edit
Visual
imported>Luis-Hebendanz
No edit summary
Preisi (talk | contribs)
1 edit
m Increase image size as 1024 is no longer sufficient
 
(10 intermediate revisions by 3 users not shown)
Line 6: Line 6:


For kernel dependencies,
For kernel dependencies,
create a <code>shell.nix</code> file in the cloned repo
create a <code>shell.nix</code> file in the cloned repo:
 
<syntaxhighlight lang="nix">
{ pkgs ? import <nixpkgs> {} }:
 
pkgs.stdenv.mkDerivation {
  name = "linux-kernel-build";
  nativeBuildInputs = with pkgs; [
    getopt
    flex
    bison
    gcc
    gnumake
    bc
    pkg-config
    binutils
  ];
  buildInputs = with pkgs; [
    elfutils
    ncurses
    openssl
    zlib
  ];
}
</syntaxhighlight>
 
Older kernel versions might buildFHSUserEnv as they have absolute shebangs:
 
<syntaxhighlight lang="nix">
<syntaxhighlight lang="nix">
{ pkgs ? import <nixpkgs> {} }:
{ pkgs ? import <nixpkgs> {} }:
Line 12: Line 39:
(pkgs.buildFHSUserEnv {
(pkgs.buildFHSUserEnv {
   name = "linux-kernel-build";
   name = "linux-kernel-build";
   targetPkgs = pkgs: (with pkgs;
   targetPkgs = pkgs: (with pkgs; [
  [
     getopt
     getopt
     flex
     flex
     bison
     bison
     libelf
     elfutils
    binutils
     ncurses.dev
     ncurses.dev
     openssl.dev
     openssl.dev
    zlib.dev
     gcc
     gcc
     gnumake
     gnumake
     bc
     bc
   ]);
   ]);
   runScript = "bash";
   runScript = "bash";
Line 34: Line 61:
<syntaxhighlight lang="console">
<syntaxhighlight lang="console">
$ cd linux
$ cd linux
$ make mrproper # Clears all artifacts
$ make mrproper # Clears all artifacts, do this especially if you upgrade from a significant old version
$ nix-shell shell.nix
$ nix-shell shell.nix
$ make x86_64_defconfig
$ make defconfig kvm_guest.config
$ make kvmconfig
$ scripts/config --set-val DEBUG_INFO y # For gdb debug symbols
$ scripts/config --set-val DEBUG_INFO y # For gdb debug symbols
$ scripts/config --set-val DEBUG y # All pr_debug messages get printed
$ scripts/config --set-val DEBUG y # All pr_debug messages get printed
$ scripts/config --set-val GDB_SCRIPTS y
$ scripts/config --set-val GDB_SCRIPTS y
$ scripts/config --set-val DEBUG_DRIVER y # Enable printk messages in drivers
$ scripts/config --set-val DEBUG_DRIVER y # Enable printk messages in drivers
# everything as one command for copy'n'paste
$ scripts/config --set-val DEBUG_INFO y --set-val DEBUG y  --set-val GDB_SCRIPTS y --set-val DEBUG_DRIVER y
# this might ask for further options, just press enter for every question
$ make -j$(nproc)
$ make -j$(nproc)
</syntaxhighlight>
</syntaxhighlight>
== Create a bootable NixOS image with no kernel ==
Save this as <code>nixos-image.nix</code>:
<syntaxhighlight lang="nix">{ pkgs ? import <nixpkgs> {} }:
import (pkgs.path + "/nixos/lib/make-disk-image.nix") {
  config = (import (pkgs.path + "/nixos/lib/eval-config.nix") {
    inherit (pkgs) system;
    modules = [{
      imports = [ ./nixos-config.nix ];
    }];
  }).config;
  inherit pkgs;
  inherit (pkgs) lib;
  diskSize = 2048;
  partitionTableType = "none";
  # for a different format
  format = "qcow2";
}</syntaxhighlight>
Than follows the nixos configuration in a file named <code>nixos-config.nix</code>
<syntaxhighlight lang="nix">
{ pkgs, lib, modulesPath, ... }:
{
  imports = [
    (modulesPath + "/profiles/qemu-guest.nix")
  ];
  boot.loader.grub.enable = false;
  boot.initrd.enable = false;
  boot.isContainer = true;
  boot.loader.initScript.enable = true;
  ## login with empty password
  users.extraUsers.root.initialHashedPassword = "";
  networking.firewall.enable = false;
  services.getty.helpLine = ''
    Log in as "root" with an empty password.
    If you are connect via serial console:
    Type Ctrl-a c to switch to the qemu console
    and `quit` to stop the VM.
  '';
  services.getty.autologinUser = lib.mkDefault "root";
  documentation.doc.enable = false;
  documentation.man.enable = false;
  documentation.nixos.enable = false;
  documentation.info.enable = false;
  programs.bash.enableCompletion = false;
  programs.command-not-found.enable = false;
}
</syntaxhighlight>
Than build with the following commands:
<syntaxhighlight lang="console">
$ nix-build
# copy out
$ install -m644 result/nixos.qcow2 qemu-image.img
</syntaxhighlight>
Than follow with the next step is launching qemu.


== Create a bootable Debian image with replaceable kernel ==
== Create a bootable Debian image with replaceable kernel ==
If you want to build a different Linux distro you can use the following instructions to build a debian instead:
<syntaxhighlight lang="console">
<syntaxhighlight lang="console">
  $ nix-shell -p debootstrap qemu
  $ nix-shell -p debootstrap qemu
Line 58: Line 156:
  $ exit
  $ exit
  $ sudo umount mount-point.dir
  $ sudo umount mount-point.dir
</syntaxhighlight>
=== Installing tools to the image ===
The filesystem is mounted read only so to add tools like lspci. Mount and chroot then use apt to install the needed binaries.
<syntaxhighlight lang="console">
$ sudo  mount -o loop qemu-image.img mount-point.dir
$ sudo chroot mount-point.dir /bin/bash -i
$ export PATH=$PATH:/bin
$ apt install pciutils tree
$ sudo umount mount-point.dir
</syntaxhighlight>
</syntaxhighlight>


== Launch qemu ==
== Launch qemu ==
You can find a slighty stripped version of qemu in a package called <code>qemu_kvm</code> (qemu without emulation support for other cpu architectures).
The <code>nokaslr</code> kernel flag is important to be able to set breakpoints in kernel memory.
The <code>nokaslr</code> kernel flag is important to be able to set breakpoints in kernel memory.
You can also skip the <code>-S</code> to not make qemu break on startup and waiting for gdb.
<syntaxhighlight lang="console">
<syntaxhighlight lang="console">
  $ qemu-system-x86_64 -s -S \
  $ qemu-system-x86_64 -s -S \
Line 81: Line 192:
If a breakpoint is not triggered as expected try to set the breakpoint later when the VM
If a breakpoint is not triggered as expected try to set the breakpoint later when the VM
is fully booted.
is fully booted.
== Installing tools to the image ==
The filesystem is mounted read only so to add tools like lspci. Mount and chroot then use apt to install the needed binaries.
<syntaxhighlight lang="console">
$ sudo  mount -o loop qemu-image.img mount-point.dir
$ sudo chroot mount-point.dir /bin/bash -i
$ export PATH=$PATH:/bin
$ apt install pciutils tree
$ sudo umount mount-point.dir
</syntaxhighlight>


== Language server support ==
== Language server support ==
If you want language server support for the kernel code you can generate a compile_commands.json with
If you want language server support for the kernel code you can generate a compile_commands.json with
<syntaxhighlight lang="console">
<syntaxhighlight lang="console">
$ python ./scripts/gen_compile_commands.py  
$ python ./scripts/clang-tools/gen_compile_commands.py  
</syntaxhighlight>
</syntaxhighlight>
This can be used for example in combination with clangd, which scales well to size of the linux kernel.


== Debugging drivers ==
== Debugging drivers ==
Line 104: Line 207:
</syntaxhighlight>
</syntaxhighlight>
press <code>F8</code> and search for your driver, and check if it is set to "Module" with <code><M></code>. After compilation copy the driver.ko into the mounted <code>qemu-image.img</code>. Unmount start the kernel and break at the <code>load_module</code> function and <code>insmod driver.ko</code>. Happy hacking!
press <code>F8</code> and search for your driver, and check if it is set to "Module" with <code><M></code>. After compilation copy the driver.ko into the mounted <code>qemu-image.img</code>. Unmount start the kernel and break at the <code>load_module</code> function and <code>insmod driver.ko</code>. Happy hacking!
== Bugs ==
1. With the nixos-config provided above, the console does not work properly. boot.isContainer = true; implies console.enable = false; that disables console. The following can be used as a workaround.
<syntaxhighlight lang="nix">
console.enable = true;
systemd.services."serial-getty@ttyS0".enable = true;
</syntaxhighlight>
== Using ktest with NixOS ==
Yellow onion has integrated nixos vms into ktest:
https://github.com/YellowOnion/ktest/commit/73fadcff949236927133141fcba4bfd76df632e7
This integration also allows to use incremental kernel builds for rapid development. Checkout the commit message for details.
[[Category:Virtualization]]
Retrieved from "https://wiki.nixos.org/wiki/Kernel_Debugging_with_QEMU"