Linux kernel: Difference between revisions
imported>Toraritte m Add NixOS category |
m Update Nix code for out-of-tree kernel modules according to changes made during a PR review by Aleksanaa in https://github.com/NixOS/nixpkgs/pull/334445 |
||
(36 intermediate revisions by 21 users not shown) | |||
Line 3: | Line 3: | ||
== Configuration == | == Configuration == | ||
You can | You can change the installed kernel using {{nixos:option|boot.kernelPackages}}. To use the latest release: | ||
{{file|/etc/nixos/configuration.nix|nix|<nowiki> | |||
{ | |||
boot.kernelPackages = pkgs.linuxPackages_latest; | boot.kernelPackages = pkgs.linuxPackages_latest; | ||
</ | } | ||
</nowiki>}} | |||
Then rebuild your system and reboot to use your new kernel: | |||
<syntaxHighlight lang="console"> | <syntaxHighlight lang="console"> | ||
$ sudo nixos-rebuild boot | $ sudo nixos-rebuild boot | ||
Line 17: | Line 19: | ||
=== List available kernels === | === List available kernels === | ||
You can list available kernels using <code>nix repl</code> (previously <code>nix-repl</code>) by typing the package name and using the tab completion: | You can list available kernels using <code>nix repl</code> (previously <code>nix-repl</code>) by typing the package name and using the tab completion: | ||
<syntaxHighlight lang="console"> | <syntaxHighlight lang="console"> | ||
$ nix repl | $ nix repl | ||
Line 26: | Line 28: | ||
nix-repl> pkgs.linuxPackages | nix-repl> pkgs.linuxPackages | ||
pkgs.linuxPackages pkgs. | pkgs.linuxPackages pkgs.linuxPackages_custom | ||
pkgs.linuxPackages-libre pkgs. | pkgs.linuxPackages-libre pkgs.linuxPackages_custom_tinyconfig_kernel | ||
pkgs.linuxPackages-rt pkgs. | pkgs.linuxPackages-rt pkgs.linuxPackages_hardened | ||
pkgs.linuxPackages- | pkgs.linuxPackages-rt_latest pkgs.linuxPackages_latest | ||
pkgs. | pkgs.linuxPackagesFor pkgs.linuxPackages_latest-libre | ||
pkgs. | pkgs.linuxPackages_4_14 pkgs.linuxPackages_latest_hardened | ||
pkgs. | pkgs.linuxPackages_4_19 pkgs.linuxPackages_latest_xen_dom0 | ||
pkgs. | pkgs.linuxPackages_4_19_hardened pkgs.linuxPackages_latest_xen_dom0_hardened | ||
pkgs. | pkgs.linuxPackages_4_9 pkgs.linuxPackages_lqx | ||
pkgs. | pkgs.linuxPackages_5_10 pkgs.linuxPackages_rpi0 | ||
pkgs. | pkgs.linuxPackages_5_10_hardened pkgs.linuxPackages_rpi02w | ||
pkgs.linuxPackages_5_4 pkgs.linuxPackages_testing | pkgs.linuxPackages_5_15 pkgs.linuxPackages_rpi1 | ||
pkgs. | pkgs.linuxPackages_5_15_hardened pkgs.linuxPackages_rpi2 | ||
pkgs. | pkgs.linuxPackages_5_18 pkgs.linuxPackages_rpi3 | ||
pkgs. | pkgs.linuxPackages_5_19 pkgs.linuxPackages_rpi4 | ||
pkgs. | pkgs.linuxPackages_5_4 pkgs.linuxPackages_rt_5_10 | ||
pkgs. | pkgs.linuxPackages_5_4_hardened pkgs.linuxPackages_rt_5_15 | ||
pkgs. | pkgs.linuxPackages_6_0 pkgs.linuxPackages_rt_5_4 | ||
pkgs.linuxPackages_6_1 pkgs.linuxPackages_rt_6_1 | |||
pkgs.linuxPackages_6_1_hardened pkgs.linuxPackages_testing | |||
pkgs.linuxPackages_6_2 pkgs.linuxPackages_testing_bcachefs | |||
pkgs.linuxPackages_6_3 pkgs.linuxPackages_xanmod | |||
pkgs.linuxPackages_6_4 pkgs.linuxPackages_xanmod_latest | |||
pkgs.linuxPackages_6_5 pkgs.linuxPackages_xanmod_stable | |||
pkgs.linuxPackages_6_6 pkgs.linuxPackages_xen_dom0 | |||
pkgs.linuxPackages_6_6_hardened pkgs.linuxPackages_xen_dom0_hardened | |||
pkgs.linuxPackages_6_7 pkgs.linuxPackages_zen | |||
pkgs.linuxPackages_6_8 | |||
</syntaxhighlight> | </syntaxhighlight> | ||
Line 51: | Line 62: | ||
Note that if you deviate from the default kernel version, you should also take extra care that extra kernel modules must match the same version. The safest way to do this is to use <code>config.boot.kernelPackages</code> to select the correct module set: | Note that if you deviate from the default kernel version, you should also take extra care that extra kernel modules must match the same version. The safest way to do this is to use <code>config.boot.kernelPackages</code> to select the correct module set: | ||
< | {{file|/etc/nixos/configuration.nix|nix|<nowiki> | ||
{ config, ... }: | { config, ... }: | ||
{ | { | ||
boot.extraModulePackages = with config.boot.kernelPackages; [ | boot.extraModulePackages = with config.boot.kernelPackages; [ bbswitch ]; | ||
} | } | ||
</ | </nowiki>}} | ||
=== Customizing kernel module parameters === | |||
You can use {{nixos:option|boot.extraModprobeConfig}}, which is analogous to creating modprobe.conf files in /etc/modprobe.d/ in regular Linux distributions. This can be used for both built-in and loadable kernel modules. | |||
{{file|/etc/nixos/configuration.nix|nix|<nowiki> | |||
{ | |||
boot.extraModprobeConfig = '' | |||
# example settings | |||
options yourmodulename optionA=valueA optionB=valueB # syntax | |||
options thinkpad_acpi fan_control=1 # example #1 kernel module parameter | |||
options usbcore blinkenlights=1 # example #2 kernel module parameter | |||
''; | |||
} | |||
</nowiki>}} | |||
{{nixos:option|boot.kernelParams}} can be used to set additional kernel command line arguments at boot time. It can only be used for built-in modules. For example: | |||
{ | |||
{{file|/etc/nixos/configuration.nix|nix|<nowiki> | |||
{ | { | ||
boot.kernelParams = [ | boot.kernelParams = [ | ||
"quiet" | |||
"splash" | |||
"usbcore.blinkenlights=1" | |||
]; | |||
} | } | ||
</ | </nowiki>}} | ||
=== Enable SysRq === | |||
The Linux kernel is known<ref>https://github.com/hakavlad/nohang?tab=readme-ov-file#what-is-the-problem</ref> to not handle out-of-memory situation properly and can freeze for a long time, often leaving no option but doing a hard reboot. The [https://en.wikipedia.org/wiki/Magic_SysRq_key SysRq shortcuts] can be used to trigger a more graceful reboot. However, most keys are disabled by default on NixOS. To enable: | |||
{{file|/etc/nixos/configuration.nix|nix|<nowiki> | |||
{ | |||
boot.kernel.sysctl."kernel.sysrq" = 1; | |||
} | |||
</nowiki>}} | |||
Useful shortcuts, triggered using {{ic|Alt+SysRq+<key>}}: | |||
* {{ic|h}}: Print help to the system log. | |||
* {{ic|f}}: Trigger the kernel oom killer. | |||
* {{ic|s}}: Sync data to disk before triggering the reset options below. | |||
* {{ic|e}}: SIGTERM all processes except PID 0. | |||
* {{ic|i}}: SIGKILL all processes except PID 0. | |||
* {{ic|b}}: Reboot the system. | |||
Check <code>journalctl</code> to see if you are triggering the shortcuts correctly, which might be different for your keyboard, as noted in the Wikipedia page. | |||
Also see {{nixos:option|services.earlyoom.enable}} and {{nixos:option|systemd.oomd.enable}}. | |||
=== Custom configuration === | === Custom configuration === | ||
It is sometimes desirable to change the configuration of your kernel, while keeping the kernel version itself managed through Nixpkgs | It is sometimes desirable to change the configuration of your kernel, while keeping the kernel version itself managed through Nixpkgs. | ||
One way is to use {{nixos:option|boot.kernelPatches}}<ref>[https://logs.nix.samueldr.com/nixos/2018-05-09#1525898458-1525898534; #nixos 2018-05-09]</ref><ref>[https://github.com/NixOS/nixpkgs/blob/03d4694e6110fa9c16e88ee74085ea2068c27494/nixos/modules/misc/crashdump.nix#L63-L73 <tt>nixos/modules/misc/crashdump.nix#L63-L73</tt>]</ref>. For example, {{nixos:option|boot.crashDump.enable}} is configured as shown below. Note that the {{ic|CONFIG_}} prefix is not used in the configuration names. | |||
<syntaxhighlight lang=nix> | <syntaxhighlight lang=nix> | ||
Line 94: | Line 141: | ||
</syntaxhighlight> | </syntaxhighlight> | ||
Another way | Another way is to create a overlay to configure either {{ic|extraConfig}} or {{ic|structuredExtraConfig}}. This allows for more flexibility, since you can basically override any available kernel option<ref>[https://github.com/NixOS/nixpkgs/blob/e6db435973160591fe7348876a5567c729495175/pkgs/os-specific/linux/kernel/generic.nix#L11-L56<tt>pkgs/os-specific/linux/kernel/generic.nix#L11-L56</tt>]</ref>. Using {{ic|structuredExtraConfig}} is recommended as it checks if the configured option are correct. You may also want to set {{ic|ignoreConfigErrors}} to avoid {{ic|error: unused option}} during builds. | ||
< | {{file|/etc/nixos/configuration.nix|nix|<nowiki> | ||
{ | { | ||
nixpkgs = { | nixpkgs = { | ||
overlays = [ | overlays = [ | ||
(self: super: { | (self: super: { | ||
linuxZenWMuQSS = pkgs.linuxPackagesFor (pkgs.linux_zen.override { | linuxZenWMuQSS = pkgs.linuxPackagesFor (pkgs.linux_zen.kernel.override { | ||
structuredExtraConfig = with lib.kernel; { | structuredExtraConfig = with lib.kernel; { | ||
SCHED_MUQSS = yes; | SCHED_MUQSS = yes; | ||
Line 111: | Line 158: | ||
}; | }; | ||
} | } | ||
</ | </nowiki>}} | ||
=== Pinning a kernel version === | === Pinning a kernel version === | ||
< | {{file|/etc/nixos/configuration.nix|nix|<nowiki> | ||
boot.kernelPackages = pkgs.linuxPackagesFor (pkgs.linux_4_19.override { | { | ||
boot.kernelPackages = pkgs.linuxPackagesFor (pkgs.linux_4_19.override { | |||
argsOverride = rec { | argsOverride = rec { | ||
src = pkgs.fetchurl { | src = pkgs.fetchurl { | ||
Line 124: | Line 172: | ||
version = "4.19.60"; | version = "4.19.60"; | ||
modDirVersion = "4.19.60"; | modDirVersion = "4.19.60"; | ||
}; | |||
}); | }); | ||
</ | } | ||
</nowiki>}} | |||
=== Debugging a failed configuration === | === Debugging a failed configuration === | ||
As dependencies between kernel configurations | As dependencies between kernel configurations need to be addressed manually, use [https://nixos.org/manual/nix/stable/command-ref/nix-build#opt-keep-failed --keep-failed] to inspect the intermediate nix config file after any build failures: | ||
{{bc|note: keeping build directory '/tmp/nix-build-linux-config-4.19.0-mptcp_v0.94.1.drv-0'}} | {{bc|note: keeping build directory '/tmp/nix-build-linux-config-4.19.0-mptcp_v0.94.1.drv-0'}} | ||
== Embedded Linux Cross-compile == | |||
To configure and cross-compile Linux kernels for embedded development, often distributed separately instead of using the stock kernel, you can setup a development environment as shown below: | |||
{{file|shell.nix|nix|3=let | |||
pkgs = import <nixpkgs> { }; | |||
in | |||
(pkgs.buildFHSUserEnv { | |||
name = "kernel-build-env"; | |||
targetPkgs = pkgs: (with pkgs; | |||
[ | |||
pkgconfig | |||
ncurses | |||
qt5.qtbase | |||
# new gcc usually causes issues with building kernel so use an old one | |||
pkgsCross.aarch64-multiplatform.gcc8Stdenv.cc | |||
(hiPrio gcc8) | |||
] | |||
++ pkgs.linux.nativeBuildInputs); | |||
runScript = pkgs.writeScript "init.sh" '' | |||
export ARCH=arm64 | |||
export hardeningDisable=all | |||
export CROSS_COMPILE=aarch64-unknown-linux-gnu- | |||
export PKG_CONFIG_PATH="${pkgs.ncurses.dev}/lib/pkgconfig:${pkgs.qt5.qtbase.dev}/lib/pkgconfig" | |||
export QT_QPA_PLATFORM_PLUGIN_PATH="${pkgs.qt5.qtbase.bin}/lib/qt-${pkgs.qt5.qtbase.version}/plugins" | |||
export QT_QPA_PLATFORMTHEME=qt5ct | |||
exec bash | |||
''; | |||
}).env}} | |||
Clone the kernel sources, enter the environment using {{ic|nix-shell}}, and then do development normally. | |||
== make menuconfig == | == make menuconfig == | ||
Line 176: | Line 255: | ||
src = fetchurl { | src = fetchurl { | ||
url = "https://github.com/jsakkine-intel/linux-sgx/archive/v23.tar.gz"; | url = "https://github.com/jsakkine-intel/linux-sgx/archive/v23.tar.gz"; | ||
sha256 = " | # After the first build attempt, look for "hash mismatch" and then 2 lines below at the "got:" line. | ||
# Use "sha256-....." value here. | |||
hash = ""; | |||
}; | }; | ||
kernelPatches = []; | kernelPatches = []; | ||
Line 195: | Line 276: | ||
=== Packaging out-of-tree kernel modules === | === Packaging out-of-tree kernel modules === | ||
There are a couple of steps that you will most likely need to do a couple of things. Here is an annotated example: | There are a couple of steps that you will most likely need to do a couple of things. Here is an annotated example: | ||
< | <syntaxhighlight lang="nix"> | ||
{ stdenv, lib, fetchFromGitHub, kernel, kmod }: | { stdenv, lib, fetchFromGitHub, kernel, kmod }: | ||
stdenv.mkDerivation rec { | stdenv.mkDerivation rec { | ||
pname = "v4l2loopback-dc"; | |||
version = "1.6"; | version = "1.6"; | ||
Line 206: | Line 287: | ||
repo = "droidcam"; | repo = "droidcam"; | ||
rev = "v${version}"; | rev = "v${version}"; | ||
hash = "1d9qpnmqa3pfwsrpjnxdz76ipk4w37bbxyrazchh4vslnfc886fx"; | |||
}; | }; | ||
Line 219: | Line 300: | ||
]; | ]; | ||
meta = | meta = { | ||
description = "A kernel module to create V4L2 loopback devices"; | description = "A kernel module to create V4L2 loopback devices"; | ||
homepage = "https://github.com/aramg/droidcam"; | homepage = "https://github.com/aramg/droidcam"; | ||
license = licenses.gpl2; | license = lib.licenses.gpl2; | ||
maintainers = [ maintainers.makefu ]; | maintainers = [ lib.maintainers.makefu ]; | ||
platforms = platforms.linux; | platforms = lib.platforms.linux; | ||
}; | }; | ||
} | } | ||
</ | </syntaxhighlight> | ||
1. For kernel modules it is necessary to disable <code>pic</code> in compiler hardenings as the kernel need different compiler flags. | 1. For kernel modules it is necessary to disable <code>pic</code> in compiler hardenings as the kernel need different compiler flags. | ||
Line 238: | Line 319: | ||
5. Lastly it is required to give the kernel build system the right location where to install the kernel module. This is done by setting <code>INSTALL_MOD_PATH</code> to <code>$out</code> Otherwise an error like <code>mkdir: cannot create directory '/lib': Permission denied</code> is generated. | 5. Lastly it is required to give the kernel build system the right location where to install the kernel module. This is done by setting <code>INSTALL_MOD_PATH</code> to <code>$out</code> Otherwise an error like <code>mkdir: cannot create directory '/lib': Permission denied</code> is generated. | ||
You can then call your program using something like <code>let yourprogram = config.boot.kernelPackages.callPackage ./your-derivation.nix {}; in …</code> (or if you want to compile it for the default kernel used by nix you can use <code>pkgs.linuxPackages.callPackage</code> but be aware that if you install it on a system running another kernel it will not work) and install the module in the kernel using <code>boot.extraModulePackages = [ yourprogram ];</code>. | |||
=== Developing out-of-tree kernel modules === | === Developing out-of-tree kernel modules === | ||
Line 271: | Line 354: | ||
</syntaxHighlight> | </syntaxHighlight> | ||
If wishing to develop out-of-tree kernel modules for kernels aside from the default variant shipped under <code>pkgs.linuxPackages</code>, | If wishing to develop out-of-tree kernel modules for kernels aside from the default variant shipped under <code>pkgs.linuxPackages</code>, you can replace <code>linux.dev</code> with (for instance) <code>linuxPackages_latest.kernel.dev</code>. | ||
If you want a local development environment and are only interested in Module.symvers, then you can do instead: | |||
<syntaxHighlight lang="console> | <syntaxHighlight lang="console> | ||
$ | $ cp $(nix-build -E '(import <nixpkgs> {}).linux.dev' --no-out-link)/lib/modules/*/build/Module.symvers . | ||
</syntaxHighlight> | </syntaxHighlight> | ||
Line 310: | Line 389: | ||
and the module will be automatically loaded after a reboot. If you want instead to load it at stage 1 (before the root is even mounted), you need to add it to <code>boot.initrd.availableKernelModules</code> and <code>boot.initrd.kernelModules</code>. | and the module will be automatically loaded after a reboot. If you want instead to load it at stage 1 (before the root is even mounted), you need to add it to <code>boot.initrd.availableKernelModules</code> and <code>boot.initrd.kernelModules</code>. | ||
Note that if you don't reboot, you can still load manually the module using <code>modprobe yourmodulename></code>, and to automatically enable a module during configuration switch/reboot, you can put <code>modprobe yourmodulename || true</code> inside the script of a systemctl service (it is for example what does wireguard). | Note that if you don't reboot, you can still load manually the module using <code>modprobe <yourmodulename></code>, and to automatically enable a module during configuration switch/reboot, you can put <code>modprobe yourmodulename || true</code> inside the script of a systemctl service (it is for example what does wireguard). | ||
Finally, if you want to define some options by default (used when you load manually a module using <code>modprobe</code>, or when the system boots), you can specify them in: | Finally, if you want to define some options by default (used when you load manually a module using <code>modprobe</code>, or when the system boots), you can specify them in: | ||
Line 318: | Line 397: | ||
options yourmodulename optionA=valueA optionB=valueB | options yourmodulename optionA=valueA optionB=valueB | ||
''; | ''; | ||
</syntaxHighlight> | |||
If you have developed a Nix package for your module, and want to only add the module to your <code>configuration.nix</code> instead of complete rebuilding the system based on your local <code>nixpkgs</code>, you need to update <code>boot.kernelPackages</code> as well, so kernel and modules can match each other: | |||
<syntaxHighlight lang=nix> | |||
{ | |||
boot = { | |||
kernelPackages = pkgs.wip.linuxPackages; | |||
extraModulePackages = with config.boot.kernelPackages; [ yourmodulename ]; | |||
} | |||
nixpkgs.config = { | |||
# Allow unfree modules | |||
allowUnfree = true; | |||
packageOverrides = pkgs: { | |||
wip = import (fetchGit { url = "/home/user/nixpkgs"; shallow = true;}) { | |||
config = config.nixpkgs.config; | |||
}; | |||
}; | |||
}; | |||
}; | |||
</syntaxHighlight> | </syntaxHighlight> | ||
Line 324: | Line 423: | ||
Save the following as <code>shell.nix</code> for your nix-shell: | Save the following as <code>shell.nix</code> for your nix-shell: | ||
< | <syntaxhighlight lang="nix"> | ||
{ pkgs ? import <nixpkgs> {} }: | { pkgs ? import <nixpkgs> {} }: | ||
let | let | ||
Line 333: | Line 432: | ||
# the override is optional if you need for example more build dependencies | # the override is optional if you need for example more build dependencies | ||
nativeBuildInputs = old.nativeBuildInputs ++ [ pkgs.gllvm ]; | nativeBuildInputs = old.nativeBuildInputs ++ [ pkgs.gllvm ]; | ||
} | }) | ||
</ | </syntaxhighlight> | ||
Then you can run the following commands (in the nix-shell <code>$makeFlags</code> will contain | |||
the necessary cross-compiling arguments) | the necessary cross-compiling arguments) | ||
Line 358: | Line 457: | ||
</syntaxHighlight> | </syntaxHighlight> | ||
Then you can use: | |||
<syntaxHighlight lang=console> | <syntaxHighlight lang=console> | ||
Line 396: | Line 495: | ||
</syntaxHighlight> | </syntaxHighlight> | ||
== Patching a single In-tree kernel module == | |||
If you wish to patch a single kernel module, you can avoid rebuilding the entire linux kernel | |||
by packaging that module applying patches to it. | |||
=== Packaging a single kernel module === | |||
Here is an example for how to package the <code>amdgpu</code> kernel module. | |||
<syntaxHighlight lang=nix> | |||
{ pkgs | |||
, lib | |||
, kernel ? pkgs.linuxPackages_latest.kernel | |||
}: | |||
pkgs.stdenv.mkDerivation { | |||
pname = "amdgpu-kernel-module"; | |||
inherit (kernel) src version postPatch nativeBuildInputs; | |||
kernel_dev = kernel.dev; | |||
kernelVersion = kernel.modDirVersion; | |||
modulePath = "drivers/gpu/drm/amd/amdgpu"; | |||
buildPhase = '' | |||
BUILT_KERNEL=$kernel_dev/lib/modules/$kernelVersion/build | |||
cp $BUILT_KERNEL/Module.symvers . | |||
cp $BUILT_KERNEL/.config . | |||
cp $kernel_dev/vmlinux . | |||
make "-j$NIX_BUILD_CORES" modules_prepare | |||
make "-j$NIX_BUILD_CORES" M=$modulePath modules | |||
''; | |||
installPhase = '' | |||
make \ | |||
INSTALL_MOD_PATH="$out" \ | |||
XZ="xz -T$NIX_BUILD_CORES" \ | |||
M="$modulePath" \ | |||
modules_install | |||
''; | |||
meta = { | |||
description = "AMD GPU kernel module"; | |||
license = lib.licenses.gpl3; | |||
}; | |||
} | |||
</syntaxHighlight> | |||
=== Replacing default kernel modules === | |||
After packaging the kernel module you can add it to your system just like an out-of-tree kernel module, it will replace the default module provided by the linux package because it has the same name: | |||
<syntaxhighlight lang="nix"> | |||
{pkgs, config, ...}: | |||
let | |||
amdgpu-kernel-module = pkgs.callPackage ./amdgpu-kernel-module.nix { | |||
# Make sure the module targets the same kernel as your system is using. | |||
kernel = config.boot.kernelPackages.kernel; | |||
}; | |||
in | |||
{ | |||
boot.extraModulePackages = [ | |||
(amdgpu-kernel-module.overrideAttrs (_: { | |||
patches = [ ./patches/amdgpu-foo-bar.patch ]; | |||
})) | |||
]; | |||
} | |||
</syntaxhighlight> | |||
== Troubleshooting == | == Troubleshooting == |