Intel NPU

The Intel NPU (Intel AI Boost) is the neural processing unit in Intel Core Ultra processors. Applications use it through OpenVINO.

Note: Requires NixOS 26.11 or newer, or a nixos-unstable revision that includes NixOS/nixpkgs#570230. Older releases cannot compile models for the NPU.

Setup

hardware.cpu.intel.npu.enable = true;

Verification

Check each layer in order; the first one that fails is where the problem is.

The NPU is a PCI device of class 1200, bound to the intel_vpu kernel driver:

$ lspci -knnd ::1200
00:0b.0 Processing accelerators [1200]: Intel Corporation Core Ultra 200 Series Processors NPU [8086:ad1d] (rev 01)
	...
	Kernel driver in use: intel_vpu
	Kernel modules: intel_vpu
$ journalctl -kg intel_vpu
...
[drm] Initialized intel_vpu 1.0.0 for 0000:00:0b.0 on minor 0
$ ls -l /dev/accel/
crw-rw-rw- 1 root render 261, 0 ... accel0

The userspace driver is installed where Level Zero looks for it:

$ ls /run/opengl-driver/lib | grep ze_intel_npu
libze_intel_npu.so
libze_intel_npu.so.1
libze_intel_npu.so.1.38.0

OpenVINO sees the NPU:

$ nix-shell -p 'python3.withPackages (ps: [ ps.openvino ])' --run \
    "python -c 'import openvino as ov; c = ov.Core(); print(c.available_devices, c.get_property(\"NPU\", \"FULL_DEVICE_NAME\"))'"
['CPU', 'GPU', 'NPU'] Intel(R) AI Boost

A model compiles and runs on the NPU:

import numpy as np
import openvino as ov
import openvino.opset13 as ops

x = ops.parameter([1, 16], np.float32, name="x")
model = ov.Model([ops.relu(ops.add(x, ops.constant(np.ones((1, 16), np.float32))))], [x])

compiled = ov.Core().compile_model(model, "NPU")
print(compiled(np.arange(-8, 8, dtype=np.float32).reshape(1, 16))[0])
# [[0. 0. 0. 0. 0. 0. 0. 0. 1. 2. 3. 4. 5. 6. 7. 8.]]

If the driver library is missing, hardware.cpu.intel.npu.enable is not active yet; switch to the new configuration. If OpenVINO lists no NPU although the earlier checks pass, the process may not see /dev/accel/accel0; see #Systemd services.

Usage

The usual way to run models on the NPU is OpenVINO GenAI (python3Packages.openvino-genai). It provides ready-made pipelines for common tasks: LLMPipeline for text generation, VLMPipeline for vision-language models, WhisperPipeline for speech recognition, and others. Each pipeline takes a model directory and a device name; pass "NPU".

The workflow is:

Get a model in OpenVINO format. The OpenVINO organization on Hugging Face publishes converted models, including VLMs optimized for NPU and LLMs optimized for NPU.
Load it with the matching pipeline on "NPU", and set CACHE_DIR. Compiling a large model for the NPU can take over a minute; with the cache, later runs load the compiled model in seconds.

To convert other Hugging Face models, use Optimum Intel's optimum-cli export openvino. It is not packaged in nixpkgs, and LLMs need specific quantization flags for the NPU; see OpenVINO GenAI on NPU.

Example: speech recognition

Download a Whisper model:

$ nix-shell -p python3Packages.huggingface-hub --run \
    'hf download OpenVINO/whisper-large-v3-turbo-int8-ov --local-dir whisper-large-v3-turbo-int8-ov'

Transcribe a WAV file in 16 kHz, mono, 16-bit format:

import wave

import numpy as np
import openvino_genai

with wave.open("speech.wav") as f:
    samples = np.frombuffer(f.readframes(f.getnframes()), np.int16).astype(np.float32) / 32768

pipe = openvino_genai.WhisperPipeline("whisper-large-v3-turbo-int8-ov", "NPU", CACHE_DIR="npu-cache")
print(pipe.generate(samples).texts[0])

The first run compiles the model for the NPU, which takes about 70 seconds on an Arrow Lake NPU; later runs start in seconds.

Using the OpenVINO runtime directly

For models outside GenAI's pipelines, use the OpenVINO runtime (python3Packages.openvino) and pass "NPU" to compile_model, as in #Verification. The NPU needs static input shapes, so reshape a model to fixed input sizes before compiling it. See OpenVINO: NPU Device.

How it works

The kernel's intel_vpu driver exposes the NPU as /dev/accel/accel0. The option above adds the NPU firmware and puts the userspace driver intel-npu-driver in /run/opengl-driver/lib, where OpenVINO finds it through Level Zero. The driver then loads the NPU compiler from the openvino-intel-npu-compiler package.

Systemd services

A sandboxed service needs access to the accel device, and OpenVINO generates code at runtime:

systemd.services.<name>.serviceConfig = {
  PrivateDevices = false;
  DeviceAllow = [ "char-accel rw" ];
  MemoryDenyWriteExecute = false;
};