Ollama - Revision history

Librepup: Re-Routed CUDA Link to In-Wiki Page

2026-07-18T22:40:42Z

Re-Routed CUDA Link to In-Wiki Page

← Older revision		Revision as of 22:40, 18 July 2026
Line 25:		Line 25:
	* ollama-vulkan: supported by most modern GPUs on Linux		* ollama-vulkan: supported by most modern GPUs on Linux

	'''Note:''' When using the ollama-cuda package, you may want to add the [https://cache.nixos-cuda.org/ CUDA Substitute Binary Cache], since compiling CUDA from source may take a long time. (See the [~~https://nixos.wiki/wiki/CUDA~~ CUDA] Wiki Entry for more Information)		'''Note:''' When using the ollama-cuda package, you may want to add the [https://cache.nixos-cuda.org/ CUDA Substitute Binary Cache], since compiling CUDA from source may take a long time. (See the [[CUDA]] Wiki Entry for more Information)

Librepup: Re-phrased CUDA notice and added external Link to CUDA Cache Site

2026-07-18T22:21:09Z

Re-phrased CUDA notice and added external Link to CUDA Cache Site

← Older revision		Revision as of 22:21, 18 July 2026
Line 25:		Line 25:
	* ollama-vulkan: supported by most modern GPUs on Linux		* ollama-vulkan: supported by most modern GPUs on Linux

	'''Note:''' When using the ollama-cuda package, you may want to add the CUDA Substitute Binary Cache, since ~~most household Computers will not be able to compile~~ CUDA from source. (See the [https://nixos.wiki/wiki/CUDA CUDA] Wiki Entry for more Information)		'''Note:''' When using the ollama-cuda package, you may want to add the [https://cache.nixos-cuda.org/ CUDA Substitute Binary Cache], since compiling CUDA from source may take a long time. (See the [https://nixos.wiki/wiki/CUDA CUDA] Wiki Entry for more Information)

Librepup: Added a useful Hint/Warning Text after the list of available ollama packages, informing users that it may be advised to add the CUDA Substitute, so as to not compile CUDA from source.

2026-07-18T22:15:30Z

Added a useful Hint/Warning Text after the list of available ollama packages, informing users that it may be advised to add the CUDA Substitute, so as to not compile CUDA from source.

← Older revision		Revision as of 22:15, 18 July 2026
Line 24:		Line 24:
	* ollama-cuda: supported by most modern NVIDIA GPUs		* ollama-cuda: supported by most modern NVIDIA GPUs
	* ollama-vulkan: supported by most modern GPUs on Linux		* ollama-vulkan: supported by most modern GPUs on Linux

			'''Note:''' When using the ollama-cuda package, you may want to add the CUDA Substitute Binary Cache, since most household Computers will not be able to compile CUDA from source. (See the [https://nixos.wiki/wiki/CUDA CUDA] Wiki Entry for more Information)

Mathtician: Updated syntax for systemd service config (acceleration -> package); have not tested standalone package config

2026-01-26T19:49:04Z

Updated syntax for systemd service config (acceleration -> package); have not tested standalone package config

← Older revision		Revision as of 19:49, 26 January 2026
Line 19:		Line 19:

	== Configuration of GPU acceleration ==		== Configuration of GPU acceleration ==
	~~Its possible to use following values for acceleration~~:		Acceleration is configured by selecting a package:
	* ~~false~~: disable GPU, only use CPU		* ollama-cpu: disable GPU, only use CPU
	* "rocm": supported by most modern AMD GPUs		* ollama-rocm: supported by most modern AMD GPUs
	* "cuda": supported by most modern NVIDIA GPUs		* ollama-cuda: supported by most modern NVIDIA GPUs
	* "vulkan": supported by most modern GPUs on Linux		* ollama-vulkan: supported by most modern GPUs on Linux


Line 41:		Line 41:
	services.ollama = {		services.ollama = {
	enable = true;		enable = true;
	~~acceleration~~ = "cuda";		package = pkgs.ollama-cuda;
	};		};
	</syntaxhighlight>		</syntaxhighlight>
Line 128:		Line 128:
	services.ollama = {		services.ollama = {
	enable = true;		enable = true;
	~~acceleration~~ = "rocm";		package = pkgs.ollama-rocm;
	environmentVariables = {		environmentVariables = {
	HCC_AMDGPU_TARGET = "gfx1031"; # used to be necessary, but doesn't seem to anymore		HCC_AMDGPU_TARGET = "gfx1031"; # used to be necessary, but doesn't seem to anymore

Rebuilt: /* Configuration of GPU acceleration */ Document vulkan option

2025-12-01T04:34:35Z

Configuration of GPU acceleration: Document vulkan option

← Older revision		Revision as of 04:34, 1 December 2025
Line 23:		Line 23:
	* "rocm": supported by most modern AMD GPUs		* "rocm": supported by most modern AMD GPUs
	* "cuda": supported by most modern NVIDIA GPUs		* "cuda": supported by most modern NVIDIA GPUs
			* "vulkan": supported by most modern GPUs on Linux

Kini: fix ref link

2025-11-20T16:25:23Z

fix ref link

← Older revision		Revision as of 16:25, 20 November 2025
Line 109:		Line 109:
	In certain cases Ollama might not allow your system to use GPU acceleration if it cannot be sure your GPU/driver is compatible.		In certain cases Ollama might not allow your system to use GPU acceleration if it cannot be sure your GPU/driver is compatible.

	However you can attempt to force-enable the usage of your GPU by overriding the LLVM target. <ref>https://github.com/ollama/ollama/blob/main/docs/gpu.md#overrides</ref>		However you can attempt to force-enable the usage of your GPU by overriding the LLVM target. <ref>https://github.com/ollama/ollama/blob/main/docs/gpu.mdx#overrides-on-linux</ref>

	You can get the version for your GPU from the logs or like so:		You can get the version for your GPU from the logs or like so:

Chafey at 18:41, 6 November 2025

2025-11-06T18:41:43Z

@@ Line 94: / Line 94: @@
 == Troubleshooting ==
 === AMD GPU with open source driver ===
 In certain cases Ollama might not allow your system to use GPU acceleration if it cannot be sure your GPU/driver is compatible.

Yuannan: This has been changed so that it actually overrides the proper env vars. The last format shouldn't work as the convention is to use 10.3.0 for other rocm devices.

2025-04-20T19:05:29Z

This has been changed so that it actually overrides the proper env vars. The last format shouldn't work as the convention is to use 10.3.0 for other rocm devices.

← Older revision		Revision as of 19:05, 20 April 2025
Line 119:		Line 119:
	HCC_AMDGPU_TARGET = "gfx1031"; # used to be necessary, but doesn't seem to anymore		HCC_AMDGPU_TARGET = "gfx1031"; # used to be necessary, but doesn't seem to anymore
	};		};
	# results in environment variable "HSA_OVERRIDE_GFX_VERSION=10.3.1"		# results in environment variable "HSA_OVERRIDE_GFX_VERSION=10.3.0"
	rocmOverrideGfx = "10.3.1";		rocmOverrideGfx = "10.3.0";
	};		};
	</syntaxhighlight>		</syntaxhighlight>
Line 126:		Line 126:
	or via an environment variable in front of the standalone app		or via an environment variable in front of the standalone app
	<syntaxhighlight lang="bash">		<syntaxhighlight lang="bash">
	HSA_OVERRIDE_GFX_VERSION=10.3.1 ollama serve		HSA_OVERRIDE_GFX_VERSION=10.3.0 ollama serve
	</syntaxhighlight>		</syntaxhighlight>

Malteneuss: Fix example

2025-04-04T10:37:33Z

Fix example

← Older revision		Revision as of 10:37, 4 April 2025
Line 104:		Line 104:
	# classical		# classical
	$ nix-shell -p "rocmPackages.rocminfo" --run "rocminfo" \| grep "gfx"		$ nix-shell -p "rocmPackages.rocminfo" --run "rocminfo" \| grep "gfx"
			Name: gfx1031

	# flakes		# flakes
	$ nix run nixpkgs#"rocmPackages.rocminfo" -- --run "rocminfo" \| grep "gfx"		$ nix run nixpkgs#"rocmPackages.rocminfo" -- --run "rocminfo" \| grep "gfx"

Malteneuss: Add rocm nix flake example

2025-04-04T10:36:47Z

Add rocm nix flake example

← Older revision		Revision as of 10:36, 4 April 2025
Line 102:		Line 102:

	<syntaxhighlight lang="bash">		<syntaxhighlight lang="bash">
			# classical
	$ nix-shell -p "rocmPackages.rocminfo" --run "rocminfo" \| grep "gfx"		$ nix-shell -p "rocmPackages.rocminfo" --run "rocminfo" \| grep "gfx"
			# flakes
			$ nix run nixpkgs#"rocmPackages.rocminfo" -- --run "rocminfo" \| grep "gfx"
	Name: gfx1031		Name: gfx1031
	</syntaxhighlight>		</syntaxhighlight>