Ollama: Difference between revisions
Malteneuss (talk | contribs) m Add rocm nix flake example |
This has been changed so that it actually overrides the proper env vars. The last format shouldn't work as the convention is to use 10.3.0 for other rocm devices. |
||
| (One intermediate revision by one other user not shown) | |||
| Line 104: | Line 104: | ||
# classical | # classical | ||
$ nix-shell -p "rocmPackages.rocminfo" --run "rocminfo" | grep "gfx" | $ nix-shell -p "rocmPackages.rocminfo" --run "rocminfo" | grep "gfx" | ||
Name: gfx1031 | |||
# flakes | # flakes | ||
$ nix run nixpkgs#"rocmPackages.rocminfo" -- --run "rocminfo" | grep "gfx" | $ nix run nixpkgs#"rocmPackages.rocminfo" -- --run "rocminfo" | grep "gfx" | ||
| Line 117: | Line 119: | ||
HCC_AMDGPU_TARGET = "gfx1031"; # used to be necessary, but doesn't seem to anymore | HCC_AMDGPU_TARGET = "gfx1031"; # used to be necessary, but doesn't seem to anymore | ||
}; | }; | ||
# results in environment variable "HSA_OVERRIDE_GFX_VERSION=10.3. | # results in environment variable "HSA_OVERRIDE_GFX_VERSION=10.3.0" | ||
rocmOverrideGfx = "10.3. | rocmOverrideGfx = "10.3.0"; | ||
}; | }; | ||
</syntaxhighlight> | </syntaxhighlight> | ||
| Line 124: | Line 126: | ||
or via an environment variable in front of the standalone app | or via an environment variable in front of the standalone app | ||
<syntaxhighlight lang="bash"> | <syntaxhighlight lang="bash"> | ||
HSA_OVERRIDE_GFX_VERSION=10.3. | HSA_OVERRIDE_GFX_VERSION=10.3.0 ollama serve | ||
</syntaxhighlight> | </syntaxhighlight> | ||