Talk:Llama-cpp: Difference between revisions
Latest comment: 15 July by Hans4687 in topic Command llama-cli gives error
→Review the AMD ROCm section: new section |
→Command llama-cli gives error: new section |
||
| Line 6: | Line 6: | ||
Thanks [[User:Woile|Woile]] ([[User talk:Woile|talk]]) 12:36, 19 May 2026 (UTC) | Thanks [[User:Woile|Woile]] ([[User talk:Woile|talk]]) 12:36, 19 May 2026 (UTC) | ||
== Command llama-cli gives error == | |||
I tried this command on nix-shell with llama-cpp version: 6981 (647b960). | |||
Error: | |||
<syntaxhighlight lang="mediawiki"> | |||
llama_model_load: error loading model: error loading model architecture: unknown model architecture: 'qwen3next' llama_model_load_from_file_impl: failed to load model | |||
</syntaxhighlight> | |||
The model is not supported (not yet) bij llama-cpp | |||
If I try another model e.g. | |||
<syntaxhighlight lang="mediawiki"> | |||
llama-cli -hf bartowski/Qwen2.5-Coder-3B-Instruct-GGUF:Q6_K_L --temp 1.0 --top-p 0.95 --top-k 40 -p "briefly explain journalctl in one paragraph" | |||
... | |||
== Running in interactive mode. == | |||
- Press Ctrl+C to interject at any time. | |||
- Press Return to return control to the AI. | |||
- To return control without starting a new line, end your input with '/'. | |||
- If you want to submit another line, end your input with '\'. | |||
- Not using system message. To change it, set a different value via -sys PROMPT | |||
user | |||
briefly explain journalctl in one paragraph | |||
assistant | |||
Journalctl is a powerful command-line utility provided by the systemd system and service manager that allows you to view, search, and monitor the system's logs in real-time. It provides detailed information about vario | |||
us aspects of the system, such as system boot, services, and applications, and can be used to troubleshoot issues and understand system behavior. Journalctl supports various log levels, filtering options, and can be us | |||
ed to monitor specific services, users, or applications by using various filters and selectors. It is a versatile tool that can be used to gain insights into the system's activities and operations. | |||
</syntaxhighlight>[[User:Hans4687|Hans4687]] ([[User talk:Hans4687|talk]]) 14:36, 15 July 2026 (UTC) | |||
Revision as of 14:36, 15 July 2026
Review the AMD ROCm section
I wrote from memory, and I'm not entirely sure my knowledge is accurate. When changing the env variable, I've noticed performance changes in llama-cpp. But again, I'm not sure. Could someone verify what I wrote?
I also wonder if one should compile llama-cpp on their own machine with rocmGpuTargets ??
Thanks Woile (talk) 12:36, 19 May 2026 (UTC)
Command llama-cli gives error
I tried this command on nix-shell with llama-cpp version: 6981 (647b960).
Error:
llama_model_load: error loading model: error loading model architecture: unknown model architecture: 'qwen3next' llama_model_load_from_file_impl: failed to load model
The model is not supported (not yet) bij llama-cpp
If I try another model e.g.
llama-cli -hf bartowski/Qwen2.5-Coder-3B-Instruct-GGUF:Q6_K_L --temp 1.0 --top-p 0.95 --top-k 40 -p "briefly explain journalctl in one paragraph"
...
== Running in interactive mode. ==
- Press Ctrl+C to interject at any time.
- Press Return to return control to the AI.
- To return control without starting a new line, end your input with '/'.
- If you want to submit another line, end your input with '\'.
- Not using system message. To change it, set a different value via -sys PROMPT
user
briefly explain journalctl in one paragraph
assistant
Journalctl is a powerful command-line utility provided by the systemd system and service manager that allows you to view, search, and monitor the system's logs in real-time. It provides detailed information about vario
us aspects of the system, such as system boot, services, and applications, and can be used to troubleshoot issues and understand system behavior. Journalctl supports various log levels, filtering options, and can be us
ed to monitor specific services, users, or applications by using various filters and selectors. It is a versatile tool that can be used to gain insights into the system's activities and operations.