Llama-cpp: Difference between revisions
improve styles and added troubleshooting |
corrrections and style |
||
| Line 46: | Line 46: | ||
=== AMD ROCm === | === AMD ROCm === | ||
Sometimes, ROCm might not be using the correct GPU architecture, or you simply want to try a different one, because it might work better. To tell ROCm which GPU architecture to use, you can use the HSA_OVERRIDE_GFX_VERSION environmental variable. | Sometimes, ROCm might not be using the correct GPU architecture, or you simply want to try a different one, because it might work better. To tell ROCm which GPU architecture to use, you can use the <code>HSA_OVERRIDE_GFX_VERSION</code> environmental variable. | ||
E.g:<syntaxhighlight lang="bash"> | E.g:<syntaxhighlight lang="bash"> | ||
| Line 86: | Line 86: | ||
And more general purpose models, may include multiple options sets for different purposes, e.g: chat, coding, agent, etc. | And more general purpose models, may include multiple options sets for different purposes, e.g: chat, coding, agent, etc. | ||
=== | === How much RAM do I need? === | ||
To calculate how much RAM you are going to need, the first signal is the number of parameters. You can make a loose estimate that 80B params will require 80GB RAM. | To calculate how much RAM you are going to need, the first signal is the number of parameters. You can make a loose estimate that 80B params will require 80GB RAM. | ||