Run a local model, send a prompt, and check ollama ps while it is loaded. Read the Processor column to see whether Ollama placed the model on the GPU, CPU, or a mix of both. Compare the result with system monitoring and your actual prompt speed before deciding whether anything is wrong.
Check the right process while the model is loaded
Open a terminal on the computer that runs Ollama. If your app connects to another machine, monitoring the client computer will not tell you where the model is running. Choose a local model you already have and send a short prompt. Then run ollama ps in a second terminal before the model is unloaded. The Ollama FAQ explains the Processor column and how to interpret its CPU and GPU labels. Record the model tag and the output you see.
An empty list can mean the model is no longer loaded or that the command is looking at a different Ollama instance. Repeat with a fresh prompt or check the process while the request is active. If you use a client that targets both local and remote models, confirm which server and model handled the request. Our local Ollama Chat API guide shows how to send a known request to the local endpoint. That gives you a repeatable check independent of a third-party interface.
Interpret CPU, GPU and split placement
A GPU result indicates that Ollama reports the model as loaded in GPU memory. A CPU result indicates host placement. A split result indicates the model is distributed across CPU and GPU memory. These values describe placement reported by Ollama. Treat them separately from a live utilization percentage in your operating system. A GPU may show a short burst of work and then sit idle while the model remains loaded. An idle reading after the response is complete is normal.
Compare platform-appropriate counters while generation is happening. On a computer with dedicated graphics memory, note the change in GPU memory use and select the relevant compute view if the monitor distinguishes it from graphics or video. On hardware with unified memory, system and graphics use may not appear as neatly separated pools. Avoid comparing raw readings across different platforms without understanding what each tool counts. Our RAM and VRAM guide covers why these distinctions matter for choosing a model.
Compare a small model and your usual workload
Begin with a smaller compatible model and a short prompt. If it fits on the GPU, repeat with the model you intend to use. Keep the first prompt similar and record the Processor column for each run. Then repeat with your actual context size and typical other apps running. More context can require additional memory, so a short demonstration can behave differently from a full document workflow. A configuration that only works with everything else closed may be inconvenient for normal use.
Record what is loaded at the same time. Other applications and models can consume available memory. If you have finished using a model, check the CLI help for the supported stop command and unload that model before repeating an isolated test. If an application intentionally leaves a model loaded, check its keep-alive behavior before treating that as a fault. Our Ollama quantization guide explains how to compare a smaller variant without changing the underlying model family.
Investigate unexpected CPU placement
Check the official Ollama compatibility information for your operating system, runtime version and exact GPU model. Review the vendor’s supported driver and whether your installation has access to the GPU. A host tool seeing the card does not prove that a container or virtual machine can use it. If the model runs inside a container, review the documented device access for that environment. Use supported settings for the deployment rather than randomly copying variables from a forum post.
If the small compatible model also reports CPU placement, review Ollama’s startup or server logs using the method documented for your platform. Look for evidence that the runtime detected the adapter or reported a compatibility problem. Keep your log summary narrow if you share it, and remove identifying information that the recipient does not need. Recheck after addressing a specific driver or runtime issue. Changing three settings at once makes a successful outcome difficult to reproduce.
Measure whether the result improves your workflow
Use the same model, prompt and context settings when comparing two configurations. Separate the first request, when weights may need to load, from a later request with the model already in memory. Measure the time until useful output starts and the time to complete. Write down whether the answer itself still meets your task requirements. For a small prototype those two timings can be easier to interpret than a single GPU utilization snapshot.
CPU or split placement can still be workable for an occasional short prompt. Conversely, GPU placement does not guarantee that a long workflow feels fast. Document parsing, retrieval and prompt processing can all add time before generation. Check the step that actually delays the user. Our local model benchmarking guide gives a way to hold inputs constant while evaluating speed and quality.
Keep a baseline for future upgrades
Record the runtime version, device, model tag and context length when you find a working configuration. Include a typical short prompt and a longer task. Repeat those checks after an update if behavior changes. If placement looks the same but response time rises, check other running work and compare cold and warm requests again. Saving the baseline makes it possible to tell a model change from a machine change.
If you have two GPUs, look at which one is doing work instead of assuming the larger adapter was selected. Compare the process listing with the operating system’s adapter-specific counters. Consult Ollama’s documented device controls if your setup requires a specific GPU. Test with the apps you normally run, since a display, game or other compute application can change the memory available to the model.
When reporting a regression, include whether it happens for every model or only one tag. A small model that still uses the GPU gives you evidence that detection works. A specific larger model falling back under long prompts suggests that memory demand deserves a closer look.
Before changing hardware, decide what improvement would justify it. For a chat interface, you might care most about the pause before the first words appear. For a batch job, completion time across several documents may matter more. Test the measure you actually need and save the result alongside the processor reading.
Frequently asked questions
Does low GPU utilization mean Ollama is on the CPU?
Check placement with ollama ps while the model is loaded, then monitor during generation. A low reading between requests is expected.
Can the model use CPU and GPU together?
Ollama can report split placement. Compare speed and memory use with a smaller variant if the split is causing a problem.
Why does ollama ps show nothing?
Send a request and check again while the model is loaded. Also confirm that the command and client are pointed at the same server.
