The Ollama API can report how long it spent loading a model, processing your prompt and generating the reply. To measure generation speed, divide the generated token count by the generation duration converted to seconds. Record that number alongside the total request duration. A model can generate quickly once it starts and still feel slow when it has to load or process a long prompt. Measuring each stage helps you explain that difference and identify which change would matter to your workload.
Start with a controlled request
Run Ollama locally and choose a model tag from ollama list. Use the same tag, prompt and generation options for each comparison. If you are comparing two models, record both exact tags and keep the task the same. Save the output or at least note whether the model followed the prompt. An unusually short, irrelevant response can look quick while failing to finish the work. Read the timing field definitions in the Ollama API reference before combining the numbers.
For an initial run, use the native /api/generate endpoint with stream set to false. This gives you one JSON response with the generated text and final statistics. For example, in a shell with curl available, set the JSON body to {"model":"YOUR_MODEL_TAG","prompt":"Explain why bicycle tires need air in four short paragraphs.","stream":false} and submit it as a POST to http://localhost:11434/api/generate with a content type of application/json. Replace the tag with a model you have installed. For a chat workflow, our Python streaming guide explains how to read a native stream through its final record.
Convert durations and calculate throughput
The duration values returned by the native API are in nanoseconds. Divide a duration by one billion to convert it to seconds. The eval_count field counts generated tokens, and eval_duration measures the generation phase. If the response reports 180 generated tokens and an evaluation duration of 9,000,000,000 nanoseconds, that is 180 divided by 9, or 20 tokens per second. Use the fields together and skip the rate calculation when the duration is zero or missing.
The prompt phase has its own fields. Use prompt_eval_count and prompt_eval_duration to compute a separate rate when both are available and the duration is positive. A rate for reading the prompt answers a different question from the rate for producing the reply. Prompt caching and reused conversation prefixes can affect what happens across repeated runs. If a cold comparison matters, record when a request reused context or when you deliberately changed its input to check a new prompt.
Keep total_duration and load_duration alongside those rates. Total duration is a server-reported measurement and can differ from elapsed time measured around the request in your application. The application may also spend time sending the request, reading the response or updating its display. If you need to explain what a reader perceives, measure wall-clock time separately with a monotonic clock. The timing breakdown does not have to add up to the exact figure the interface reports.
Compare a cold model with a warm model
Check currently loaded models with ollama ps. Run the planned request when the model is unloaded, then run it again while the model remains available. Compare load duration, total duration and generation rate. If you want to explicitly unload a locally running model between tests, use Ollama’s supported model stop or keep-alive controls and confirm the state before the next run. Do this on a machine where the model is yours to test, because unloading affects requests that share the same process.
Run several trials in each condition. Record the median and any unusually slow result with context. A cold model in Ollama can still benefit from files cached by the operating system. Describe the state you actually controlled, such as whether the model was loaded before the request. If a result needs to reflect a machine freshly started for the day, make that a separate experiment and compare it with the ordinary warm workload.
Here is an illustrative comparison. One cold request takes twelve seconds, including five seconds of model load and six seconds of generation. A warm request takes seven seconds and still spends six seconds generating. Keeping the model available makes the next reply arrive sooner in this example. It does little for generation throughput itself. If a different model generates the same quality of answer in three seconds, that change addresses a different part of the wait.
Record conditions that explain differences
Save the model tag, hardware, Ollama version, context settings, token counts, and whether the machine was busy. A comparison between a short prompt and a long pasted document is difficult to interpret unless you split prompt processing from output generation. If a local service is under load, test once with other work quiet and separately under the concurrency you expect. A single isolated run tells you how the system behaves for one request, while the concurrent run helps explain what users see when requests overlap.
If speed changes unexpectedly, verify how the model is running before buying hardware or changing several settings. Our guide to checking Ollama GPU use explains how to inspect the loaded model. Then compare a representative request under the original configuration and one deliberate change. Consider recording the quality of the answer alongside time. A faster model that omits details your workflow needs can create more work downstream.
For streaming comparisons, measure how long it takes until the first visible text as well as the time to completion. The first network event may contain metadata or an empty text field. If you want to test how the interface feels, use the time when it actually displays a meaningful part of the reply. A report of tokens per second is still useful, but it answers what happens during generation and can miss the wait before the screen changes.
For a useful comparison spreadsheet, put each request on one row. Keep the original nanosecond values in their own columns and compute seconds and rates separately. Include prompt and output token counts so you can spot runs that did much less work than the rest. Group results by model tag and load state before calculating medians. Give unusually short replies a label explaining why they stopped, such as reaching an output limit or finishing naturally. Keep unsuccessful requests in the notes so reliability remains visible even when the speed summary uses successful runs only.
Frequently asked questions
Can I compare tokens per second across different models?
You can compare the reported rates, but models can tokenize the same text differently and produce different answers. Keep the task fixed and also compare elapsed time and whether the output is useful.
Why is the first answer slower than the next one?
Check the model load duration and prompt statistics for both responses. A model that is already loaded may start faster, and a repeated prompt can behave differently if the runtime reuses context. Record the conditions alongside the result.
Is total duration the same as generation time?
Total duration includes more of the server’s handling than the generation phase alone. Use the evaluation duration for generation throughput, and measure the application’s elapsed time if you care about the full user-visible wait.
What should I do if an API timing field is missing?
Check the endpoint and whether you captured a completed response. Do not turn a missing or zero duration into a speed estimate. Record that observation and rerun the request under known conditions.
