Ollama’s context length controls how much text the runtime makes available to a model for a request. Increase it when a real prompt needs more room for instructions, documents, and conversation history. Then check that the new setting actually took effect and that the machine can handle it at a usable speed.
Understand what uses the context window
A language model processes text in tokens, which can be whole words, pieces of words, punctuation, or other units. The exact number varies with the model and the input. A page of ordinary prose and a page of minified code can produce very different token counts. Avoid estimating a tight fit solely from a word count. Use an actual example of the text the model will see.
A context window has to accommodate more than the latest question. The runtime may include system instructions, earlier conversation turns, source text, and other content the application adds. It also needs room for the response as it generates. A chat window can show the entire transcript on screen even when the application sends only part of it to the model. Check your interface’s behavior if old messages stop affecting answers.
The advertised limit for a model and the context configured in Ollama answer different questions. The first describes a capability of that model variant. The second tells you what the runtime allocates in your current setup. Check both before trying to read a large document in a single request. If you need an installation and first-run walkthrough, start with our local AI guide.
Check the context Ollama actually uses
Open a terminal and start the local model you already use. Send a short test prompt to make sure it is loaded. In a separate terminal, run ollama ps and look for the context and processor columns. Record the exact model tag and the values you see. The official Ollama context length documentation shows the command and explains the runtime’s context setting.
Use the documented context control for the installation you have. If you use the desktop app, look for the context setting in its preferences. If you run Ollama yourself as a server, the docs describe setting OLLAMA_CONTEXT_LENGTH for that server process. Changing an environment variable in a new terminal will not reconfigure a different server process that is already running. Identify which process your client is talking to before you restart anything.
After changing the setting, start a fresh chat with the same model and check ollama ps again. Compare its allocation to your original notes. Some clients supply their own model options, so a change in one app may not carry over to another. If the observed value does not match what you expected, check the client settings and consult the documentation for your installed version.
Test with a document you can check
Build a short test document with three facts that are easy to verify. Put one near the start, one in the middle, and one near the end. Ask for those facts and tell the model to include the relevant section heading for each. Run a baseline, then repeat with a longer document that resembles your real workload. Use content you can share with your runtime and avoid inserting secrets into a new tool.
Compare the returned facts with your original. If the model misses details from one position, check how the input was delivered. A file upload feature may retrieve selected passages rather than insert the whole document. Your own pasted text may also be clipped by the application. Try the same question against a small excerpt containing the missing fact. If it still fails with the short excerpt, a larger window alone may not solve it.
Repeat the longer test in a fresh conversation after changing the context. Keep the same prompt and model tag. Record whether the answer is correct, whether the model cites the right section, and how long it takes to respond. A correct answer with an invented reference still needs review. If the workflow is for notes or research, keep the source document available so you can verify quotations and facts independently.
Choose a context length that fits the workload
A larger allocation needs more memory and can change where the runtime places the model. Check ollama ps during the larger test and compare system memory or GPU memory with the baseline. If part of the workload moves off the GPU, the extra room may come with a speed cost. Our RAM and VRAM guide explains how to measure that on your machine.
If the result is slow, try breaking the document into coherent sections and ask the model to extract a structured set of facts from each. Then combine those notes in a final prompt, keeping the original section references. For questions that depend on information spread across multiple sections, check those facts directly against the source. Splitting works well for some extraction jobs and can lose important relationships in others.
Keep the smallest context that reliably fits each recurring task. Write down two settings if you use one workflow for short questions and another for long documents. After a model or application update, rerun the same sample so you know whether its behavior changed. To make comparisons fair, also record how many source pages or sections you provided, whether they were pasted or uploaded, and roughly how long you asked the answer to be. Longer answers and different input paths can make the same configuration feel very different.
Keep a short record of any limits your client adds, including file-size caps and conversation trimming. Test the same model through the terminal and through your usual interface if the results differ. A terminal test with pasted text gives you a direct baseline. If the chat app’s document feature gives a different answer, inspect how it reads the file. Scanned pages may need text recognition, tables may be extracted in an unexpected order, and some apps send selected chunks. Check a small portion you know well before assuming a bigger context is the remedy. Save a representative input and expected answer so you can repeat this comparison after a software update without having to reconstruct the original session.
Frequently asked questions
Does increasing context improve every answer?
It can help when the model needs information that otherwise does not fit. Short questions may see little benefit, and the answers still need to be checked against the source.
Is the context length the maximum answer length?
Context includes the input and the text generated during the response. A client can also impose a separate limit on output length.
What should I check if Ollama becomes slower?
Compare the processor split and memory use with your earlier run, then test a shorter context or a smaller model against the same document.
Can I keep an indefinitely long chat by raising context?
The model and runtime still have finite limits. Start a fresh chat when useful and bring forward only the facts and instructions that the next task needs.
