To show a local model’s answer as it arrives, call Ollama’s native chat endpoint with streaming enabled and process each newline-delimited JSON record in order. Append the content from each record, print it or send it to your interface, and wait for the final completion signal before saving the assistant turn. This makes a terminal or app feel more responsive during generation and gives you a clear way to handle a connection that closes before the reply is complete.

Use the native chat endpoint

Start Ollama locally and choose a chat model already available on the computer. You can run ollama list to see the model names. This example uses Python 3 and its standard library to call http://localhost:11434/api/chat. Replace the placeholder in the example with the exact model tag from your installation. If you have never made a local request, our Ollama Chat API guide covers the underlying endpoint and the message array.

Ollama’s native streaming REST endpoints return a series of JSON objects separated by newlines. Each record can carry part of the text or other information. The parts are not guaranteed to end at a word boundary. Join them as received without adding a space between chunks. Ollama’s streaming documentation describes the behavior and how it differs by client. Set stream explicitly so the next person who reads the code can see which response format it expects.

Read and display one record at a time

Use the following script as a minimal terminal client. The timeout applies to the underlying blocking operations supported by the connection, so treat it as a practical safeguard against stalled I/O. The script returns the assembled text only after it sees a successful completion record. A readable partial answer can appear in the terminal even if the request ultimately raises an exception.

import json
from urllib.request import Request, urlopen

def stream_chat(model, messages):
    payload = {"model": model, "messages": messages, "stream": True}
    request = Request(
        "http://localhost:11434/api/chat",
        data=json.dumps(payload).encode("utf-8"),
        headers={"Content-Type": "application/json"},
    )
    parts = []
    with urlopen(request, timeout=120) as response:
        for line in response:
            if not line.strip():
                continue
            event = json.loads(line)
            if "error" in event:
                raise RuntimeError(event["error"])
            text = event.get("message", {}).get("content", "")
            if text:
                parts.append(text)
                print(text, end="", flush=True)
            if event.get("done") is True:
                print()
                return "".join(parts)
    raise RuntimeError("The stream ended before Ollama signaled completion.")

reply = stream_chat(
    "YOUR_LOCAL_MODEL",
    [{"role": "user", "content": "Explain why the sky is blue in two sentences."}],
)

Save the file and run it from a terminal. The repeated prints may look like ordinary typing if the model is fast. Try a slightly longer request to see the incremental updates. Then print the returned value separately in a test to confirm it matches the displayed response. If the first run takes time before the first chunk, try the same prompt again and observe whether starting a model that was already loaded makes a difference. Keep both measurements if you are deciding what progress indicator to show in an interface.

Handle early failures and partial replies

A connection failure, missing model or rejected request can occur before the response starts. Catch the relevant exception in the code that calls the helper and show a useful message. For example, if the connection was refused, prompt the user to check whether Ollama is running locally and whether the configured URL is correct. If the requested model is not available, show the configured model name and have the user choose one installed on their machine. Separating those cases matters because the fixes are different.

A response can also fail after some chunks appeared. Keep the partial text visible if it is useful and label its state in the interface. Offer a retry from the last complete conversation state. If you let a user cancel generation, treat that as a distinct outcome and close or cancel the in-flight request through the client you are using. A GUI should indicate when a reply is still running and reenable the relevant controls when the request ends, fails or is canceled.

Append the complete assistant reply to the message history once after success. If your interface adds the pending user message before sending, track that pending request separately so a retry does not add a second copy of the same user turn. Try a two-turn prompt with a deliberately interrupted first attempt to see whether the final request context includes only the intended messages. The visible conversation and the request history should agree when the user starts the next turn.

Choose how much to process incrementally

This example displays only the chat record’s text content. Some models and workflows return additional fields, including reasoning or tool calls where supported. Handle those according to the client and model you use, and preserve any fields needed to continue that conversation. For a first integration, keep the prompt plain and verify the basic text path before extending it. Try a response that has empty content chunks to make sure the interface does not insert extra blank lines.

If the model is returning machine-readable JSON, the incremental text can be incomplete until the final chunk. Collect the text and validate it before passing it to the next component. If there is no benefit to showing partial data, request a single response with stream set to false and parse the complete result. Our guide to structured JSON output with Ollama covers how to specify and verify the expected shape.

For a small regression test, feed the parsing logic a saved fixture with two text records and a final completion record. Check that the joined text is exact. Then remove the last record and verify that the failure path runs. This exercises both outcomes without depending on how a model happens to split its response on a particular run.

If you are integrating streaming into a graphical app, keep the network loop away from the UI thread. Pass text updates through the framework’s supported queue or event mechanism, and attach a request identifier so late events from an old request do not appear in a new conversation. Test by starting a reply, navigating away, and returning to the chat. Then look at what the interface displays and what was written to the saved conversation. Avoid measuring only the total completion time. The time until the first visible text and the pause between visible updates usually tell you more about how the interface feels.

Keep a non-streaming request with the same model and messages available for diagnosis. If both requests fail before any content appears, inspect the endpoint and payload first. If the non-streaming request succeeds while your streaming display stays blank until the end, check whether a wrapper reads the entire body before handing it to your parser. A server-side bridge can also buffer writes even when your Python code reads lines correctly. Compare when the first record reaches the Python process with when the interface paints the first text. Logging only timestamps and record types is often enough for this check and avoids writing the prompt and response to a debug log.

Frequently asked questions

Why does reading the entire response as one JSON object fail?

A native streaming response can contain multiple JSON objects, one per line. Parse each line separately or request a non-streaming response when the consumer expects a single JSON document.

Why do the displayed words arrive in uneven pieces?

The chunks represent pieces of the generated text, and their boundaries can fall inside a word or around punctuation. Concatenate the content in order. Limit interface redraw frequency separately if the display flickers or updates too often.

Does this work with the OpenAI-compatible endpoint?

The example targets the native Ollama chat endpoint and reads its record format. When you use an OpenAI-compatible client, use the streaming interface provided by that client and inspect its event format before reusing a parser.

How do I know a reply is complete?

Watch for the native completion record. If the connection ends without it, report that the result is incomplete. Keep the incomplete text separate from the finalized assistant message so the next request can start from the intended conversation state.