You can run a text-based AI assistant on a PC or Mac with Ollama and a downloaded language model. Start with a small model, give it one short task you can check, and repeat that task with the computer disconnected from the internet. That sequence tells you whether the model fits your machine, whether the results are worth using, and whether your setup can work offline.
Check your hardware before downloading a model
Start with the memory you have available while doing normal work. On Windows, open Task Manager and look at the Memory and GPU entries under Performance. On a Mac, use Activity Monitor and check Memory Pressure while your usual apps are open. If the computer already struggles with several browser tabs and a video call, close those apps before your first run. A model can run on a machine that feels busy, but your first test should give it a fair amount of free memory.
A dedicated GPU has its own video memory. Apple silicon Macs use unified memory shared across the CPU and GPU. Some PCs use an integrated GPU that shares system memory. These configurations make a single universal RAM recommendation misleading. The downloaded file is only part of the budget. The runtime also needs working memory, and longer conversations increase the memory needed for context. Ollama explains how to inspect context size and GPU offloading in its context length documentation.
For a first text chat, use a small model. Ollama’s Llama 3.2 model listing provides 1B and 3B options. Try the 1B version if you have limited headroom, or start with 3B for a little more room to handle instructions. These examples give you a repeatable starting point. The model that suits your hardware is the one that completes your actual task at a speed you are willing to use.
Install Ollama and run a model
Download Ollama for your operating system from its official download page. Follow the installation instructions for your operating system, then open a fresh Terminal window on macOS or Linux, or PowerShell on Windows. The first command checks whether the terminal can find the Ollama executable.
ollama --version
ollama run llama3.2:3bIf you want the smaller model, use ollama run llama3.2:1b. The first run downloads the selected model, so leave the computer online until it finishes. Once the chat prompt appears, enter a request such as “Explain the difference between system RAM and disk storage in three sentences for someone buying their first laptop.” Keep your first prompt short so that it tests the basic setup. If the shell says it cannot find Ollama, reopen the terminal after installing it and check that the app or service is running.
When you finish the chat, enter /bye to return to the shell. Run ollama list to see the models downloaded to the computer. Use the exact model tag in your notes if you plan to compare variants. A reusable note with the model tag, your prompt, and the date makes the result easier to reproduce later, especially if you install a second model or change runtime settings.
Test the answers on a task you can check
Use a short piece of source text with concrete details. For example, paste this mock note into the chat: “The monitor arrived Tuesday. Its stand is missing. The return window ends on 30 November. We have yet to contact the retailer.” Ask the model for three bullets: what happened, what is missing, and what should happen next. Tell it to label any suggestion as a suggestion and to use only the note for factual statements.
Read the answer against the original note. Check whether the model copied the date correctly, kept the missing stand distinct from a defective monitor, and avoided claiming that the retailer has already responded. Then change one fact in the note and run the prompt again in a fresh chat. If the model keeps repeating the original fact after a change, that failure matters more for this workflow than the polish of its prose.
Use the same method for a task you actually do. A developer might ask the model to explain a short function with no project secrets. Someone sorting meeting notes can give it a paragraph and ask it to extract names, dates, and open decisions. Keep a separate record of correct answers for facts that matter. A model can confidently misread an instruction or invent a detail, so a short source passage and a clear expected output make errors easier to catch.
Verify that your setup works offline
After the model is downloaded, exit the chat. Disconnect Wi-Fi and Ethernet, open a new terminal, and run the same model again. Use a new prompt and check that it produces an answer while disconnected. This tests the path from your prompt to the model and back under the conditions you care about. Reconnect when you need to download a new model or update the software.
Use the same test for any extra interface you add. A desktop chat client or browser front end can call a local Ollama server while also offering optional services with their own network connections. Keep sensitive material out of an unfamiliar interface until you understand where it sends requests and how it stores chat history. For a shared computer, also check who can access your files and saved conversations through the operating system.
Troubleshoot slow responses
Try a short prompt with a short expected answer. If it is slow from the first message, open a second terminal while the model is loaded and run ollama ps. Check the processor column to see whether the model is using the CPU, GPU, or both. Compare the result after closing applications that consume memory. A CPU-heavy run can be usable for occasional prompts, but it may feel too slow for a chat you depend on throughout the day.
If the first response is quick and the chat slows after several turns, start a fresh conversation and paste only the parts of the old one that matter. Compare the same task at the same output length with both model sizes. If the smaller model preserves the facts and the required format, keep it for that task. Save the larger one for prompts where your own comparison shows an improvement. That gives you a reason to spend the extra memory.
For the first week, keep your setup simple: one local model, one terminal, and a handful of prompts with answers you can verify. Once that works reliably, decide whether you need a graphical interface or a model better suited to your language or subject. Model cards and your own repeatable examples make that decision easier than a generic leaderboard.
Browse our AI coverage for more guides on choosing and using AI tools.
Frequently asked questions
Can I run local AI without a dedicated graphics card?
Yes. Try a small model and a short prompt first. CPU performance varies by machine, and the result you care about is whether it produces a useful answer quickly enough for your workflow.
Do I need internet access every time?
You need internet access to download a model initially. A downloaded local model can answer offline. Disconnect and test the exact model and app you intend to use.
Does more memory make the answers more accurate?
More available memory can let you load a larger model or use a longer context. Check accuracy separately with a short set of questions whose answers you know.
Can I use the same model for images?
The Llama 3.2 1B and 3B variants in this guide are text models. For image input, choose a model whose listing explicitly documents vision support and test it on a representative image.