Local AI on Your Laptop: Ollama, LM Studio and Privacy

Running a model on your own laptop used to be a hobby project. Now it takes about ten minutes, and for privacy-minded professionals it solves a real problem: you can summarise a confidential contract or draft a reply to a sensitive email without sending a word to anyone's servers. This guide covers the setup, the hardware maths and the privacy settings that actually matter.

Why run AI locally at all?

Local models trade some quality for control. The control is worth it in a few clear cases:

• Confidential material. Client documents, health or legal notes, unreleased financials, anything your contract or policy says cannot go to a third-party AI service.
• Offline work. Flights, trains and sites with poor connections.
• Predictable cost. No per-token bill for heavy, repetitive tasks such as tagging or rewriting.
• Building and testing. A local model behind an OpenAI-compatible API is a cheap way to develop AI features before you switch to a hosted model.

The trade-off is that a model small enough for a laptop is weaker than the frontier models behind cloud APIs, and slower on long answers. I explain where small models are good enough in small language models and on-device AI. In short: summarising, extracting, classifying and short drafting work well; long multi-step reasoning is where you notice the gap.

Ollama vs LM Studio: which should you install?

Both are free and both run open-weight models on your own hardware. The difference is how you use them.

Sources: Ollama FAQ, Ollama OpenAI compatibility, LM Studio OpenAI compatibility.

If you want a chat window today, start with LM Studio. If you want to call a local model from code, start with Ollama. Many people end up with both.

What hardware do you need to run an LLM locally?

Memory decides almost everything. The model's weights must fit in RAM (or in GPU memory on a PC with a separate graphics card), with room left for the conversation and your other apps.

LM Studio's system requirements say 16 GB or more of RAM is recommended on both macOS and Windows. On a Mac it needs Apple Silicon and macOS 14 or newer; on Windows, an x64 processor with AVX2 support or an ARM (Snapdragon X Elite) machine; on Linux, Ubuntu 20.04 or newer.

To estimate whether a model fits, use simple arithmetic. Most local models are downloaded in 4-bit quantised form, which means about half a byte per parameter:

• An 8-billion-parameter model at 4 bits is roughly 4 GB of weights.
• A 14-billion-parameter model at 4 bits is roughly 7 GB.
• A 32-billion-parameter model at 4 bits is roughly 16 GB.

Add a few gigabytes for the conversation context and leave room for your browser and operating system. In practice, a 16 GB laptop is comfortable with models up to around 8 billion parameters, and a 32 GB machine can try the mid-size ones. Download pages in both tools show the file size, which is a good proxy for the memory you need.

How to set up Ollama and LM Studio

LM Studio is the quicker route for most people:

• Install the app for your operating system.
• Open the Discover tab, search for a model, and pick a size your memory can hold.
• Load it and start chatting. To ask questions about a file, attach it to the chat; LM Studio's offline guide says "all document processing is done locally".

Ollama suits developers:

• Install Ollama, which runs a small server in the background.
• Run ollama pull with a model name to download it, then ollama run with the same name to chat in the terminal.
• Call it from code by pointing any OpenAI client at http://localhost:11434/v1/. The compatibility docs note that the client needs an API key value but "Ollama ignores it", so any placeholder works.

One setting catches people out. The Ollama FAQ gives a default context length of 4,096 tokens, so long documents get cut off without warning. Raise it with the OLLAMA_CONTEXT_LENGTH environment variable, or num_ctx per request, if your memory allows.

Which model should you try first?

Start small and move up only if the answers are not good enough. A model that fits comfortably in memory answers faster and leaves your laptop usable, which matters more for daily work than a slightly better benchmark score.

A practical approach:

• Pick a well-known open-weight family in the 3 to 8 billion parameter range, in a 4-bit version.
• Give it five real tasks from your week, such as a summary, an extraction and a short draft.
• If it fails on the tasks you care about, try the next size up, or a model tuned for that job, such as one built for code.
• Keep one small, fast model for quick jobs and one larger model for harder ones, if your memory allows.

Both tools list a model's size before you download, so you can check the fit first.

The privacy settings that actually matter

Local does not automatically mean private. Check four things before you put confidential material into either tool.

1. Turn off cloud features you do not use. Ollama now offers cloud-hosted models and web search alongside local ones. Its FAQ says that in local mode it does not see your prompts, and that cloud prompts are processed to provide the service but not stored or used for training. If you want a guarantee that nothing leaves the laptop, disable cloud features: set OLLAMA_NO_CLOUD=1 or add "disable_ollama_cloud": true to ~/.ollama/server.json, then restart.

2. Keep the server on localhost. Ollama listens on 127.0.0.1:11434 by default, which only your own machine can reach. Changing OLLAMA_HOST to 0.0.0.0 exposes it to your network, and anyone who can reach it can use your model and see what it is asked. Only do that behind a firewall you control.

3. Know what goes online. LM Studio's offline guide says "nothing you enter into LM Studio when chatting with LLMs leaves your device", but model search, catalogue stats, downloads and app updates do make network requests. Download models first, then work offline if you want certainty.

4. Check the model licence. Open-weight models come with their own licences, and some restrict commercial use. Read the licence on the model page before using it for client work.

If you run a team, write these rules down. Which local tools are approved, which models, and what data may go into them belongs in your AI usage policy. And if the same work later moves to a cloud model, the controls change; I cover those in securing LLM apps in the cloud.

What local models are good for, in practice

In my own work I treat a local model as a private assistant for narrow tasks, not a replacement for a frontier model. The jobs that suit it:

• Summarising and extracting fields from confidential documents.
• Rewriting and tidying text that should not leave the machine.
• Classifying or tagging batches of records where a per-token bill would add up.
• A free development endpoint while building an AI feature, before switching to a hosted model.

For long reasoning, complex code changes or anything customer-facing, I still use the strongest hosted model that the client's data rules allow.

Work with me

If you want AI on confidential data, either running locally or on a cloud set up to your security team's standards, I can help you choose the setup and build it into your workflow. See AI integration and full-stack development, or browse my work.