Skip to main content
Local LLM Guide

Runs models for other apps

Ollama

Use Ollama for desktop chat on Windows and Mac, terminal commands, or a local API for other apps.

Use it when

you want desktop chat on Windows or Mac, terminal commands, or a local API for other apps.

Avoid it when

you want a complete document workspace without connecting another app.

Where your data goes

Downloaded local models process prompts on this computer. Cloud models and web search send requests outside it. Set OLLAMA_NO_CLOUD=1 and restart for local-only mode; the log should say Ollama cloud disabled: true. Keep the default loopback address unless other devices need access. Downloads still need internet.

Easiest alternative

LM Studio when you want to browse GGUF files and loading controls in the app

What this product does

Use Ollama for desktop chat on Windows and Mac, terminal commands, or a local API for other apps.

  • Runs a downloaded model on your computer
  • Lets you control it by typing commands
  • Lets other apps use a model running on your computer
  • Can also run automated tasks or connect to cloud models when you choose those features

Where it fits in a local AI setup

  1. 1App or terminal
  2. 2Ollama
  3. 3Local model
  4. 4Response

Connecting other apps through an API

Chat completions
POST /v1/chat/completions: chat messages and streamed answers.
Responses
POST /v1/responses: supported Responses requests; compatibility is a subset of the hosted API.
Embeddings
POST /v1/embeddings: requires an embedding model, not an arbitrary chat model.
Model listing
GET /v1/models: use a returned model identifier in requests.
Tool calling
Tool-capable models can emit tool calls; the client executes them and returns results.
Structured output
JSON/schema requests still need a model and runtime supporting the selected schema. Validate the returned object.
Streaming
Request stream: true on a supporting generation endpoint.

Local base address: http://localhost:11434/v1. Use the port shown by your server. Endpoint availability does not establish every model supports every task. The native API uses /api. A local Ollama server can also route to cloud models; disable cloud features for local-only work.

First successful setup

  1. Install Ollama. On Windows and Mac, open its chat window and select the local qwen3:1.7b model; wait for the download. Choose a local model rather than a cloud model.

  2. In the desktop app, set the context length to 4096 in Settings before starting the conversation. For the terminal equivalent, run ollama run qwen3:1.7b; inside that conversation, run /set parameter num_ctx 4096.

  3. Ask: Rewrite “The meeting will commence at 10:30” in plain English. Keep the time unchanged. Check that the answer preserves 10:30.

  4. In the desktop app, stop generation if necessary and open a new chat with the same local model to use it again. In the terminal, exit with /bye. Run ollama list to see downloads and ollama ps while a model is loaded to inspect CPU/GPU placement. Reopen with the same run command. If the command is missing, follow the installation guide below; if loading fails, try qwen3:0.6b before adding integrations.

What success looks like: the selected local model returns a rewrite preserving 10:30 in the desktop app or terminal. While loaded, it also appears in ollama ps.

Choose something else when

  • document uploads without a companion interface: use a document-capable app;
  • graphical model browsing and load controls: compare LM Studio;
  • a model architecture unsupported by the installed runner: choose a supported runner for that exact download.

License

The application source uses the MIT license. Retain its required notices when distributing it. Downloaded models and hosted services have separate terms.

License: MIT. Read the controlling software terms