Local LLM Guide

Model runner and API

Ollama

Use Ollama when you want a lightweight local model runner, command-line workflow, or API that other applications can call.

Version in this record: 0.33.0
Use it when

running downloaded models through a simple command-line runtime;

Avoid it when

you only want a polished graphical chat app and do not need an API.

Data flow

Map model downloads, providers, web tools, sync, telemetry, and remote access for the exact configuration you use.

Easiest alternative

LM Studio when you want a graphical desktop app

What this product does

Local inference engine, CLI, REST API server, and agent layer documented by Ollama and commonly paired with other local AI apps.

  • Model runner
  • CLI
  • Local API server
  • Optional agent and cloud gateway

Where it fits in a local AI setup

1App or terminal
2Ollama
3Local model
4Response

Map model downloads, providers, web tools, sync, telemetry, and remote access for the exact configuration you use.

API capabilities that matter

Chat completionsdocumented; endpoint-specific behavior remains version-scoped
Responsesdocumented; endpoint-specific behavior remains version-scoped
Embeddingsdocumented; endpoint-specific behavior remains version-scoped
Model listingdocumented; endpoint-specific behavior remains version-scoped
Tool callingdocumented; model support is separate from runtime support
Structured outputdocumented; model and schema reliability are separate
Streamingdocumented; endpoint-specific behavior remains version-scoped

First successful setup

  1. Install Ollama.
  2. Run one known model from the terminal.
  3. Ask a simple question and confirm a normal response before adding Open WebUI or document tools.

What success looks like: the model stays loaded long enough to return a normal terminal response and appears through the local API.

Choose something else when

  • users who assume every Ollama command stays on-device;
  • built-in PDF chat without another interface;
  • sensitive work before model tags, web tools, and server exposure are reviewed;
  • performance claims without named hardware and reproduced testing.

License

Check the publisher’s software license before redistribution, modification, or hosted deployment. Model licenses are separate from the application license.