LOCAL_AI_STACK

Local AI runtimes and tools

LOCAL_AI_STACK / runtimes(7)
v2.3 · 2026-08-09
TYPE7 of 7 runtimes/tools
OllamaRUNTIME
**-moderate
FEATURES
OpenAI API: partial
Document chat: possible with integrations
Docker: optional
Multi-user: no
GPU support: yes
FIT_NOTES
Local inference engine, CLI, REST API server, and agent layer documented by Ollama and commonly paired with other local AI apps.
API_BEHAVIOR
Local REST API; privacy and network exposure depend on host binding, providers, pulls, and configuration.
BEST_FOR
running downloaded models through a simple command-line runtime;
local API experiments and application integrations;
pairing a local model backend with Open WebUI;
developers who explicitly choose between local and hosted models.
NOT_GOOD_FOR
users who assume every Ollama command stays on-device;
built-in PDF chat without another interface;
sensitive work before model tags, web tools, and server exposure are reviewed;
performance claims without named hardware and reproduced testing.
mac / windows / linux · hybrid-> view full record
LM StudioRUNTIME
*--beginner
FEATURES
OpenAI API: yes
Document chat: built in
Docker: not required
Multi-user: no
GPU support: yes
FIT_NOTES
Desktop local model GUI, local API server, and headless runtime; the vendor also offers Bionic as a separate hybrid agent application.
API_BEHAVIOR
Desktop app can expose local API/server modes; serving beyond localhost changes the risk boundary.
BEST_FOR
beginner-friendly local model browsing and chat;
low-level control over quantization, context, and runtime selection;
document attachments and local document experiments;
optional local API and headless-server workflows.
NOT_GOOD_FOR
treating the vendor's entire product line as local-only;
multi-user self-hosted web interfaces;
assuming offline or private behavior without checking the selected path;
performance claims without named hardware and reproduced testing.
mac / windows / linux · hybrid-> view full record
**-moderate
FEATURES
OpenAI API: partial
Document chat: provider dependent
Docker: common path
Multi-user: yes
GPU support: unknown
FIT_NOTES
Self-hosted browser UI and provider layer with documented Ollama/provider connections, files, knowledge bases, RAG, and MCP-related tooling.
API_BEHAVIOR
Connects to Ollama or other providers; the provider choice controls where model calls go.
BEST_FOR
a browser workspace over one or more model providers;
self-hosted personal, home-lab, or small-team workflows;
shared folders, document workflows, and multi-user access;
administrators comfortable with server, identity, and network settings.
NOT_GOOD_FOR
users who do not want to administer a server;
assuming that a self-hosted interface means the model and tools are local;
sensitive documents before provider, embedding, storage, and sharing settings are reviewed;
public exposure without deliberate authentication and access controls.
web / docker / linux / mac / windows · hybrid-> view full record
**-moderate
FEATURES
OpenAI API: unknown
Document chat: built in
Docker: optional
Multi-user: yes
GPU support: unknown
FIT_NOTES
Document-chat, RAG, and agent workspace whose version 1.15 desktop features can also act on text and screen context outside the application window.
API_BEHAVIOR
Provider layer; LLM, embedder, vector database, and tool choices determine local/cloud behavior.
BEST_FOR
document chat and RAG-style workflows;
workspaces organized around files;
local or private knowledge-base experiments;
users who want optional agent and system-wide desktop features.
NOT_GOOD_FOR
assuming local-only behavior without reviewing every provider;
granting system-wide access without understanding the scope;
benchmark or citation-accuracy claims without testing;
sensitive use before storage, telemetry, and tool settings are reviewed.
mac / windows / linux / docker · hybrid-> view full record
JanRUNTIME
*--beginner
FEATURES
OpenAI API: unknown
Document chat: unknown
Docker: not required
Multi-user: no
GPU support: unknown
FIT_NOTES
Actively maintained local-first desktop application with a local API server, optional remote providers, and opt-in web tools.
API_BEHAVIOR
Current provider and API behavior needs current official documentation before stronger site guidance.
BEST_FOR
users comparing local desktop apps;
local chat with a built-in API server;
local and remote provider experimentation with an explicit privacy review;
people who want llama.cpp and MLX options in a desktop interface.
NOT_GOOD_FOR
assuming every enabled tool is local;
privacy-sensitive work before provider and credential settings are reviewed;
performance or hardware-fit claims without testing.
mac / windows / linux · hybrid-> view full record
llama.cppLIBRARY
***advanced
FEATURES
OpenAI API: unknown
Document chat: possible with integrations
Docker: optional
Multi-user: no
GPU support: unknown
FIT_NOTES
Technical local inference library and server used directly by advanced users and indirectly by local AI apps.
API_BEHAVIOR
Backend/library path; app behavior depends on build flags, serving mode, model format, and wrapper.
BEST_FOR
technical users who need direct runtime control;
GGUF and quantized-model experimentation;
explicit GPU layer offload and backend selection;
embedding, reranking, server, router, and speculative-decoding workflows.
NOT_GOOD_FOR
one-click beginner setup;
casual users who only want a chat window;
exact model and GPU compatibility claims without current testing;
copied commands from old guides without checking current flags.
mac / windows / linux · local-first-> view full record
GPT4AllRUNTIME
*--beginner
FEATURES
OpenAI API: unknown
Document chat: unknown
Docker: not required
Multi-user: no
GPU support: unknown
FIT_NOTES
GPT4All is a local AI app/project category entry. It belongs in the ecosystem map, but stronger recommendations need current official documentation.
API_BEHAVIOR
Current app/API behavior needs current official documentation before stronger site guidance.
BEST_FOR
users researching desktop local AI apps;
comparison completeness;
future local AI app alternatives pages.
NOT_GOOD_FOR
high-confidence launch recommendations before current review;
benchmark claims;
exact compatibility claims.
mac / windows / linux · unknown-> view full record
RUNTIME DIRECTORY CONTEXT

How to compare local AI runtimes and tools

Intro

This directory explains the difference between local runtimes, desktop apps, browser interfaces, RAG/document apps, and lower-level libraries.

Category explanation

CategoryMeaningExamples
RuntimeRuns models locally and exposes CLI/API behavior.Ollama, llama.cpp
Desktop appGives users a GUI for downloading and chatting with models.LM Studio, Jan, GPT4All
Browser UIWeb interface layered over local or cloud providers.Open WebUI
RAG/document appFocuses on document ingestion, retrieval, and workspace workflows.AnythingLLM

Directory caution

Do not treat all tools as equivalent. A runtime, GUI, RAG app, and self-hosted web UI solve different problems and carry different privacy risks.