Ollama 0.35 adds a local decision API for choosing labels, answering yes/no questions, and scoring text. You can use it to route a support message to billing or technical support without asking a chatbot to write an explanation. This guide uses Nimble's explicit GGUF tag and a fictional ticket; it does not connect to a real inbox or send a reply.
Ollama announced the feature on September 29. The practical change is the new /v1/systemone endpoint. Your application supplies the allowed outcomes, and the response uses named fields that a script can read.
Before you start
- Install or update Ollama using the installation guide. The running server must be 0.35.0 or newer.
- Have enough free disk and memory for the model and its working state.
nimble:9b-q4_K_Mis a 5.6GB download, with Q4_K_M quantization; that file size is not a total RAM requirement. Use the hardware checker to understand memory headroom. - Use Terminal on Mac/Linux or PowerShell on Windows. Start the Ollama app, or your existing Linux service, before making requests.
- Keep the service on its default loopback address. This recipe calls
127.0.0.1:11434on your own computer.
The decision endpoint requires a compatible GGUF model and scoring runner. Do not substitute Nimble's MLX tags or a cloud model simply because their names look similar.
1. Download the exact model
Run these commands:
ollama --version
ollama pull nimble:9b-q4_K_M
ollama list
Look for nimble:9b-q4_K_M in the list. Nimble is a roughly 9B model from Bespoke Labs, based on Qwen3.5; its listed license is Apache 2.0. The explicit tag selects the smaller GGUF variant instead of the changing default tag.
2. Save a complete decision request
Create a plain-text file named ticket.json in your working folder and paste this JSON:
{
"model": "nimble:9b-q4_K_M",
"state": {
"ticket": "I paid for the same order twice. Please refund the second payment."
},
"questions": {
"team": {
"type": "choice",
"instructions": "Which team should handle this message?",
"criteria": {
"billing": "Duplicate payments, charges, or refunds",
"technical": "Software faults or broken integrations",
"other": "The message does not fit either team"
}
},
"refund_requested": {
"type": "noul",
"instructions": "Does the customer explicitly request a refund?",
"criteria": {
"true": "The customer asks for money to be returned",
"false": "The customer does not request money back"
}
}
}
}
noul is the API's name for a yes/no probability. Keep the spelling exactly as shown. Both questions use the same supplied ticket; one answer does not become the input to the next question.
3. Send it to your local server
On Mac or Linux, run from the folder containing ticket.json:
curl --fail-with-body http://127.0.0.1:11434/v1/systemone \
-H 'Content-Type: application/json' \
--data-binary @ticket.json
On Windows PowerShell:
$decision = Invoke-RestMethod -Uri 'http://127.0.0.1:11434/v1/systemone' -Method Post -ContentType 'application/json' -InFile '.\ticket.json'
$decision | ConvertTo-Json -Depth 10
The first result to check is a JSON object with answers.team.choice, the probabilities for each team, and answers.refund_requested.noul. For this fictional message, the intended label is billing and the refund probability should favor true. Inspect your actual response; no particular number is guaranteed.
Change the ticket to "The app crashes whenever I upload a photo" and resend it. The intended team now becomes technical. These two examples give you a simple way to spot a reversed rule or a request that never changed.
Read probabilities without overtrusting them
The returned probabilities describe which answer the model favors. They do not establish that the classification is correct. A model can favor a wrong answer strongly, so inspect the selected label against the supplied text.
For a first workflow, use the label to prepare a queue for human review. Include an other outcome for text outside your categories. Before automating routing, collect representative messages with known answers and inspect mistakes, including messages containing several issues.
The third question type, score, accepts an ordered list of descriptions. Its result is a probability-weighted average of levels numbered from zero. For three levels, read it on a 0–2 scale; it is not automatically a percentage.
If the request fails
| What happens | What to check |
|---|---|
| Connection refused | Start Ollama and confirm the address. Follow the Ollama service and API checks. |
| Endpoint returns 404 | Check the running server's version and the exact /v1/systemone path; updating only a separate CLI does not update a still-running old server. |
| Model not found or unsupported | Compare the request's tag with ollama list; use the GGUF tag above, not an MLX or hosted variant. |
| Invalid request or 400 | Check valid JSON, the question type, and named criteria. Begin with the unchanged example. |
| 413 or context error | Shorten the state. Text-only bodies must fit within 64 KiB, and the complete input must fit the loaded context. |
| Out of memory | Close other model sessions and applications; use the memory failure guide. |
Start with short tickets. Sending an entire document collection to a classifier makes both your intended question and its errors harder to inspect.
What about the smaller Tev1 models?
Ollama also lists experimental Tev1 models in 4B and 0.8B sizes. The smaller model is an alternative for investigating tight memory budgets, but its training and behavior differ from Nimble. Its model card says the fine-tuned weight license is still being finalized. The project's MIT-licensed training code does not establish permission to redistribute or commercially deploy those weights. Resolve that license before choosing Tev1 for such a use.
Keep the whole workflow local
This request uses local inference without an API key. Downloads still use the internet. A separate app that forwards the ticket to a hosted model or web tool creates another network path.
Ollama documents OLLAMA_NO_CLOUD=1 as a way to disable its cloud models and web search; restart Ollama after changing its server environment. That setting does not control unrelated tools or apps. Use the local AI privacy guide before connecting real documents or messages.
Next, keep this as a small reviewed classifier, or follow the PDF workflow if you need answers drawn from longer documents.