Skip to main content
Local LLM Guide

Model

Dolphin 3.0 Llama 3.1 8B

Start with a supported compressed build and short context. 16GB RAM estimate. Best use: steerable local assistant workflows

8.03BText chat

Best use

steerable local assistant workflows · coding and math assistance in the 8B class

Memory planning

16GB RAM estimate

Expected pace

Depends on your hardware, app, and conversation length

License

Llama 3.1 Community License; read the model’s terms before use

Who should use this model?

  • steerable local assistant workflows
  • coding and math assistance in the 8B class
  • function-calling and agentic local experiments
  • users who want a less refusal-heavy local model

Choose something else when

  • you need official Meta safety alignment
  • you need a model with a clearly supported release date
  • you need supported Q5_K_M or Q8_0 file sizes for the publisher GGUF artifact

What hardware makes sense?

The download is only part of the memory needed. Leave room for the app, the input it processes, and other open programs. For chat models, context includes the instructions, conversation, and answer. Longer context needs more memory.

CPU only
A possible starting point with a compressed model and short conversation; check the exact file size.
8GB system memory
Memory or response time may limit use. Try a smaller model first.
16GB system memory
A possible starting point with a compressed model and short conversation; check the exact file size.
32GB system memory
A possible starting point with a compressed model and short conversation; check the exact file size.
8GB GPU memory
A possible starting point with a compressed model and short conversation; check the exact file size.
12GB GPU memory
A possible starting point with a compressed model and short conversation; check the exact file size.
24GB GPU memory
A possible starting point with a compressed model and short conversation; check the exact file size.
Apple unified memory
A possible starting point with a compressed model and short conversation; check the exact file size.

Model facts that change the decision

Parameters
8.03B
Input
Text
Published context limit
128,000 tokens
Model format
safetensors · GGUF · Ollama
Suggested compression
Q4_K_M
Typical Q4 file
4.92GB before runtime and context overhead
Languages
English

First setup

Ollama

ollama run dolphin3:8b

LM Studio: Use dphn/Dolphin3.0-Llama3.1-8B-GGUF and select Q4_K_M; check the file size before downloading a less-compressed version.

llama.cpp

llama-server -hf dphn/Dolphin3.0-Llama3.1-8B-GGUF:Q4_K_M

Conditions that change the answer

  • Memory needs include the model file, the app, and conversation memory. Longer conversations and other open apps need more space.
  • Only the Q4_K_M file size was supported from the fetched publisher GGUF listing; q5FileSizeGb and
  • This is a derivative of Llama 3.1 8B and remains subject to the Llama 3.1 license.
  • The context-window value is inherited from the Llama 3.1 8B base family rather than independently stated as a Dolphin-specific training limit in the fetched model card.

Read the controlling license text →