Skip to main content
Local LLM Guide

Model

LLaVA-Llama-3 8B v1.1

Start with a supported compressed build and short context. 16GB RAM estimate. Best use: Local Llama-3-based image chat

8.03BVision language

Best use

Local Llama-3-based image chat · Commodity multimodal demos with a newer Llama base than Vicuna LLaVA 1.6

Memory planning

16GB RAM estimate

Expected pace

Depends on your hardware, app, and conversation length

License

Meta Llama 3 Community License plus XTuner/LLaVA component notices; read the model’s terms before use

Who should use this model?

  • Local Llama-3-based image chat
  • Commodity multimodal demos with a newer Llama base than Vicuna LLaVA 1.6
  • LLaVA-compatible llama.cpp workflows
  • Visual question-answering experiments

Choose something else when

  • You need a pure official Meta multimodal model
  • You need supported Q5_K_M or Q8_0 file sizes from the model listing
  • You need tool calling
  • You cannot manage projector/mmproj requirements in manual runtimes

What hardware makes sense?

The download is only part of the memory needed. Leave room for the app, the input it processes, and other open programs. For chat models, context includes the instructions, conversation, and answer. Longer context needs more memory.

CPU only
Memory or response time may limit use. Try a smaller model first.
8GB system memory
Memory or response time may limit use. Try a smaller model first.
16GB system memory
A possible starting point with a compressed model and short conversation; check the exact file size.
32GB system memory
A possible starting point with a compressed model and short conversation; check the exact file size.
8GB GPU memory
Memory or response time may limit use. Try a smaller model first.
12GB GPU memory
A possible starting point with a compressed model and short conversation; check the exact file size.
24GB GPU memory
A possible starting point with a compressed model and short conversation; check the exact file size.
Apple unified memory
A possible starting point with a compressed model and short conversation; check the exact file size.

Model facts that change the decision

Parameters
8.03B
Input
Text · Image
Published context limit
4,096 tokens
Model format
GGUF · Ollama · llama.cpp
Suggested compression
Q4_K_M / int4 for local vision chat; higher quantizations were not specified here from the cited file listing.
Typical Q4 file
4.9GB before runtime and context overhead
Languages
English

Using image input

This model needs an app or library that supports its image-processing components. Follow the exact model’s instructions for preparing image inputs. Some GGUF downloads also need a matching projector file; loading the text model alone does not establish image support.

First setup

Ollama

ollama run llava-llama3:8b

LM Studio: Use the XTuner GGUF and ensure the interface associates the LLaVA mmproj file with the model.

llama.cpp

Use the XTuner llama.cpp example pattern: load the int4/F16 GGUF with --mmproj pointing to the repository's mmproj F16 file and use a 4096-token context.

Conditions that change the answer

  • Q5_K_M and Q8_0 file sizes were not specified here from the cited file listing, so those fields are null.
  • The model inherits Meta Llama 3 licensing and LLaVA/XTuner component constraints.
  • RAM and VRAM estimates are derived from GGUF/Ollama package size plus conservative overhead; conversation memory, image resolution, batch size, GPU offload and runtime support can materially change memory needs.

Read the controlling license text →