Skip to main content
Local LLM Guide

Model

Aya 23 8B

Start with a supported compressed build and short context. 16GB RAM estimate. Best use: Noncommercial multilingual local chat

8BText chat

Best use

Noncommercial multilingual local chat · Translation-style assistance across the source-listed 23-language set

Memory planning

16GB RAM estimate

Expected pace

Depends on your hardware, app, and conversation length

License

CC-BY-NC-4.0 with Cohere Labs Acceptable Use Policy; commercial use is restricted

Who should use this model?

  • Noncommercial multilingual local chat
  • Translation-style assistance across the source-listed 23-language set
  • Multilingual summarization
  • Multilingual evaluation and research
  • Language coverage demos

Choose something else when

  • You need commercial use rights
  • You cannot accept gated HF model terms
  • You need tool calling
  • You need vision
  • You need the newer Aya Expanse or Aya Vision family

What hardware makes sense?

The download is only part of the memory needed. Leave room for the app, the input it processes, and other open programs. For chat models, context includes the instructions, conversation, and answer. Longer context needs more memory.

CPU only
A possible starting point with a compressed model and short conversation; check the exact file size.
8GB system memory
Memory or response time may limit use. Try a smaller model first.
16GB system memory
A possible starting point with a compressed model and short conversation; check the exact file size.
32GB system memory
A possible starting point with a compressed model and short conversation; check the exact file size.
8GB GPU memory
Memory or response time may limit use. Try a smaller model first.
12GB GPU memory
A possible starting point with a compressed model and short conversation; check the exact file size.
24GB GPU memory
A possible starting point with a compressed model and short conversation; check the exact file size.
Apple unified memory
A possible starting point with a compressed model and short conversation; check the exact file size.

Model facts that change the decision

Parameters
8B
Input
Text
Published context limit
8,192 tokens
Model format
Safetensors · GGUF · Ollama
Suggested compression
Q4_K_M for local multilingual chat and summarization; Q5_K_M if memory allows.
Typical Q4 file
5.05GB before runtime and context overhead
Languages
Arabic · Chinese simplified and traditional · Czech · Dutch · English · French · German · Greek

First setup

Ollama

ollama run aya:8b-23

LM Studio: Use a community Aya 23 8B GGUF such as the bartowski conversion and select Q4_K_M or Q5_K_M.

llama.cpp

Use a standard text GGUF loader against bartowski/aya-23-8B-GGUF, subject to the original model license terms.

Conditions that change the answer

  • Aya 23 8B is gated on Hugging Face and governed by CC-BY-NC-4.0 plus Cohere Labs Acceptable Use Policy, so it is not suitable for commercial use.
  • The cited GGUF is a community conversion.
  • The model card identifies Aya 23 as an older version relative to Aya Expanse/Aya Vision, but Aya 23 8B remains a clear local text-only multilingual target.
  • RAM and VRAM estimates are derived from GGUF/Ollama package size plus conservative overhead; conversation memory, context length, batch size, runtime and GPU offload choices can materially change memory needs.

Read the controlling license text →