Begin with a model that fits and a coding task you can check. Qwen2.5-Coder 7B Instruct is a sensible code-focused candidate; Qwen3 4B is a smaller general-purpose candidate when memory is limited. Neither is a universal winner, and a model that explains a function may still struggle with a whole repository or an autonomous coding agent.
This guide suits people who want explanations, small edits, tests, or help understanding error messages on their own computer. For an app that can edit files and run commands, use the local coding agents guide after the direct model test succeeds.
Pick the task before the model
| What you want | What matters most |
|---|---|
| Explain a short function | Clear instructions and enough room for the function |
| Write or repair one function | Code-focused training and tests you can run |
| Suggest completions in an editor | The extension's completion format and support for the model |
| Edit a repository through an agent | Tool use, reliable edits, larger context, and the host app's model requirements |
| Discuss a new library release | Current documentation supplied in the prompt or through an approved tool |
An Instruct model is intended to follow conversational instructions. A completion model and a model used for tool calls may need different prompt formats. Do not substitute a similarly named base checkpoint and expect the same behaviour.
Start with a specific download
On a computer with enough available RAM or VRAM, inspect Qwen2.5-Coder 7B Instruct. For LM Studio, the official Qwen/Qwen2.5-Coder-7B-Instruct-GGUF repository provides quantized files. Begin by assessing Q4_K_M in the memory calculator, with a modest context; Q5 or Q8 needs more memory.
For Ollama, the packaged starting command is:
ollama run qwen2.5-coder:7b
The first run can download the model. This Ollama tag and a GGUF file from Hugging Face are different downloads, even when they originate from the same checkpoint. Record which one you used when comparing answers.
If the 7B model does not leave usable memory, try Qwen3 4B at a supported 4-bit setting. Qwen's cards identify the original Qwen3 4B and Qwen2.5-Coder 7B Instruct under Apache 2.0; follow the exact checkpoint and download's license. A permissive license does not establish that generated code is correct or suitable for your project.
Run a small test that can fail
In a fresh chat, supply a function and a concrete requirement. For example:
Write a Python function
unique_in_order(values)that removes duplicate strings while keeping the first occurrence. It must preserve case, return an empty list for an empty input, and leave the input unchanged. Include three tests. Explain any assumptions.
Read the answer before running it. Use a scratch folder, then run the tests yourself. Add an input the model did not choose, such as ['A', 'a', 'A']. The expected output is ['A', 'a']. Check that the function does not mutate the original list.
A useful result is code that meets the stated requirements and passes the additional case. Plausible prose, a confident explanation, and tests written by the same model are insufficient on their own.
Next try a small function from your project with secrets removed. Supply the language, library version, relevant surrounding code, and the exact error. Ask for a minimal change and the reason it helps. Review the diff and run your project's relevant checks.
Know the common failure modes
- Invented library calls: supply the current API documentation. A larger model does not automatically know a recent release.
- Wrong edit format or ignored tools: check the editor or agent's required model capability. Ordinary chat support does not establish agent support.
- Lost requirements: shorten the prompt, include the relevant files, and check context length.
- Very slow replies: reduce model size or context and inspect whether the model is actually using the GPU.
- Unwanted reasoning text: use the app's supported reasoning controls for that exact model rather than deleting arbitrary text from the answer.
Stop an agent if it repeatedly makes unrelated edits or runs unexpected commands. Improve the task and inspect the changes before granting it more access. Code running locally can still read files, contact the internet, or cause damage if the surrounding app gives it those abilities.