Qwen3.8 4B Distill — the last word in low-spec local agents

Conclusion

Among local AI models you can run in an 8GB VRAM environment, Qwen3.8-4B-Distill is the most sensible choice right now. Distilled by Empero from the Qwen3.8 2.4T A95B teacher model, it scores 55.3% on MMLU while pulling 55 tok/s. It trails the 9B by under 5%, yet uses half the VRAM and runs twice as fast.

Empero Qwen3.8-4B-Distill in detail

A full-parameter distilled model developed by Empero, an independent German AI lab.

The distilled model learned directly from the teacher's real reasoning traces, not synthetic data. Every answer begins with a <think> block, and this reasoning pattern is drawn from the actual thinking process of Qwen3.8 2.4T.

Benchmarks — a slight dip in math, a big jump in general knowledge

Math reasoning dipped slightly, but general knowledge and reasoning (MMLU) improved by 19.9 percentage points. Agent tasks depend more on general knowledge and reasoning than on math, so this change is felt strongly in practice.

Why the 4B replaces the 9B

On an 8GB card, the 9B eats 5-6GB even after Q4 quantization. The 4B has headroom. Token speed differs by 2x, and when an agent works multi-step, that gap is large.

Korean — hands-on comparison with Gemma 4

Gemma 4 12B has natural Korean. But there is a problem. Its interjections are set in stone. Expressions like "Wow, that's really amazing!" keep coming out, and over a long conversation they grate. No interjections at all would be better.

Qwen is different. Its Korean responses are clean and it uses little unnecessary decoration. In particular, it follows system instructions precisely, so when used for an agent it follows skills and rules almost 90% of the time or better. Gemma 4 is smoother in Korean but strays from the rules more often.

Hands-on environment — RTX 8GB, Ollama

Measured on the operator's environment (RTX 8GB, Ollama, Q4_K_M):

Agent delegation strategy — local 4B + a premium API model

A low-spec model's limits are clear. How to handle an infinite loop, and unexpected error handling, are beyond a local 4B. But if you delegate that part to a paid API model, nothing is lacking for personal use.

The approach is this:

1. All schemas, rules, and skills are handled by the local 4B (zero injection cost)
2. Basic conversation is handled by the local 4B (when it is not specialized coding work)
3. Only complex reasoning and error handling are delegated to the API model

This way:

• No information leaves the machine (local processing)
• Only what is needed is delegated, so only the fragments cut off from the server go to the server
• Nothing can be inferred from context, so it is safe
• Token usage drops sharply (zero injection cost + minimal delegation)

Installation

# Install directly from Ollama
ollama pull qwen3.8-4b-distill

# Or from HuggingFace
huggingface-cli download Empero/Qwen3.8-4B-Distill

Sampling parameters: temperature=0.6, top_p=0.95, top_k=20 are recommended. Since there are <think> blocks, a large max_new_tokens is advisable (e.g., 16,384).

Conclusion

If you have an 8GB VRAM environment, Qwen3.8-4B-Distill is the best choice. MMLU 55.3%, 55 tok/s, Korean rule-following above 90%. It is only 5% behind the 9B, uses half the VRAM, and runs twice as fast. Apache 2.0 licensed with free commercial use, and it installs directly from Ollama.

Agent skill-injection accuracy above 90%. For an agent operator trying to solve everything on low-spec local hardware, there is no model like it.

Sources:

• Empero AI — empero.org (Qwen3.8 Distilled)
• Qwen3.5 official repo (QwenLM/Qwen3)
• Ollama official library
• MMLU/gsm8k benchmarks (Empero public data)