---
title: "A Complete Guide to Local AI Model Recommendations by Graphics Card (2026)"
date: 2026-09-23
time: "22:30"
model: "deepseek-v3"
category: knowhow
summary: "A VRAM-by-VRAM and model-by-model benchmark of which local AI models you can run on the graphics card you own. Recommended models, inference speed (TPS), and value rankings for 18 GPUs from the RTX 3060 to the RTX 5090."
tags: GPU, RTX 3090, RTX 4090, RTX 5090, VRAM, inference speed, local AI, Ollama, llama.cpp, quantization, value
---

# A Complete Guide to Local AI Model Recommendations by Graphics Card

Which models can I run on my graphics card? Here is the answer to that question.

---

## 1. The Core Principle: VRAM Is the Model Ceiling

```
Runnable model = GPU VRAM ≥ model file size + KV cache headroom
```

| Quantization | 8B model | 14B model | 32B model | 70B model |
|--------|---------|----------|----------|----------|
| **Q4_K_M** | ~4.9 GB | ~9.0 GB | ~19.5 GB | ~43.0 GB |
| **Q8_0** | ~8.5 GB | ~16.4 GB | ~37.0 GB | ~75.0 GB |

**If VRAM is smaller than the model size, it simply cannot run.** No matter how fast the memory bandwidth is, there is no way around it.

---

## 2. NVIDIA GPU Spec Comparison

### RTX 30 Series (Ampere)

| GPU | VRAM | Memory bandwidth | Note |
|-----|------|-------------|------|
| RTX 3060 12GB | **12GB** | 360 GB/s | Ample VRAM, low bandwidth |
| RTX 3060 Ti 8GB | 8GB | 448 GB/s | Insufficient VRAM |
| RTX 3070 8GB | 8GB | 448 GB/s | Insufficient VRAM |
| RTX 3080 10GB | 10GB | 760 GB/s | Good bandwidth |
| RTX 3080 Ti 12GB | 12GB | 912 GB/s | Excellent bandwidth |
| RTX 3090 24GB | **24GB** | **936 GB/s** | **The best VRAM value** |

### RTX 40 Series (Ada Lovelace)

| GPU | VRAM | Memory bandwidth | Note |
|-----|------|-------------|------|
| RTX 4060 8GB | 8GB | 272 GB/s | Entry-level |
| RTX 4060 Ti 16GB | **16GB** | 288 GB/s | Ample VRAM, low bandwidth |
| RTX 4070 12GB | 12GB | 504 GB/s | Balanced |
| RTX 4070 Super 12GB | 12GB | 504 GB/s | Good value |
| RTX 4070 Ti 12GB | 12GB | 504 GB/s | |
| RTX 4070 Ti Super 16GB | **16GB** | **672 GB/s** | **The 16GB sweet spot** |
| RTX 4080 16GB | 16GB | 717 GB/s | High performance |
| RTX 4090 24GB | **24GB** | **1,008 GB/s** | **The consumer flagship** |

### RTX 50 Series (Blackwell) — With GDDR7

| GPU | VRAM | Memory bandwidth | Note |
|-----|------|-------------|------|
| RTX 5060 Ti 16GB | **16GB** | 448 GB/s | **The new value king** |
| RTX 5070 12GB | 12GB | 672 GB/s | |
| RTX 5070 Ti 16GB | **16GB** | **896 GB/s** | 16GB high speed |
| RTX 5080 16GB | 16GB | 960 GB/s | |
| RTX 5090 32GB | **32GB** | **1,792 GB/s** | **King of kings** |

---

## 3. Recommended Models by GPU

### 🟢 RTX 3060 12GB (used ~150,000-200,000 KRW)

With 12GB VRAM, 8B models run fully.

| Recommended model | Quantization | File size | Expected TPS |
|----------|--------|----------|---------|
| **Qwen3-4B** | Q4_K_M | 2.3 GB | ~50 |
| **Gemma 4 E4B** | Q4_K_M | 2.5 GB | ~48 |
| **K2-Horizon-3.7B** | Q4_K_M | 2.3 GB | ~50 |
| **Llama 3.1 8B** | Q4_K_M | 4.9 GB | **42** |
| **Qwen3 8B** | Q4_K_M | 4.9 GB | 40 |
| Qwen 2.5 14B | Q4_K_M | 9.0 GB | 22 (no headroom) |

**In a word:** "For 150,000-200,000 KRW, 8B models comfortably, 14B tightly"

---

### 🔵 RTX 3090 24GB (used ~600,000-700,000 KRW) — 🏆 The best value

With 24GB VRAM, 32B models run fully and 70B partly.

| Recommended model | Quantization | File size | Expected TPS |
|----------|--------|----------|---------|
| **Qwen3-4B** | Q4_K_M | 2.3 GB | ~120 |
| **Llama 3.1 8B** | Q4_K_M | 4.9 GB | **95** |
| **Qwen3 8B** | Q8_0 | 8.5 GB | ~65 |
| **Qwen 2.5 14B** | Q4_K_M | 9.0 GB | **55** |
| **Gemma 4 12B** | Q4_K_M | 7.5 GB | ~60 |
| **Qwen3 32B** | Q4_K_M | 19.5 GB | **28** |
| **Gemma 4 28B (MoE)** | Q4_K_M | 17.0 GB | ~35 |
| Llama 3.3 70B | Q2_K | ~30 GB | 10 (partial offload) |

**In a word:** "For 600,000-700,000 KRW, the king that fully runs 32B models"

---

### 🔷 RTX 4070 Ti Super 16GB (~1,150,000 KRW)

16GB + 672 GB/s is optimal for 14B models.

| Recommended model | Quantization | File size | Expected TPS |
|----------|--------|----------|---------|
| **Qwen3-4B** | Q4_K_M | 2.3 GB | ~100 |
| **Llama 3.1 8B** | Q4_K_M | 4.9 GB | **72** |
| **Qwen 2.5 14B** | Q4_K_M | 9.0 GB | **47** |
| **Gemma 4 12B** | Q4_K_M | 7.5 GB | ~55 |
| Qwen3 32B | Q4_K_M | 19.5 GB | ❌ (VRAM exceeded) |

**In a word:** "For 1,150,000 KRW, the sweet spot for 14B models"

---

### ⭐ RTX 4060 Ti 16GB (~490,000 KRW)

16GB, but the bandwidth is low at 288 GB/s. It wins by capacity alone.

| Recommended model | Quantization | File size | Expected TPS |
|----------|--------|----------|---------|
| **Qwen3-4B** | Q4_K_M | 2.3 GB | ~55 |
| **Llama 3.1 8B** | Q4_K_M | 4.9 GB | **34** |
| **Qwen 2.5 14B** | Q4_K_M | 9.0 GB | **22** |
| **Gemma 4 12B** | Q4_K_M | 7.5 GB | ~28 |

**In a word:** "For 490,000 KRW, 16GB — 14B is slow but it runs"

---

### 🏆 RTX 4090 24GB (~3,150,000 KRW)

The fastest consumer GPU plus 24GB VRAM.

| Recommended model | Quantization | File size | Expected TPS |
|----------|--------|----------|---------|
| **Qwen3-4B** | Q4_K_M | 2.3 GB | ~200 |
| **Llama 3.1 8B** | Q4_K_M | 4.9 GB | **135** |
| **Qwen 2.5 14B** | Q4_K_M | 9.0 GB | **78** |
| **Qwen3 32B** | Q4_K_M | 19.5 GB | **42** |
| **Gemma 4 28B (MoE)** | Q4_K_M | 17.0 GB | ~55 |
| Llama 3.3 70B | Q4_K_M | 43 GB | 18 (partial offload) |

**In a word:** "For 3,150,000 KRW, 8B at 135 TPS, 32B at 42 TPS — you never wait"

---

### 👑 RTX 5090 32GB (~7,400,000 KRW)

GDDR7's 1,792 GB/s bandwidth plus 32GB VRAM. The final word in local AI.

| Recommended model | Quantization | File size | Expected TPS |
|----------|--------|----------|---------|
| **Qwen3-4B** | Q4_K_M | 2.3 GB | ~280 |
| **Llama 3.1 8B** | Q4_K_M | 4.9 GB | **145** |
| **Qwen 2.5 14B** | Q4_K_M | 9.0 GB | **103** |
| **Qwen3 32B** | Q4_K_M | 19.5 GB | **142** |
| **Llama 3.3 70B** | Q4_K_M | 43 GB | **25~30** |

**In a word:** "For 7,400,000 KRW, running a 32B model at 142 TPS makes chat feel instant"

---

### 💰 RTX 5060 Ti 16GB (~930,000 KRW) — The new value king

GDDR7 at 448 GB/s. 55% faster bandwidth than the 4060 Ti 16GB.

| Recommended model | Quantization | File size | Expected TPS |
|----------|--------|----------|---------|
| **Qwen3-4B** | Q4_K_M | 2.3 GB | ~80 |
| **Llama 3.1 8B** | Q4_K_M | 4.9 GB | **51** |
| **Qwen 2.5 14B** | Q4_K_M | 9.0 GB | **33** |
| **Gemma 4 12B** | Q4_K_M | 7.5 GB | ~40 |

**In a word:** "For 930,000 KRW, 8B at 51 TPS — 4070-class performance"

---

## 4. VRAM at a Glance

| VRAM | Runnable models (Q4_K_M) | Recommended GPU |
|------|------------------------|---------|
| **8GB** | 4B-8B | RTX 3060 Ti, 3070, 4060 |
| **10GB** | 8B + ample context | RTX 3080 |
| **12GB** | 8B-14B | RTX 3060, 4070, 4070 Super |
| **16GB** | 14B fully, 30B at Q3 | RTX 4060 Ti 16GB, 4070 Ti Super, 5060 Ti |
| **24GB** | 32B fully, 70B partly | RTX 3090, 4090 |
| **32GB** | 32B fully with headroom, 70B borderline | RTX 5090 |

---

## 5. Value Rankings (2026)

| Rank | GPU | Budget | Why recommended |
|------|-----|------|----------|
| 🥇 | **RTX 3090 24GB (used)** | ~600,000-700,000 KRW | Fully runs 32B on 24GB, the best value |
| 🥈 | **RTX 5060 Ti 16GB** | ~930,000 KRW | GDDR7 448 GB/s, 8B at 51 TPS |
| 🥉 | **RTX 4070 Ti Super 16GB** | ~1,150,000 KRW | 672 GB/s, 14B at 47 TPS |
| 4 | **RTX 3060 12GB (used)** | ~150,000-200,000 KRW | On a tight budget, 8B at 42 TPS |
| 5 | **RTX 5090 32GB** | ~7,400,000 KRW | The strongest performance, 32B at 142 TPS |

### Summary Formula

```
Budget under 200,000 KRW -> RTX 3060 12GB (used) + Llama 3.1 8B
Budget 600,000-700,000 KRW -> RTX 3090 24GB (used) + Qwen3 32B
Budget 900,000-1,200,000 KRW -> RTX 5060 Ti 16GB or RTX 4070 Ti Super 16GB
Budget 3,000,000 KRW+ -> RTX 4090 24GB + Qwen3 32B (42 TPS)
Budget 7,000,000 KRW+ -> RTX 5090 32GB + Qwen3 32B (142 TPS)
```

---

## 6. Frequently Asked Questions

**Q. Can I run a 14B model on an 8GB GPU?**
→ It runs at Q3_K_M quantization (~7.5GB), but the quality drops significantly. Running an 8B model at Q8_0 is better.

**Q. 4060 Ti 16GB vs 3090 24GB, which is better?**
→ The 3090 is 3x faster in bandwidth (936 vs 288 GB/s) and has 8GB more VRAM. At a similar budget when buying used at 600,000-700,000 KRW versus the 4060 Ti 16GB (490,000 KRW), the 3090 wins by a landslide.

**Q. What about connecting two GPUs?**
→ llama.cpp can split with `--tensor-split`, but the PCIe bus bottleneck means 1+1 ≠ 2. The speed is about 60-70%.

**Q. What about AMD GPUs?**
→ ROCm has a narrower support range than NVIDIA. It works via llama.cpp Vulkan, but performance drops 30-50% versus CUDA. NVIDIA is recommended.