---
title: "The Reality of DGX Spark: 128GB of VRAM, So Why Is It Slower Than an RTX 4090?"
date: 2026-09-23
time: "22:10"
model: "deepseek-v3"
category: knowhow
summary: "An analysis of the NVIDIA DGX Spark (128GB unified memory). It breaks the illusion that more VRAM means faster, and explains from a bandwidth standpoint why an RTX 4090 (24GB) is 3x faster on an 8B model. It sorts out the cases where the DGX Spark is genuinely useful (70B+ fine-tuning) and where a GPU is better."
tags: DGX Spark, NVIDIA, GB10, unified memory, VRAM, inference speed, RTX 4090, memory bandwidth, mini supercomputer, local AI
---

# The Reality of DGX Spark: 128GB, So Why Is It Slower Than an RTX 4090?

The DGX Spark is a revolutionary device that can run 100B+ models locally with its 128GB of unified memory, and yet **its inference speed is 3x slower than an RTX 4090**. The key is "bandwidth, not capacity."

---

## 1. What Is the DGX Spark

### Hardware Specs

| Item | DGX Spark | RTX 4090 |
|------|-----------|----------|
| **Launch price** | $3,999 | $1,599 |
| **Processor** | GB10 Grace Blackwell | AD102 |
| **Memory** | 128GB LPDDR5X | 24GB GDDR6X |
| **Bandwidth** | **273 GB/s** | **1,008 GB/s** |
| **CUDA cores** | 6,144 | 16,384 |
| **Tensor cores** | 192 (5th gen) | 512 (4th gen) |
| **Power draw** | 240W (about 100W actual) | 450W |
| **Size** | 150x150x50.5mm | 304x137mm |

### The Magic of Unified Memory

The DGX Spark shares **the same 128GB memory pool** between CPU and GPU via NVLink-C2C. Conventional GPUs have VRAM separately and exchange data over the PCIe bus, but the DGX Spark removes that bottleneck.

Thanks to this, it can **run a 70B model at FP16 (140GB needed) on a single system without quantization**. An RTX 4090 cannot run a 70B model on its own.

### The Lineup

| Product | Price | Feature |
|------|------|------|
| **NVIDIA DGX Spark** | $3,999 | Official model, DGX OS |
| **ASUS Ascent GX10** | ~$3,999 | Most compact (150x150x51mm) |
| **GIGABYTE AI TOP ATOM** | ~$3,999 | Value-focused |
| **Dell Pro Max with GB10** | ~$4,500+ | Enterprise adoption oriented |

All products use the same GB10 chip, and connecting two units gives 256GB, enough for a 405B model.

---

## 2. Why 128GB Is Slower: The Bandwidth Bottleneck

### The Core Formula

```
Token generation speed (TPS) ≈ memory bandwidth (GB/s) ÷ model size (GB)
```

Every token generation requires reading the entire model weights from memory. So **the read speed (bandwidth) is the speed**.

### Real Calculation: 8B Model (Q4_K_M, ~4.9GB)

| Hardware | Bandwidth | Theoretical TPS | Actual TPS |
|----------|--------|---------|---------|
| **RTX 4090** | 1,008 GB/s | ~206 | **116** |
| **RTX 5090** | 1,792 GB/s | ~366 | **140-150** |
| **DGX Spark** | 273 GB/s | ~56 | **35** |
| **Mac Mini M4** | ~100 GB/s | ~20 | **42-52** |

The RTX 4090 is **3.3x faster** than the DGX Spark. A 24GB gaming graphics card outpaces a 128GB AI supercomputer.

### Why This Happens

**Memory bandwidth comparison:**

| Memory type | Bandwidth | Used in |
|-------------|--------|--------|
| LPDDR5X (DGX Spark) | **273 GB/s** | Unified memory |
| GDDR6X (RTX 4090) | **1,008 GB/s** | Consumer GPU |
| GDDR7 (RTX 5090) | **1,792 GB/s** | Consumer GPU |
| HBM3 (H100) | **3,350 GB/s** | Data center |
| HBM3e (B200) | **8,000 GB/s** | Data center |

The LPDDR5X the DGX Spark uses is **memory specialized for power efficiency**. It sacrificed bandwidth to reduce power draw. GDDR6X and HBM, by contrast, focus on performance.

**By analogy:**
- RTX 4090 = a 4-lane highway (narrow but fast)
- DGX Spark = an 8-lane city avenue (wide but slow because of waiting at lights)

---

## 3. Real-World Benchmarks by Model

### Small Models (8B and under): RTX 4090 Wins Decisively

| Model | RTX 4090 | DGX Spark | Difference |
|------|----------|-----------|------|
| Llama 3.1 8B Q4 | **116 tps** | 35 tps | RTX 4090 **3.3x** |
| Qwen3 4B Q4 | **165 tps** | ~50 tps | RTX 4090 **3.3x** |
| Gemma 4 E4B Q4 | **150 tps** | ~45 tps | RTX 4090 **3.3x** |

### Mid-Size Models (13-30B): DGX Spark Catches Up

| Model | RTX 4090 | DGX Spark | Difference |
|------|----------|-----------|------|
| Qwen 2.5 14B Q4 | 45 tps | **75-95 tps** | DGX Spark **1.8x** |
| Llama 3.1 70B Q4 | **Cannot run** | **35-45 tps** | DGX Spark wins |
| Qwen3 32B Q4 | **Cannot run** (VRAM exceeded) | 50-65 tps | DGX Spark wins |

### Large Models (70B+): Only the DGX Spark Can Run Them

| Model | RTX 4090 | DGX Spark | Note |
|------|----------|-----------|------|
| Llama 3.1 70B FP16 | **Impossible** (needs 140GB) | **15-20 tps** | DGX Spark only |
| Llama 3.1 70B Q4 | **Impossible** (needs 35GB) | **35-45 tps** | |
| Qwen3 235B (dual) | **Impossible** | **11.73 tps** | Two units connected |

### NVIDIA Official Benchmark (NVFP4, TRT-LLM)

| Model | Prompt processing | Token generation |
|------|--------------|-----------|
| Llama 3.1 8B NVFP4 | 10,257 tok/s | **38.65 tok/s** |
| Qwen3 14B NVFP4 | 5,929 tok/s | **22.71 tok/s** |
| GPT-OSS-20B MXFP4 | 3,670 tok/s | **28.74 tok/s** |
| GPT-OSS-120B MXFP4 | 1,725 tok/s | **55.37 tok/s** |
| Qwen3 235B (dual) | 23,477 tok/s | **11.73 tok/s** |

---

## 4. Cases Where the DGX Spark Is Genuinely Useful

### DGX Spark Wins

**1. Running 70B+ models**
- An RTX 4090 cannot run a 70B model on its own
- The DGX Spark runs a 70B model at FP16 with no quantization
- It is a "can it run at all?" problem, so bandwidth does not matter

**2. Fine-tuning**
- 128GB of memory is essential when fine-tuning a 70B model with QLoRA
- NVIDIA's official benchmark: 5,079 tok/s when QLoRA fine-tuning a 70B model

**3. Large context + large model**
- 70B model + 128K context = 16-30GB consumed by the KV cache alone
- Handled comfortably on the 128GB unified memory

**4. Fully air-gapped security**
- Runs a large model locally with no cloud at 128GB
- Ideal for sensitive data in finance, healthcare, and defense

**5. Multimodal workloads**
- Because CPU and GPU share the same memory, no copy is needed for image/video processing

### RTX 4090/5090 Wins

**1. Small models of 8B and under**
- The RTX 4090 is **3.3x faster** (116 vs 35 tps)
- An 8B model fits comfortably within 24GB of VRAM

**2. Work that needs fast token generation**
- Chatbots, coding assistants, and anything where response speed matters
- The RTX 5090 at **1,792 GB/s** is **6.6x faster** than the DGX Spark

**3. Cost efficiency**
- RTX 4090 $1,599 vs DGX Spark $3,999
- For 8B models: 2.5x cheaper and 3.3x faster

**4. Gaming plus AI**
- The RTX 4090 can do gaming and AI together
- The DGX Spark is AI-only (ARM Linux)

---

## 5. Which Device Should You Choose

### Decision Table

| Situation | Recommended device | Reason |
|------|----------|------|
| **Local AI at 8B or under** | RTX 4090/5090 | 3x faster and 2.5x cheaper |
| **Running a 70B model** | DGX Spark | The only option |
| **70B fine-tuning** | DGX Spark | 128GB of memory required |
| **Budget under 1,000,000 KRW** | Mac Mini M4 24GB | 8B at 42-52 TPS |
| **Budget 1,800,000 KRW** | Mac Mini M4 Pro 48GB | 32B at 12-18 TPS |
| **Budget 4,000,000 KRW+** | DGX Spark | 70B+ at FP16 |

### The Mathematical Criterion

```
Model size (GB) > VRAM (GB) -> DGX Spark wins (it is about whether it can run at all)
Model size (GB) < VRAM (GB) -> RTX 4090 wins (bandwidth decides the speed)
```

---

## 6. Conclusion

The DGX Spark's **128GB unified memory is a "wide road,"** and the RTX 4090's **1,008 GB/s GDDR6X is a "highway."**

- That it **can run a 70B model at 128GB** is revolutionary
- But **for an 8B model, the RTX 4090 is 3x faster**
- The illusion that "more VRAM is faster" overlooks the bottleneck of **memory bandwidth**

**Key lesson:** Inference speed is decided not by VRAM capacity but by **memory bandwidth**. The DGX Spark is "a device that can run larger models," not "a faster device."