The Case Against "Linux Wins at Local LLMs" — What the Measured Numbers Say About Where It Actually Matters

These days, "Linux is faster for LLMs" gets repeated almost like common sense. The assumption being that if you load the same model on the same GPU, Linux obviously has the edge. Every forum says so, every YouTube title says so.

My take is that the direction is right but the multiplier is oversold. Dig through the published controlled benchmarks and the OS-driven speed difference mostly lands in the 1~3% range, and there are actually cases where Windows wins outright. Where Linux genuinely and decisively wins is not speed — it's the range of what you can do at all.

And while I was trying to verify that claim, I measured one thing on my own PC, and the result flipped one very common story.

First, the counterexample: the controlled test Windows won

The benchmark Code Craft Studio published on 2026-09-17 sits at the center of this debate. The key conclusion is this one line:

The gap runs two to three percentage, sometimes even under 1%. One controlled Ollama test found the opposite. Windows hit 137 tokens per second native. WSL2 hit 124.

The conditions were identical: RTX 4080 + Llama 3.2 3B Q4_K_M + Ollama.

The thing worth noticing here is that WSL2 came in slower than Windows native, and slower than native Linux too. Even if "use Linux and you'll be faster" were true, most of that gain comes from running Linux directly rather than through WSL2. That's a different claim from "use a Linux-family OS and you win."

The same conclusion shows up in llama.cpp's official performance discussion #15013. On an RTX 3060 12GB, WSL2 (Ubuntu 24.04, CUDA 13.3) posted 75.58 tok/s for tg128 (generation speed), while the native 3060 entry posted 72.09 tok/s. WSL2 was actually slightly ahead. Prompt processing (pp512) came in at 2407.93 versus 2407.67 — a 0.01% difference, effectively identical.

Borrowing the poster's own reading of their numbers, the WSL2 penalty is about 1.7% for generation tokens and negligible for prompt processing.

So when does Linux actually win

The fact that controlled tests land in the 2~3% range also means that under specific conditions, gaps of 15~25% can open up. Looking at the cases where that happened, the pattern is clear.

The AMD path where the Windows driver is already heavily tuned. Phoronix compared Windows 11 and Ubuntu 24.04 on the same machine in 2025 — a Ryzen 9 9950X3D with a Radeon RX 9700 XT, running llama.cpp on both. The result was that Linux's Vulkan driver outright beat Windows's Vulkan driver. It's the one case where Linux came out ahead regardless of conditions.

But there's an important twist here. When asked where the scorecard numbers in llama.cpp's Vulkan discussion #10879 came from, llama.cpp contributor netrunnereve replied:

Yeah the scorecard results are on Windows, those drivers are generally much more optimized than the Mesa ones.

On the AMD Vulkan path, that means Windows's existing AMD drivers are more mature than Mesa's. So for AMD users, "Linux supports AMD well" holds up, but "Linux is faster on AMD" didn't hold up at first. Phoronix flipped that result later by moving to newer hardware — the software stack wasn't ahead from the start.

The gap widens on large models. In a Lubuntu 26.04 versus Windows 11 comparison (RTX 5080 16GB GDDR7 + i9-14900KF, llama.cpp b4280c, Vulkan and CUDA 12.4, Llama 3.1 70B Q4_K_M), the separation becomes clear.

A 2% gap at 3B opens to 15~25% at 70B, and at the minimum values, a 26% difference. The right way to read this is that the OS layer's overhead grows in absolute terms as the model gets bigger.

Put together: on small models (around 3B), the OS barely matters, and on large models (around 70B), Linux starts to actually pull ahead. That's where the evidence for advising someone to "migrate to Linux" comes from.

The VRAM numbers I saw on my own PC

Here's one measurement I took myself. The claim you often see in older posts is that "a Linux desktop uses under 100MB of VRAM." In my environment, that was not true.

jw-ms7c94 — Ubuntu 25.10, KDE Plasma 6, Xorg, RTX 3070 8GB, measured with no model loaded.

$ nvidia-smi --query-gpu=memory.used,memory.total --format=csv
memory.used [MiB], memory.total [MiB]
1163 MiB, 8192 MiB

$ nvidia-smi --query-compute-apps=pid,used_memory --format=csv
(zero compute processes)

Idle, it sits at 1163 MiB — 14.2% of the total. That's with zero compute processes: purely what Xorg and KDE are holding.

This matters because it ties directly into the DWM argument. There's a widely repeated premise that Windows's DWM holds 1~2GB. My Linux desktop holds 1.1GB. Compare a Windows desktop to a Linux desktop and the difference is on the order of a few hundred MB, not a 1GB blowout. Marketing-style claims like "Linux saves you 2GB" are genuinely oversold for a real desktop environment, as opposed to a headless server.

One more angle. The Linux VRAM advantage Benchmarkai observed in its 2026 16GB card test was roughly 800MB under equivalent load. Same order of magnitude as my 1.1GB reading.

"Linux wins whether you're headless or on a desktop" is the same logic either way, which is why misreading headless figures as desktop figures is a fairly common mistake.

Where Linux genuinely wins: not speed, but range

That was the speed story. And the speed gap is only 1~3%. That's why it can't carry the argument for a "hardware upgrade." The real reason to move to Linux is that things you simply couldn't do become possible.

First, vLLM does not officially support Windows. The requirements section of vLLM's official AMD install docs reads:

OS: Linux

And the fork repository's description is more precise still:

Upstream vLLM supports AMD ROCm, but its official GPU requirements list Linux and explicitly state that vLLM does not support Windows natively.

Windows ROCm forks have appeared, but all of them state that they're "unvalidated." If you want to build an API server whose whole purpose is handling concurrent requests, not just a lightweight single-user setup, Linux is the only option on the table.

Second, the gap in that earlier vLLM ROCm benchmark is a kernel problem, not an OS problem. On the same ROCm stack, TPS varies 2.7~4.4x depending on the attention backend. If you're the one handling AMD server hardware and have to touch layers like that, there's plenty of reason to leave Windows from the start.

Third, WSL2 doesn't give memory back. WSL2 issue #4166, opened in June 2019, has stayed open ever since:

I have seen it grow until nearly 100% of my system memory is in use, and it will not release it until I shut down the WSL 2 VM.

Reports are widespread that Vmmem ignores .wslconfig limits, and global settings only take effect on Windows build 19041 and later. On top of that, the same answer keeps coming up on both Microsoft Q&A and the Docker forums: you have to run wsl --shutdown after changing settings. Local LLM work involves holding and growing context constantly, so a memory-release failure leads straight to degraded performance.

The trap AMD users should know about

One more thing from having owned the hardware after measuring it. Ollama probes the GPU only once, at boot. If you've been using the driver, touched the BIOS, or upgraded Ollama while the service was running, the detection result doesn't refresh — it silently falls back to CPU with no error.

There's a well-documented case on ortamarco.me. On an RX 7900 XTX: CPU at 15.89 tok/s versus GPU at 132.48 tok/s — an 8.3x penalty. The only trace in the logs is library=cpu, with no warning.

There's a worse variant on top of that. Ollama updates reinstall the ROCm bundle. Even if you deliberately switched to Vulkan, an update quietly reverts you to ROCm, and nobody warns you. Weeks of accumulated measurements just turn into different numbers.

The same MSDN hex thread still has a report of the Windows Vulkan loader failing to find an AMD GPU (Ollama issue #16677, RX 6650 XT). When the bundled loader can't find the device:

inference falls back to CPU

No warning to the user at all. It even comes with measured figures recovering from 9.6 tok/s to 52 tok/s. On Linux the system loader takes that job instead, so this problem doesn't occur.

Which is where the practical rules come from.

1. If you have a Ryzen integrated GPU, disable it in the BIOS. It costs 0.2 GiB and removes one ambiguity.
2. Restart Ollama every time you touch the driver, the BIOS, or the Ollama version. This is not optional — it only probes once.
3. Check library= in the logs, not compute=.
4. Don't trust any measurement until ollama ps says 100% GPU.
5. For context length, use the actual value read from /api/ps, not the value you sent.
6. Judge memory by system VRAM, not the ollama ps column.
7. Don't run it inside WSL. Point Windows Ollama at localhost:11434 instead.
8. Close Chrome, Spotify, and Slack before measuring. For tok/s you need quiet; for reach, you need much less.

Worth knowing. Item 8 lines up exactly with the earlier llama.cpp discussion #15013 measurements. That poster got tg128 at 67.27 tok/s on a normal desktop, and after closing all host GPU-consuming processes and re-measuring, got 75.58 tok/s — +12% — with the variance tightening roughly 15x. Not a dramatic number, but it means the noise floor is wider than the gap. A 2% difference can vanish entirely depending on measurement conditions.

Two common misconceptions

"llama.cpp build settings matter more than the OS." There's a real case where Linux beat Windows by over 19%. It's documented on ianlpaterson.com. An RTX 2060 SUPER was delivering 52.8 tok/s, and it turned out the binary had been compiled for an older Maxwell card — the driver was translating PTX at runtime. Rebuilding for the native architecture:

Generation rose about 19%, and prompt processing more than tripled. That means there's more on the table there than an OS change would get you.

"What's the fastest server?" Mozilla AI compared llama.cpp, llamafile, LM Studio, and Ollama across three environments: a Mac Studio M4, a Linux server with an NVIDIA L40S, and a Steam Deck (Vulkan). Prompt processing was broadly close across all four, and generation speed is where they separate — llama.cpp and llamafile lead at 0.8B and 9B, while Ollama led at 27B. At 27B, Ollama had speculative decoding enabled with draft length 2. In other words, the answer to "which server" changes with model size. You can't declare one superior in advance; the answer only shows up once you've picked a model size.

Conclusion

The precise form of "Linux wins" is this:

On speed alone, the OS barely matters for small models (1~3%), and Linux opens a 15~25% gap on large models. That's enough for a genuine reversal to be possible, which is why it's hard to use as grounds for insisting on a migration.

But the reasons to move to Linux are still solid. You get to use vLLM, the non-CUDA path (AMD Vulkan) is mature, and you avoid WSL2's memory-release defect. That's a difference in what's possible, not in speed.

And my own PC measurement took out one of the most widely repeated myths in this whole debate. My Linux desktop's idle VRAM wasn't 100MB — it was 1163 MiB. Using a headless server's advantage as evidence for a desktop environment is a misreading. If you want to be promised VRAM headroom, you have to make a separate decision to go headless, and that's a different tradeoff.

If you're considering Linux going forward, check two things before you change the OS. Does the model fully fit on the GPU (does it stay at 100% to the end), and does your driver stack actually support that card properly. The second one is the real culprit far more often than the OS is.