What hardware runs DeepSeek R1 Distill Qwen 32B?
DeepSeek R1 Distill Qwen 32B is a 32.76B-parameter dense model (mit license). The largest R1 distill that fits 24 GB. Strongest local math-and-logic per dollar of hardware.
All figures assume an f16 KV cache, a 0.6 GB display reserve on the GPU, and 64 GB of DDR5 system RAM for the offload tiers. Tune these in the calculator.
Minimum VRAM by quant and context
| Quant | File size | 8K | 16K | 32K | 64K | 128K |
|---|---|---|---|---|---|---|
| Q2_K * | 11.5 GB | 15.3 GB | 17.4 GB | 21.5 GB | 29.8 GB | 46.3 GB |
| Q3_K_M * | 14.8 GB | 18.7 GB | 20.8 GB | 24.9 GB | 33.1 GB | 49.6 GB |
| Q4_K_M | 18.5 GB | 22.4 GB | 24.4 GB | 28.5 GB | 36.8 GB | 53.3 GB |
| Q5_K_M | 21.7 GB | 25.5 GB | 27.6 GB | 31.7 GB | 40 GB | 56.5 GB |
| Q6_K | 25.0 GB | 28.9 GB | 31 GB | 35.1 GB | 43.3 GB | 59.8 GB |
| Q8_0 | 32.4 GB | 36.3 GB | 38.4 GB | 42.5 GB | 50.7 GB | 67.2 GB |
Full-GPU figures: weights + f16 KV cache + overhead. * below our recommended floor of Q4_K_M.
GPU compatibility
| GPU | 8K | 16K | 32K | 64K | 128K |
|---|---|---|---|---|---|
| NVIDIA GeForce RTX 5090 | Q6_K | Q6_K | Q5_K_M | Q4_K_M | Q4_K_M |
| NVIDIA GeForce RTX 5080 | Q4_K_M | Q4_K_M | Q4_K_M | Q8_0 | Q4_K_M |
| NVIDIA GeForce RTX 5070 Ti | Q4_K_M | Q4_K_M | Q4_K_M | Q8_0 | Q4_K_M |
| NVIDIA GeForce RTX 5070 | Q4_K_M | Q4_K_M | Q4_K_M | Q8_0 | Q4_K_M |
| NVIDIA GeForce RTX 5060 Ti 16GB | Q4_K_M | Q4_K_M | Q4_K_M | Q8_0 | Q4_K_M |
| NVIDIA GeForce RTX 5060 | Q4_K_M | Q4_K_M | Q8_0 | Q8_0 | Q4_K_M |
| NVIDIA GeForce RTX 4090 | Q4_K_M | Q4_K_M | Q4_K_M | Q4_K_M | Q4_K_M |
| NVIDIA GeForce RTX 4080 SUPER | Q4_K_M | Q4_K_M | Q4_K_M | Q8_0 | Q4_K_M |
| NVIDIA GeForce RTX 4070 Ti SUPER | Q4_K_M | Q4_K_M | Q4_K_M | Q8_0 | Q4_K_M |
| NVIDIA GeForce RTX 4070 | Q4_K_M | Q4_K_M | Q4_K_M | Q8_0 | Q4_K_M |
| NVIDIA GeForce RTX 4060 Ti 16GB | Q4_K_M | Q4_K_M | Q4_K_M | Q8_0 | Q4_K_M |
| NVIDIA GeForce RTX 4060 | Q4_K_M | Q4_K_M | Q8_0 | Q8_0 | Q4_K_M |
| NVIDIA GeForce RTX 3090 | Q4_K_M | Q4_K_M | Q4_K_M | Q4_K_M | Q4_K_M |
| NVIDIA GeForce RTX 3080 10GB | Q4_K_M | Q4_K_M | Q8_0 | Q8_0 | Q4_K_M |
| NVIDIA GeForce RTX 3070 | Q4_K_M | Q4_K_M | Q8_0 | Q8_0 | Q4_K_M |
| NVIDIA GeForce RTX 3060 Ti | Q4_K_M | Q4_K_M | Q8_0 | Q8_0 | Q4_K_M |
| NVIDIA GeForce RTX 3060 12GB | Q4_K_M | Q4_K_M | Q4_K_M | Q8_0 | Q4_K_M |
| NVIDIA GeForce RTX 2080 Ti | Q4_K_M | Q4_K_M | Q4_K_M | Q8_0 | Q4_K_M |
| NVIDIA GeForce GTX 1080 Ti | Q4_K_M | Q4_K_M | Q4_K_M | Q8_0 | Q4_K_M |
| AMD Radeon RX 9070 XT | Q4_K_M | Q4_K_M | Q4_K_M | Q8_0 | Q4_K_M |
| AMD Radeon RX 7900 XTX | Q4_K_M | Q4_K_M | Q4_K_M | Q4_K_M | Q4_K_M |
| AMD Radeon RX 7900 XT | Q4_K_M | Q4_K_M | Q4_K_M | Q4_K_M | Q4_K_M |
| AMD Radeon RX 7800 XT | Q4_K_M | Q4_K_M | Q4_K_M | Q8_0 | Q4_K_M |
| AMD Radeon RX 7600 XT | Q4_K_M | Q4_K_M | Q4_K_M | Q8_0 | Q4_K_M |
| AMD Radeon RX 6800 XT | Q4_K_M | Q4_K_M | Q4_K_M | Q8_0 | Q4_K_M |
| AMD Radeon RX 6700 XT | Q4_K_M | Q4_K_M | Q4_K_M | Q8_0 | Q4_K_M |
| Intel Arc B580 | Q4_K_M | Q4_K_M | Q4_K_M | Q8_0 | Q4_K_M |
| Intel Arc A770 16GB | Q4_K_M | Q4_K_M | Q4_K_M | Q8_0 | Q4_K_M |
| Apple M2 (16GB) | Q4_K_M | Q4_K_M | Q4_K_M | Q8_0 | Q4_K_M |
| Apple M2 Pro (32GB) | Q4_K_M | Q4_K_M | Q4_K_M | Q4_K_M | Q4_K_M |
| Apple M2 Max (64GB) | Q8_0 | Q8_0 | Q8_0 | Q6_K | Q4_K_M |
| Apple M2 Ultra (128GB) | Q8_0 | Q8_0 | Q8_0 | Q8_0 | Q8_0 |
| Apple M3 (16GB) | Q4_K_M | Q4_K_M | Q4_K_M | Q8_0 | Q4_K_M |
| Apple M3 Pro (36GB) | Q5_K_M | Q4_K_M | Q4_K_M | Q4_K_M | Q4_K_M |
| Apple M3 Max (64GB) | Q8_0 | Q8_0 | Q8_0 | Q6_K | Q4_K_M |
| Apple M3 Ultra (96GB) | Q8_0 | Q8_0 | Q8_0 | Q8_0 | Q8_0 |
| Apple M4 (16GB) | Q4_K_M | Q4_K_M | Q4_K_M | Q8_0 | Q4_K_M |
| Apple M4 Pro (48GB) | Q6_K | Q6_K | Q6_K | Q4_K_M | Q4_K_M |
| Apple M4 Max (64GB) | Q8_0 | Q8_0 | Q8_0 | Q6_K | Q4_K_M |
| CPU only / integrated graphics | Q8_0 | Q8_0 | Q8_0 | Q8_0 | Q4_K_M |
Fits on GPUExpert offloadPartial offloadCPU only
Quant guidance
Our floor for DeepSeek R1 Distill Qwen 32B is Q4_K_M — below that, quality degrades faster than the VRAM savings are worth. Prefer the highest quant that still lands "Fits on GPU" at your context length in the table above.
Recommended run command
Q4_K_M at 32K on a NVIDIA GeForce RTX 5090-class GPU (Fits on GPU):
llama-server -m DeepSeek-R1-Distill-Qwen-32B-Q4_K_M.gguf -c 32768 --flash-attn -ngl 99
Frequently asked questions
How much VRAM does DeepSeek R1 Distill Qwen 32B need?
At Q4_K_M and 32K context, DeepSeek R1 Distill Qwen 32B needs about 28.5 GB of VRAM to run fully on GPU (weights + KV cache + overhead).
What is the smallest GPU that can run DeepSeek R1 Distill Qwen 32B?
The NVIDIA GeForce RTX 5090 (32 GB) is the smallest GPU in our set that runs DeepSeek R1 Distill Qwen 32B well at 32K context.
What quantization should I use for DeepSeek R1 Distill Qwen 32B?
We recommend Q4_K_M or higher. Q4_K_M weighs 18.5 GB (4.85 bits/weight); going below Q4_K_M costs noticeable quality on a model this size.
How long a context can DeepSeek R1 Distill Qwen 32B handle?
DeepSeek R1 Distill Qwen 32B supports up to 128K tokens. KV cache grows linearly with context.
Can I run DeepSeek R1 Distill Qwen 32B without a GPU?
Yes, at reduced speed: on CPU with 64 GB of DDR5 it manages roughly 1 tokens/sec at 8K context (Q8_0).