dexiio

What hardware runs Qwen3.5 27B?

Qwen3.5 27B is a 26.9B-parameter dense model (apache-2.0 license). The dense 24GB workhorse. If you want one model on a 3090 and no surprises, it's this.

All figures assume an f16 KV cache, a 0.6 GB display reserve on the GPU, and 64 GB of DDR5 system RAM for the offload tiers. Tune these in the calculator.

Minimum VRAM by quant and context

QuantFile size8K16K32K64K128K
Q3_K_M *12.6 GB16.4 GB18.5 GB22.6 GB30.9 GB47.4 GB
Q4_K_M15.6 GB19.5 GB21.5 GB25.6 GB33.9 GB50.4 GB
Q5_K_M18.3 GB22.1 GB24.2 GB28.3 GB36.6 GB53.1 GB
Q6_K20.9 GB24.8 GB26.8 GB31 GB39.2 GB55.7 GB
Q8_026.6 GB30.5 GB32.6 GB36.7 GB44.9 GB61.4 GB
IQ4_XS13.9 GB17.8 GB19.9 GB24 GB32.2 GB48.7 GB

Full-GPU figures: weights + f16 KV cache + overhead. * below our recommended floor of Q4_K_M.

GPU compatibility

GPU8K16K32K64K128K
NVIDIA GeForce RTX 5090Q8_0Q6_KQ6_KIQ4_XSQ5_K_M
NVIDIA GeForce RTX 5080IQ4_XSIQ4_XSIQ4_XSQ8_0Q5_K_M
NVIDIA GeForce RTX 5070 TiIQ4_XSIQ4_XSIQ4_XSQ8_0Q5_K_M
NVIDIA GeForce RTX 5070IQ4_XSIQ4_XSIQ4_XSQ8_0Q5_K_M
NVIDIA GeForce RTX 5060 Ti 16GBIQ4_XSIQ4_XSIQ4_XSQ8_0Q5_K_M
NVIDIA GeForce RTX 5060IQ4_XSIQ4_XSQ8_0Q8_0Q5_K_M
NVIDIA GeForce RTX 4090Q5_K_MQ4_K_MIQ4_XSIQ4_XSQ5_K_M
NVIDIA GeForce RTX 4080 SUPERIQ4_XSIQ4_XSIQ4_XSQ8_0Q5_K_M
NVIDIA GeForce RTX 4070 Ti SUPERIQ4_XSIQ4_XSIQ4_XSQ8_0Q5_K_M
NVIDIA GeForce RTX 4070IQ4_XSIQ4_XSIQ4_XSQ8_0Q5_K_M
NVIDIA GeForce RTX 4060 Ti 16GBIQ4_XSIQ4_XSIQ4_XSQ8_0Q5_K_M
NVIDIA GeForce RTX 4060IQ4_XSIQ4_XSQ8_0Q8_0Q5_K_M
NVIDIA GeForce RTX 3090Q5_K_MQ4_K_MIQ4_XSIQ4_XSQ5_K_M
NVIDIA GeForce RTX 3080 10GBIQ4_XSIQ4_XSQ8_0Q8_0Q5_K_M
NVIDIA GeForce RTX 3070IQ4_XSIQ4_XSQ8_0Q8_0Q5_K_M
NVIDIA GeForce RTX 3060 TiIQ4_XSIQ4_XSQ8_0Q8_0Q5_K_M
NVIDIA GeForce RTX 3060 12GBIQ4_XSIQ4_XSIQ4_XSQ8_0Q5_K_M
NVIDIA GeForce RTX 2080 TiIQ4_XSIQ4_XSQ4_K_MQ8_0Q5_K_M
NVIDIA GeForce GTX 1080 TiIQ4_XSIQ4_XSQ4_K_MQ8_0Q5_K_M
AMD Radeon RX 9070 XTIQ4_XSIQ4_XSIQ4_XSQ8_0Q5_K_M
AMD Radeon RX 7900 XTXQ5_K_MQ4_K_MIQ4_XSIQ4_XSQ5_K_M
AMD Radeon RX 7900 XTQ4_K_MIQ4_XSIQ4_XSIQ4_XSQ5_K_M
AMD Radeon RX 7800 XTIQ4_XSIQ4_XSIQ4_XSQ8_0Q5_K_M
AMD Radeon RX 7600 XTIQ4_XSIQ4_XSIQ4_XSQ8_0Q5_K_M
AMD Radeon RX 6800 XTIQ4_XSIQ4_XSIQ4_XSQ8_0Q5_K_M
AMD Radeon RX 6700 XTIQ4_XSIQ4_XSIQ4_XSQ8_0Q5_K_M
Intel Arc B580IQ4_XSIQ4_XSIQ4_XSQ8_0Q5_K_M
Intel Arc A770 16GBIQ4_XSIQ4_XSIQ4_XSQ8_0Q5_K_M
Apple M2 (16GB)IQ4_XSIQ4_XSIQ4_XSQ8_0Q5_K_M
Apple M2 Pro (32GB)Q5_K_MQ4_K_MIQ4_XSIQ4_XSQ5_K_M
Apple M2 Max (64GB)Q8_0Q8_0Q8_0Q8_0IQ4_XS
Apple M2 Ultra (128GB)Q8_0Q8_0Q8_0Q8_0Q8_0
Apple M3 (16GB)IQ4_XSIQ4_XSIQ4_XSQ8_0Q5_K_M
Apple M3 Pro (36GB)Q6_KQ6_KQ4_K_MIQ4_XSQ5_K_M
Apple M3 Max (64GB)Q8_0Q8_0Q8_0Q8_0IQ4_XS
Apple M3 Ultra (96GB)Q8_0Q8_0Q8_0Q8_0Q8_0
Apple M4 (16GB)IQ4_XSIQ4_XSIQ4_XSQ8_0Q5_K_M
Apple M4 Pro (48GB)Q8_0Q8_0Q6_KQ4_K_MQ4_K_M
Apple M4 Max (64GB)Q8_0Q8_0Q8_0Q8_0IQ4_XS
CPU only / integrated graphicsQ8_0Q8_0Q8_0Q8_0Q5_K_M

Fits on GPUExpert offloadPartial offloadCPU only

Quant guidance

Our floor for Qwen3.5 27B is Q4_K_M — below that, quality degrades faster than the VRAM savings are worth. Prefer the highest quant that still lands "Fits on GPU" at your context length in the table above.

Recommended run command

Q4_K_M at 32K on a NVIDIA GeForce RTX 4090-class GPU (Partial offload):

llama-server -m Qwen3.5-27B-Q4_K_M.gguf -c 32768 --flash-attn -ngl 57

Frequently asked questions

How much VRAM does Qwen3.5 27B need?

At Q4_K_M and 32K context, Qwen3.5 27B needs about 25.6 GB of VRAM to run fully on GPU (weights + KV cache + overhead).

What is the smallest GPU that can run Qwen3.5 27B?

The NVIDIA GeForce RTX 4090 (24 GB) is the smallest GPU in our set that runs Qwen3.5 27B well at 32K context.

What quantization should I use for Qwen3.5 27B?

We recommend Q4_K_M or higher. Q4_K_M weighs 15.6 GB (4.98 bits/weight); going below Q4_K_M costs noticeable quality on a model this size.

How long a context can Qwen3.5 27B handle?

Qwen3.5 27B supports up to 256K tokens. KV cache grows linearly with context.

Can I run Qwen3.5 27B without a GPU?

Yes, at reduced speed: on CPU with 64 GB of DDR5 it manages roughly 2 tokens/sec at 8K context (Q8_0).

Related guides