What hardware runs Gemma 3 4B?
Gemma 3 4B is a 3.88B-parameter dense model (gemma license). Edge Gemma with real vision. Fine as a first model on integrated graphics; outclassed by Qwen at the same size.
All figures assume an f16 KV cache, a 0.6 GB display reserve on the GPU, and 64 GB of DDR5 system RAM for the offload tiers. Tune these in the calculator.
Minimum VRAM by quant and context
| Quant | File size | 8K | 16K | 32K | 64K | 128K |
|---|---|---|---|---|---|---|
| Q2_K * | 1.6 GB | 3.7 GB | 4 GB | 4.4 GB | 5.3 GB | 7 GB |
| Q3_K_M * | 2.0 GB | 4.1 GB | 4.3 GB | 4.7 GB | 5.6 GB | 7.4 GB |
| Q4_K_M | 2.3 GB | 4.5 GB | 4.7 GB | 5.1 GB | 6 GB | 7.7 GB |
| Q5_K_M | 2.6 GB | 4.8 GB | 5 GB | 5.4 GB | 6.3 GB | 8 GB |
| Q6_K | 3.0 GB | 5.1 GB | 5.3 GB | 5.8 GB | 6.6 GB | 8.4 GB |
| Q8_0 | 3.8 GB | 6 GB | 6.2 GB | 6.6 GB | 7.5 GB | 9.3 GB |
| IQ4_XS | 2.1 GB | 4.2 GB | 4.5 GB | 4.9 GB | 5.8 GB | 7.5 GB |
Full-GPU figures: weights + f16 KV cache + overhead. * below our recommended floor of Q4_K_M.
GPU compatibility
Fits on GPUExpert offloadPartial offloadCPU only
Quant guidance
Our floor for Gemma 3 4B is Q4_K_M — below that, quality degrades faster than the VRAM savings are worth. Prefer the highest quant that still lands "Fits on GPU" at your context length in the table above.
Recommended run command
Q4_K_M at 32K on a NVIDIA GeForce RTX 5060-class GPU (Fits on GPU):
llama-server -m gemma-3-4b-it-Q4_K_M.gguf -c 32768 --flash-attn -ngl 99
Frequently asked questions
How much VRAM does Gemma 3 4B need?
At Q4_K_M and 32K context, Gemma 3 4B needs about 5.1 GB of VRAM to run fully on GPU (weights + KV cache + overhead).
What is the smallest GPU that can run Gemma 3 4B?
The NVIDIA GeForce RTX 5060 (8 GB) is the smallest GPU in our set that runs Gemma 3 4B well at 32K context.
What quantization should I use for Gemma 3 4B?
We recommend Q4_K_M or higher. Q4_K_M weighs 2.3 GB (5.13 bits/weight); going below Q4_K_M costs noticeable quality on a model this size.
How long a context can Gemma 3 4B handle?
Gemma 3 4B supports up to 128K tokens. KV cache grows linearly with context, though sliding-window layers cap most of the growth.
Can I run Gemma 3 4B without a GPU?
Yes, at reduced speed: on CPU with 64 GB of DDR5 it manages roughly 12 tokens/sec at 8K context (Q8_0).