dexiio

What hardware runs Cydonia 24B?

Cydonia 24B is a 23.57B-parameter dense model (unknown license). Rocinante's bigger sibling on a Mistral Small base. The default serious-RP pick for 24 GB cards.

All figures assume an f16 KV cache, a 0.6 GB display reserve on the GPU, and 64 GB of DDR5 system RAM for the offload tiers. Tune these in the calculator.

Minimum VRAM by quant and context

QuantFile size8K16K32K64K128K
Q2_K *8.3 GB11.4 GB12.7 GB15.3 GB20.6 GB31.1 GB
Q3_K_M *10.7 GB13.8 GB15.1 GB17.7 GB23 GB33.5 GB
Q4_K_M13.3 GB16.5 GB17.8 GB20.4 GB25.6 GB36.1 GB
Q5_K_M15.6 GB18.7 GB20 GB22.7 GB27.9 GB38.4 GB
Q6_K18.0 GB21.1 GB22.4 GB25.1 GB30.3 GB40.8 GB
Q8_023.3 GB26.4 GB27.8 GB30.4 GB35.6 GB46.1 GB

Full-GPU figures: weights + f16 KV cache + overhead. * below our recommended floor of Q4_K_M.

GPU compatibility

GPU8K16K32K64K128K
NVIDIA GeForce RTX 5090Q8_0Q8_0Q8_0Q6_KQ4_K_M
NVIDIA GeForce RTX 5080Q4_K_MQ4_K_MQ4_K_MQ4_K_MQ8_0
NVIDIA GeForce RTX 5070 TiQ4_K_MQ4_K_MQ4_K_MQ4_K_MQ8_0
NVIDIA GeForce RTX 5070Q4_K_MQ4_K_MQ4_K_MQ8_0Q8_0
NVIDIA GeForce RTX 5060 Ti 16GBQ4_K_MQ4_K_MQ4_K_MQ4_K_MQ8_0
NVIDIA GeForce RTX 5060Q4_K_MQ4_K_MQ6_KQ8_0Q8_0
NVIDIA GeForce RTX 4090Q6_KQ6_KQ5_K_MQ4_K_MQ5_K_M
NVIDIA GeForce RTX 4080 SUPERQ4_K_MQ4_K_MQ4_K_MQ4_K_MQ8_0
NVIDIA GeForce RTX 4070 Ti SUPERQ4_K_MQ4_K_MQ4_K_MQ4_K_MQ8_0
NVIDIA GeForce RTX 4070Q4_K_MQ4_K_MQ4_K_MQ8_0Q8_0
NVIDIA GeForce RTX 4060 Ti 16GBQ4_K_MQ4_K_MQ4_K_MQ4_K_MQ8_0
NVIDIA GeForce RTX 4060Q4_K_MQ4_K_MQ6_KQ8_0Q8_0
NVIDIA GeForce RTX 3090Q6_KQ6_KQ5_K_MQ4_K_MQ5_K_M
NVIDIA GeForce RTX 3080 10GBQ4_K_MQ4_K_MQ4_K_MQ8_0Q8_0
NVIDIA GeForce RTX 3070Q4_K_MQ4_K_MQ6_KQ8_0Q8_0
NVIDIA GeForce RTX 3060 TiQ4_K_MQ4_K_MQ6_KQ8_0Q8_0
NVIDIA GeForce RTX 3060 12GBQ4_K_MQ4_K_MQ4_K_MQ8_0Q8_0
NVIDIA GeForce RTX 2080 TiQ4_K_MQ4_K_MQ4_K_MQ8_0Q8_0
NVIDIA GeForce GTX 1080 TiQ4_K_MQ4_K_MQ4_K_MQ8_0Q8_0
AMD Radeon RX 9070 XTQ4_K_MQ4_K_MQ4_K_MQ4_K_MQ8_0
AMD Radeon RX 7900 XTXQ6_KQ6_KQ5_K_MQ4_K_MQ5_K_M
AMD Radeon RX 7900 XTQ5_K_MQ5_K_MQ4_K_MQ4_K_MQ8_0
AMD Radeon RX 7800 XTQ4_K_MQ4_K_MQ4_K_MQ4_K_MQ8_0
AMD Radeon RX 7600 XTQ4_K_MQ4_K_MQ4_K_MQ4_K_MQ8_0
AMD Radeon RX 6800 XTQ4_K_MQ4_K_MQ4_K_MQ4_K_MQ8_0
AMD Radeon RX 6700 XTQ4_K_MQ4_K_MQ4_K_MQ8_0Q8_0
Intel Arc B580Q4_K_MQ4_K_MQ4_K_MQ8_0Q8_0
Intel Arc A770 16GBQ4_K_MQ4_K_MQ4_K_MQ4_K_MQ8_0
Apple M2 (16GB)Q4_K_MQ4_K_MQ4_K_MQ8_0Q8_0
Apple M2 Pro (32GB)Q6_KQ6_KQ5_K_MQ4_K_MQ5_K_M
Apple M2 Max (64GB)Q8_0Q8_0Q8_0Q8_0Q8_0
Apple M2 Ultra (128GB)Q8_0Q8_0Q8_0Q8_0Q8_0
Apple M3 (16GB)Q4_K_MQ4_K_MQ4_K_MQ8_0Q8_0
Apple M3 Pro (36GB)Q8_0Q6_KQ6_KQ4_K_MQ4_K_M
Apple M3 Max (64GB)Q8_0Q8_0Q8_0Q8_0Q8_0
Apple M3 Ultra (96GB)Q8_0Q8_0Q8_0Q8_0Q8_0
Apple M4 (16GB)Q4_K_MQ4_K_MQ4_K_MQ8_0Q8_0
Apple M4 Pro (48GB)Q8_0Q8_0Q8_0Q8_0Q4_K_M
Apple M4 Max (64GB)Q8_0Q8_0Q8_0Q8_0Q8_0
CPU only / integrated graphicsQ8_0Q8_0Q8_0Q8_0Q8_0

Fits on GPUExpert offloadPartial offloadCPU only

Quant guidance

Our floor for Cydonia 24B is Q4_K_M — below that, quality degrades faster than the VRAM savings are worth. Prefer the highest quant that still lands "Fits on GPU" at your context length in the table above.

Recommended run command

Q4_K_M at 32K on a NVIDIA GeForce RTX 4090-class GPU (Fits on GPU):

llama-server -m Cydonia-24B-v4.3-Q4_K_M.gguf -c 32768 --flash-attn -ngl 99

Frequently asked questions

How much VRAM does Cydonia 24B need?

At Q4_K_M and 32K context, Cydonia 24B needs about 20.4 GB of VRAM to run fully on GPU (weights + KV cache + overhead).

What is the smallest GPU that can run Cydonia 24B?

The NVIDIA GeForce RTX 4090 (24 GB) is the smallest GPU in our set that runs Cydonia 24B well at 32K context.

What quantization should I use for Cydonia 24B?

We recommend Q4_K_M or higher. Q4_K_M weighs 13.3 GB (4.86 bits/weight); going below Q4_K_M costs noticeable quality on a model this size.

How long a context can Cydonia 24B handle?

Cydonia 24B supports up to 128K tokens. KV cache grows linearly with context.

Can I run Cydonia 24B without a GPU?

Yes, at reduced speed: on CPU with 64 GB of DDR5 it manages roughly 2 tokens/sec at 8K context (Q8_0).

Related guides