Hardware for local AI

Most powerful card for local LLMs, 32GB GDDR7, great for quantized 70B.

• 32 GB GDDR7
• 21,760 CUDA cores
• FP4/FP8
• 575W TDP

Best perf/price to run 30-70B models under quantization.

• 24 GB GDDR6X
• 16,384 CUDA cores
• 450W TDP
• DLSS 3

Affordable VRAM to start with 13-30B quantized models.

• 32 GB GDDR6
• 128-bit bus
• 165W TDP
• PCIe 4.0

Queen of the used market for local AI: 24GB on a budget, SLI possible.

• 24 GB GDDR6X
• 10,496 CUDA cores
• 350W TDP
• NVLink

Comfortable baseline for inference and light fine-tuning.

• 64 GB (2×32)
• DDR5-5600
• CL36
• 1.25V

For workstations: load large models into CPU RAM.

• 128 GB (4×32)
• DDR5-4800 ECC
• RDIMM

Critical speed to load model weights fast (7 GB/s).

• 4 TB
• PCIe Gen4
• 7000 MB/s read
• M.2 2280

Good compromise to store several GGUF models.

• 2 TB
• PCIe Gen4
• 7000 MB/s
• DRAM cache

Economical mass storage for model archives (560 MB/s).

• 8 TB
• SATA III
• 560 MB/s
• 2.5″

16 cores for data prep and CPU fallback inference.

• 16 cores / 32 threads
• Zen 5
• 5.7 GHz boost
• 170W

Required for an RTX 5090/4090 + system, stable under load.

• 1200W
• 80+ Platinum
• Modular
• ATX 3.0

Massive unified memory: runs very large LLMs without a dedicated GPU.

• 192 GB unified memory
• 24-core CPU
• 76-core GPU
• 800 GB/s bandwidth

96 cores, 4× PCIe 5.0: the ultimate multi-GPU AI platform.

• 96 cores
• 8-channel DDR5 ECC
• 4× PCIe 5.0 x16
• sTR5