GLM 5.1

754B parameters (MoE with ~40B active), a Q4 file is roughly 420 GB while a 2-bit quant is ~245 GB

• BARE MINIMUM

  Mac Studio M2 Ultra (192 GB Unified Memory) with MoE SSD off

  192 GB UNIFIED MEMORY

  ~$4,200 USED

  Too big for standard consumer hardware; runs a heavily compressed 1.5-bit to 2-bit quant with disk offloading at very sl

  
• SWEET SPOT

  Workstation with dual AMD EPYC CPUs and 512 GB DDR5 RAM plus

  48 GB VRAM + 512 GB SYSTEM RAM

  ~$5,800 USED

  Q3_K_M or dynamic 3-bit quant via llama.cpp CPU/GPU hybrid offloading, ~6 to 10 tok/s at 32k context

  
• FULL POWER

  Dedicated AI workstation with 8x NVIDIA RTX 4090 24GB or 8x

  192 GB TO 384 GB VRAM

  ~$22,000 USED

  Q4_K_M to FP8 full-GPU execution via vLLM or SGLang, fast inference ~35+ tok/s at full 200k context

  

HARDWARE PICKS & PRICES ARE RESEARCHED FROM THE LIVE WEB AND REFRESHED AUTOMATICALLY. TREAT THEM AS BALLPARK, NOT GOSPEL.

STARS ARE THE COMMUNITY SCORE FOR THAT KIND OF WORK, DOCKED FOR HOW SQUASHED THE WEIGHTS GET AT EACH TIER. A DASH MEANS NO SPEED WAS RESEARCHED FOR THAT RIG. TIMES ARE COMPUTED FROM RESEARCHED SPEEDS. BALLPARK, NOT GOSPEL.

The internet is split on GLM 5.1. Biggest praise: help with code. Biggest gripe: price.

207 POSTS · 38 COMMENTS · HACKER NEWS · X · REDDIT · GITHUB · DEV.TO · LEMMY · BLUESKY · DEV FORUMS · BLOG