I want to run GLM-5.2 at home. The 744B-parameter MoE that Z.ai open-sourced in June [1].

The target is specific: 30 tokens/s, single-stream decode.

That line isn't arbitrary. Two reference points. First, of the 32 inference providers serving GLM-5.2 on OpenRouter, 26 (81%) deliver a P50 throughput above 30 tok/s, with a provider median around 41 — amusingly, Z.ai's own endpoint sits at 21 [38]. So 30 tok/s is simply the passing grade for commercial inference; a local rig shouldn't be a full tier slower than something you can rent. Second, the closed models we use daily live in the same range: GPT-5.6's official endpoints run P50 at 37–38 tok/s, GPT-5.5 at 47–48 [39]. Below 30, you watch characters crawl across the terminal, tab out to scroll X, and lose the thread.

All 32 GLM-5.2 providers on OpenRouter with prices and P50 throughput (screenshot, 2026-07-28)

OpenRouter's provider list for z-ai/glm-5.2 (P50, screenshot taken 2026-07-28): 26 of 32 providers ≥30 tok/s, median ≈41 [38].

OpenRouter's providers for GPT-5.6 Sol with P50 throughput (screenshot, 2026-07-28)

GPT-5.6 Sol on the same day: OpenAI's official endpoint at 38 tps, Azure at 37 [39]. That's the everyday closed-model baseline.

The internet has no shortage of enthusiastic recommendations: buy a Mac Studio, stack DGX Sparks, throw four 3090s at it. I went through every community measurement I could find — r/LocalLLaMA, the NVIDIA developer forums, Hugging Face discussions, X.

Here's the conclusion up front: under one consistent yardstick (batch-1, short-to-mid context, full unpruned model), most setups don't reach 30 tok/s, and the one that clears it comfortably costs $45,000. I posted the summary table on X and it got 57K views. The replies contributed a batch of valuable first-hand measurements, some more optimistic than my table. Digesting those cases produced this article.

This post covers three things: the full data, the test cases the replies added (with each one's premises), and why numbers that look contradictory mostly aren't.

Full disclosure on method, so this doesn't sound more rigorous than it is: every number here is a community measurement — I did not buy 23 hardware configurations myself; I don't have a $50k hobby budget. The aggregation was done with Manus, and I hand-checked the key numbers and sources. Prices are July 2026, mid-memory-crisis, and volatile. Per-claim sources are in the references.

Start With the Physics

GLM-5.2 is a 744B-parameter MoE activating about 40B per token [1]. To run it, you must clear exactly two bars.

First, it has to fit. Memory needed per quantization level (Unsloth Dynamic GGUF measurements [2]):

Quant Memory Quality
1-bit (UD-IQ1_S) ~223 GB ~76% top-1, visibly lobotomized
2-bit (UD-IQ2_M) ~245 GB ~82%, usable but lossy
3-bit (UD-Q3_K_XL) 290–360 GB decent
4-bit (UD-Q4_K_XL) 372–475 GB near-lossless
NVFP4 (official) ~465 GB near-lossless, Blackwell only
FP8 (official) ~750 GB native precision

In other words, 245 GB is the entry ticket, and "not lobotomized" starts at 370 GB. Your 128 GB "AI mini beast" can't even fit the most brutal quant.

Second, it has to read fast. Decoding is a bandwidth game: every generated token requires reading all ~40B activated parameters — about 20–22 GB at 4-bit. Hitting 30 tok/s needs 600+ GB/s of effective bandwidth. For reference: DGX Spark has 273 GB/s, M3 Ultra has 819 GB/s, RTX PRO 6000 has 1.79 TB/s.

Two more variables muddy the water, and almost every disputed number below traces back to them.

MTP speculative decoding. GLM-5.2 ships with multi-token prediction, drafting 5 tokens per step [1] — worth a 1.5–2.5× speedup. But llama.cpp still hasn't implemented GLM's MTP; only vLLM/SGLang (and their community-patched forks) can use it [3]. Same machine, different software stack, 2× different speed. That's why.

The REAP pruned variant. The community used Cerebras's REAP method to cut 256 experts down to 168, dropping memory needs by 34%. The cost: Terminal-Bench falls from 81 to 70.5 [4]. Cheaper, but the brain damage is real.

The Whole Picture in One Table

This is the summary table I posted on X:

GLM-5.2 cost vs. measured decode speed by hardware setup

How to read it: different rows were measured on different quants, and every number is a community measurement — not my own re-run. Per-row quant, engine, and source are in the "Quant / engine" and "Source" columns below.

Which version does each number measure? And who measured it?

After the first draft I realized neither this table nor the underlying research report spelled out which quantization each number was measured on. That matters for fair comparison, so here it is.

This is not one exam. Each hardware class runs the best quant it can physically fit, so speeds naturally correspond to different model versions. Three tiers to keep in mind:

  • Native FP8 (~750 GB): among home setups, only 8× RTX PRO 6000 (768 GB VRAM, ~$95k+) fits it — ~106 tok/s at bs=1, measured by llamaCTO on r/LocalLLaMA [10]. That's the only "home FP8" measurement in this entire survey (DGX Station GB300's 784 GB of coherent memory could theoretically hold it too, but the community has only published REAP-pruned numbers for it [3]). The official vLLM recipes deploy FP8 on 8× H200/B200 in datacenters [19]; the GH200 FP8 data in the concurrency section comes from dnhkng's bench [34]. The main table and its verdicts are not based on FP8.
  • Near-lossless tier (4-bit class: NVFP4, IQ4_XS, AWQ-INT4, MLX 4-bit, Q4/Q5_K): the primary yardstick for PASS / BORDERLINE / FAIL in this article; quality is close to full FP8 [2].
  • Degraded tier (below 3-bit, plus REAP pruning): it runs, but it's a different product. Rows that only have 1/2-bit data (2× Spark, 4× 3090, Strix Halo) aren't lazy benchmarking — that hardware can't fit 4-bit at all, which is itself part of the verdict. Notably, 4× 3090 runs exactly as fast at 1-bit as at 2-bit [14]: the bottleneck is the CPU, so low-bit quants aren't inflating those scores.

The verdict rule in one sentence: each row is judged on the best quant that hardware can fit, per community measurements. With that in mind, the core data:

Setup Memory Quant / engine Measured tok/s Est. cost Verdict Source
4× RTX PRO 6000 Blackwell 384 GB GDDR7 REAP NVFP4·vLLM+MTP ／ full IQ4_XS·llama.cpp 45–60 ／ 38→29 $45k–55k PASS (borderline on full model) [5][6]
DGX Station GB300 (single box) 288 GB HBM3e + 496 GB LPDDR5X REAP-504B ~60 ~$100k+ PASS (pruned) [3][43]
8× GB10 cluster (ASUS GX10) 1 TB unified full AWQ-INT4·patched vLLM+MTP ／ INT4/INT8 mix 26–27 ／ 33–54 (coding) ~$28k–30k BORDERLINE [7][8]
2× Mac Studio M3 Ultra 1 TB unified full MLX 4-bit distributed 23.4 ~$28k+ (used) FAIL [11]
4× DGX Spark 512 GB unified NVFP4 DCP4@128K ／ AWQ-INT4+15% pruned ／ NF3 mix 14.5–15.2 ／ 20–22 ／ 20–33 $14k–20k FAIL [10][7][9]
1× Mac Studio M3 Ultra 512GB 512 GB unified full MLX 4-bit 16–18.8 ~$14.2k (used) FAIL [11]
2× RTX PRO 6000 + 1TB DDR5 192 GB VRAM + offload full Q4_K_XL·llama.cpp 13–15 ~$30k FAIL [3]
2× RTX 5090 + 512GB RAM 64 GB VRAM + offload Q5_K_S·llama.cpp ~12 ~$10k FAIL [12]
2× DGX Spark (linked) 256 GB unified UD-IQ1_S (1-bit)·RPC 8.1 ~$9.4k FAIL [13]
4× RTX 3090 + 192GB RAM 96 GB VRAM + offload UD-IQ2_M (2-bit) 7.3 ~$4.5k FAIL [14]
2× AMD Strix Halo (linked) 256 GB unified UD-IQ2_M (2-bit)·RPC 7.2 ~$5.2k FAIL [15]
EPYC/Xeon CPU-only 512–768 GB DDR Q2–Q4_K_XL 1.7–5.5 $2.5k–4k FAIL [16][17]
1× DGX Spark 128 GB unified doesn't fit; extreme pruning 1–4 $4.7k FAIL [18]
AMD MI300X / Intel Gaudi — no software or retail path — B2B only NO PATH [19][20]

The pattern fits in one sentence: only setups that keep every weight in high-bandwidth VRAM and use MTP get anywhere near 30. Every CPU-offload configuration — no matter how strong the GPUs — is pinned below 15 by system memory bandwidth and llama.cpp's missing MTP.

Cost vs. decode speed scatter plot: only the dark-green dots in the upper right clear the line

The scatter plot also mixes different quant recipes per point; use it for the overall cost–speed landscape and go back to the table's quant column for any specific comparison.

Taking It Apart, Line by Line

DGX Spark: one is a toy, eight are a plan

A single Spark has 128 GB — no usable quant fits, and it measures 1–4 tok/s [18]. Its 273 GB/s memory bandwidth is a hard ceiling. On price, the memory shortage pushed NVIDIA from $3,999 to $4,699 [21]; meanwhile the GB10 chip has a whole family of OEM clones, and the ASUS Ascent GX10 at $3,499 is the cheapest way in [20].

The real play is clustering over ConnectX-7. Four units run AWQ-INT4 with 15% pruning plus MTP at 20–22 tok/s [7]; the 128K-context DCP4 recipe drops to 14.5–15.2 [10]; an NF3 hybrid spikes to 20–33 at short context [9].

Eight GB10s (~$28k) top out this lane: the full model with MTP holds 26–27 [7], and a separate INT4/INT8 recipe reports 33–54 on coding output [8] — the only sub-$30k setup with any measured number above 30. The price is the steepest setup complexity in the field: a 100G switch, hand-patched vLLM, Triton kernel hacks, and forums full of scar tissue [8][10].

It also holds a hidden ace: prefill at 500–1,200 tok/s [8]. For agent workflows that matters more than decode. You'll see why when we get to the Macs.

Mac Studio: the former king, undone by prefill and the memory crisis

M3 Ultra has 819 GB/s of unified memory bandwidth and used to be the value king for big MoEs. 2026 cut it twice.

First cut: the memory crisis. Apple pulled the 256/512GB configs [22]; 512GB machines are used-market only, from about $14,200, briefly scalped to $25,000 [20].

Second cut: the measurements aren't enough. A single machine runs the full model at 16–18.8 tok/s on MLX 4-bit; two linked over Thunderbolt 5 (Ivan Fioravanti's bench) reach just 23.4 [11]. The 256GB model only fits 2-bit (3–9 tok/s) or the visibly degraded 1-bit (~21.6) [23]. MLX has no GLM MTP support yet; if it lands it could add 30–50%, and antirez is watching that space [24] — but that's the future, not the present.

The real wound is prefill. Under llama.cpp, a 33K-token prompt takes nearly 10 minutes [10]. Want a coding agent that regularly stuffs 50–100K of context? First token in 10+ minutes. Unusable in practice.

4× RTX PRO 6000: the only clean pass, and NVIDIA knows it

Four cards give 384 GB of GDDR7 at 1.79 TB/s each. All weights resident in high-bandwidth VRAM, native MTP in vLLM — the only individually-purchasable combination that checks every physical box.

Measurements: REAP-pruned NVFP4 does ~60 tok/s at short context, still ~45 at 64k–100k, prefill ~5,100 [5]; the full unpruned model in IQ4_XS starts at 38 and sags to 29 by 40k context — borderline [6].

The price tag is explicit. NVIDIA raised this card 55% in June to $13,250 [25]; the early $8,500 street price is gone for good. A four-card build lands at $45,000–55,000, pulls 2.2 kW under load, and wants a dedicated circuit. That's the price of the wall.

Want native FP8 (~750 GB)? That's 8 cards (768 GB VRAM, ~$95k+): llamaCTO measured ~106 tok/s at bs=1 [10] — double the budget, mostly buying precision rather than speed.

One trap to avoid: two cards (192 GB) can't fit any full-model quant and fall back to RAM offload at 13–15 tok/s [3]. $30k for the worst of both worlds.

DGX Station GB300: one box that matches four cards, priced like eight

The new variable that just went on sale. The GB300 Grace Blackwell superchip packs 288 GB of HBM3e (~8 TB/s) plus 496 GB LPDDR5X of coherent memory in a single box [44], and the community measures the REAP-504B pruned model at ~60 tok/s [3] — the same speed tier as 4× RTX PRO 6000, but one quiet tower on a 1,600 W PSU, no four-GPU power engineering.

Pricing is equally blunt. Distributor pi3g's channel roundup: from about $100k before tax, with OEMs (Dell, ASUS, Supermicro, HP…) quoting $99.5k–123k, lead times of 4–13 weeks, earliest arrivals in August — and a warning that memory prices will keep pushing it up [43]. In one line: double the money buys convenience and HBM headroom, not more decode speed than the four-card rig.

(Detail note: channel pages disagree slightly on memory — pi3g lists 748 GB (252+496) [43], NVIDIA's official page 784 GB (288+496) [44].)

Consumer multi-GPU: the busiest lane, the saddest results

This is the most-attempted route and the most brutal conclusion. 4× 3090 with 192GB DDR5: 7.3 tok/s. Switch to 1-bit and the speed doesn't move — the bottleneck isn't bandwidth at all, it's the CPU [14].

More counterintuitive: 6× 3090 gets 7.8 [26], while 1× 5090 plus a big-RAM Xeon gets 11 [3]. On the offload route, buying more GPUs is nearly useless. $4,000–10,000 buys 7–12 tok/s [12][14][26], with no upgrade path.

AMD & Intel: one you can't buy, one that can't run it

Two Strix Halos linked, 2-bit: 7.2 tok/s, and the owner's own verdict was "not worth it" [15]. The Instinct MI300X line is actually fine on software — vLLM/SGLang ship official ROCm recipes [19] — but it's B2B-only, from $25,000 a card [20], with essentially no retail path with a warranty.

On Intel: I found no support record for GLM-5.2's new GlmMoeDsa architecture on the IPEX/SYCL stack, and Gaudi 3 has no consumer channel. The most realistic option is Xeon AMX with ktransformers (which has an official GLM-5.2 tutorial [27]) — extrapolating a personal config lands at 10–16 tok/s. Below the bar.

Server CPUs: a $3,000 ticket to watch from the stands

A used EPYC with 512 GB DDR4 runs $2,500–3,500 and is the cheapest way to make it run at all: 1.7–5.5 tok/s pure CPU [16][17], 6–11 with two 3090s handling offload [16]. The point of this route is experiencing a 744B model for $3k — not reaching 30 tok/s.

The Pieces the Replies Filled In

After the table went out, the X replies contributed a batch of first-hand information, most of it on the DGX Spark cluster lane — exactly what a survey like this needs: more real cases. Here they are, each with a note on its premises:

KING JEW @0xSero · Blue check

"GLM-5.2 runs in 4 sparks at 30 tok/s"

Premise note: quant and context unstated. He authored the REAP pruned model and the sm120 recipe [4][5]; if this is the pruned variant, 30 on 4 Sparks squares with the bandwidth math.

3.2K views · 55 likes · original reply [28]

Jim McPherson @JamesMcPherson · Blue check

His 4 Sparks run at 40–41 tok/s, later adding: over 40 now. He also argues 8 units shouldn't be slower than 4, questioning the configuration behind my 8× figure.

Premise note: configuration unstated. A pruned model at short context with MTP-friendly coding output wouldn't conflict with the full model's 26–27 at 256K context [7] — they'd simply be measuring different things.

650 views · 9 likes · original reply [29]

Dave Charland @Dave_Charland · Blue check

Runs 4 Sparks himself; 8-Spark users he's heard from call it "fantastic". He also names the other side of the trade: an RTX 6000 rig would surely be a great experience — if you don't mind the noise, the heat, and the much larger upfront spend.

Premise note: qualitative, no numbers. The dimensions he adds — noise, heat, upfront cost — are columns my table doesn't have, and they genuinely belong in the decision.

1.3K views · 13 likes · original reply [30]

Austin S. Lin @siraustin · Blue check

Prices out a build: 4 Sparks plus a MikroTik switch plus cables, about $20k.

Premise note: a cost case. It lands at the top of my table's $14k–20k range for 4× Spark (networking included) — the two corroborate each other.

191 views · original reply [31]

Taimur Ayaz @taimurayaz · Blue check

Mentions that those machines run fp16.

Premise note: if that means weight precision, fp16 needs ~1.5 TB and 4 Sparks only have 512 GB; more likely it refers to compute precision (quantized weights, fp16 activations). The reply doesn't elaborate — filed for the record.

285 views · original reply [32]

Kristoph @kristoph · Blue check

Nails the framing in one line: whether it runs was never in question — the chart says it runs; the only dispute is what standard counts as passing.

Premise note: which is exactly this article's subject — the next two sections take the word "standard" apart.

112 views · original reply [33]

One piece of background deserves its own line: @0xSero is not a random commenter. The REAP pruned model (GLM-5.2-504B [4]) and the 4× RTX PRO 6000 sm120 vLLM recipe [5] cited in this survey are both his work. First-hand numbers from someone with that background go to the top of the reading pile.

Put every number on one axis and the positions sort themselves out:

010203040 tok/s Target: 30 tok/s Published recipes 14.5–22 (4-bit class) [7][10] NF3 recipe, short context 20–33 [9] @0xSero: 30 [28] @JamesMcPherson: 40–41 [29]
Every 4× DGX Spark number on one axis: bands are measured ranges from published, reproducible recipes [7][9][10]; dots are first-hand numbers from the replies (quant and context unstated) [28][29]. Note the recipes themselves use different quants (NVFP4 DCP4 / AWQ-INT4+pruning / NF3 mix); all community-measured, none re-run by me.

Once the yardsticks are aligned, my read is: these numbers and my table are most likely all true — they measure different things. Four variables are enough to explain the spread:

Context length. The NF3 recipe spikes to 33 at short context; the 128K DCP4 recipe sits at 14.5 [9][10]. Same hardware, 2× apart.

MTP acceptance rate. Speculative decoding accepts far more drafts on code than on free text — 1.5–2× faster [8]. Numbers measured driving a coding agent and numbers logged on a general workload are inherently incomparable.

Full model or pruned. Probably the decisive one. The REAP variant reads about a third fewer weights [4]; take the 4× Spark that does 20–22 on the full model [7], load the pruned build, and 30 pencils out on bandwidth alone. 0xSero wrote the pruned variant — his "GLM-5.2" and my table's "GLM-5.2, full" may simply not be the same model, with both numbers being right.

Single stream or aggregate. Concurrent-request throughput is a different arithmetic entirely. It gets its own section — the next one.

As for 40–41 [29]: I haven't matched it to any published recipe — the closest is NF3's 33 at short context [9]. That more likely means public documentation trails what frontline operators have tuned by hand, not that the number is wrong. Send me the vLLM flags, quant, and context length and it goes into the table as a new recipe. This is the state of local LLM benchmarking in 2026: everyone is measuring, but rarely the same thing. My table holds one yardstick — batch-1, short-to-mid context, traceable sources — and its only virtue is that you know exactly what it measures.

What If We Count Concurrency?

Some of the reply numbers are very plausibly aggregate throughput: two requests at 20 tok/s each, call it 40. That's a perfectly legitimate yardstick — multiple agents running in parallel is many people's real workload — so I ran a follow-up analysis for it. The result is more interesting than the intuition.

For dense models, concurrency is nearly free: at batch=2 the weights are read once and serve two requests, throughput roughly doubles. GLM-5.2's MoE structure cuts most of that dividend. Each layer activates 8 of 256 experts, and two concurrent requests hit almost disjoint expert sets, so routed-expert reads scale nearly linearly with batch size. Only attention and the shared experts — about a third of the read volume — get reused. Net: batch=2 yields about 1.2–1.4× aggregate, not 2×.

The measured anchors agree: dual GH200 running FP8 with heavy offload gains only 1.16× at batch=4 [34]; on Apple Silicon a MoE model gets 1.89× at 5-way concurrency [35][36]. All-VRAM setups are a different universe: 8× RTX PRO 6000 goes from 106 at bs=1 to ~420 at bs=8, nearly 4× [10].

So the aggregate-yardstick landscape:

The $28k tier genuinely flips. 8× GB10 at 2–4 concurrent hits 34–46 aggregate, moving from borderline to passing; dual M3 Ultra reaches 29–40; 4× Spark scrapes aggregate-30 at bs=4 (all extrapolated from MoE scaling factors, ±20%). If your real workload is a fleet of parallel agents and 15–25 tok/s per stream is acceptable, $14k–28k is enough. Some of the reply numbers may have been measured exactly this way.

The cheap offload tier still doesn't recover. Its bottleneck is system memory bandwidth and CPU; MoE's non-overlapping experts give it the worst batch gains in the field, and batch=4 lifts it from 7 to barely over 10. The foundation is too slow; concurrency can't save it.

Most counterintuitive: 4× RTX PRO 6000 becomes the best value. Under the single-stream yardstick it's "the expensive passing option"; under aggregate it's projected at 150–200 for 4 concurrent streams, or roughly $300–350 per aggregate tok/s — half the 8× GB10's $650–700. It can feed 5–8 agents at once, each still above 25 tok/s.

Two engineering caveats: concurrency eats KV cache, so long context and high concurrency don't coexist on 128GB-per-node Sparks (the 4× Spark 128K recipe literally sets MAX_NUM_SEQS=1 [10]); and MTP competes with concurrency — on the GH200, enabling MTP at batch=4 dropped aggregate from 23.6 to 15.2, while the same machine's AWQ recipe kept a positive gain [34]. Measure per configuration; don't assume.

Putting the Hardware Specs Side by Side

Topology first. Three encoding rules: rectangle area = compute / capacity (green = GPU, deep teal = HBM/GDDR, pale teal = LPDDR/DDR, gray = CPU/switch/NIC); line width = bandwidth, every link on one shared linear scale (8px per TB/s; capped lines grow "ellipsis" strips meaning thicker-than-drawn; sub-pixel links are drawn at the 1px floor); endpoints = physical landing points (PCIe lands on the CPU, network lands on the NIC, direct chip links land on the chip). Numbers and sources match the spec table below.

DGX Station GB300 · one box, ~$100k+REAP measured ~60 tok/s · every link on one linear scaleGB300 GPU20,000 TOPS (FP4)HBM3e288 GB8 TB/s: beyond the 48px cap — ellipsis strips = thicker than drawnGrace CPUNVLink-C2C 900 GB/sLPDDR5X496 GB396 GB/s
4× RTX PRO 6000 · ~$50kmeasured 29–60 tok/s · CPU-centered PCIe starEPYC/TR4,00096GB4,00096GB4,00096GB4,00096GBGPU→VRAM: 1.79 TB/s ×4 (14.3px)GPU↔CPU: PCIe 5.0 x16 ≈64 GB/s (0.5px, drawn at the 1px floor)No NVLink between cards: P2P DMA crosses the CPU's PCIe root complex, so every line lands on the CPU
4× DGX Spark · switch star · ~$14–20kmeasured 14.5–22 tok/s · network lines land on the NIC100GswitchGB10 SoCGPUCPU128GBCX7GB10 SoCGPUCPU128GBCX7GB10 SoCGPUCPU128GBCX7GB10 SoCGPUCPU128GBCX7CX7↔switch ≈25 GB/s (drawn at the 1px floor)SoC→unified memory 273 GB/s (2.2px) · NIC hangs off the SoC via PCIeRDMA skips the CPU cores but must physically enter the SoC · 4+ units need a switch (≈$0.8–1.5k)
3× DGX Spark · direct ring · ~$11–14kthe budget wiring for ≤3 units (no switch)GB10 SoCGPUCPU128GBCX7GB10 SoCGPUCPU128GBCX7GB10 SoCGPUCPU128GBCX7Dual QSFP ports, direct ringtwo ports of the same CX7 · ≈25 GB/s per edge (1px floor)Three units need no switch: each connects both ports to its neighbors, saving ~$1k
2× Mac Studio M3 Ultra · ~$28k usedmeasured 23.4 tok/s · memory outboard, SoCs linked directlyunified memory512GBunified memory512GBM3 Ultra SoCCPUGPUM3 Ultra SoCCPUGPU819 GB/s819 GB/sThunderbolt 5 ≈10 GB/sSoC↔SoC direct (drawn at the 1px floor)* GPU counted at Apple's official FP32 ≈28 TF (no FP4 units) — the green block nearly vanishes,big memory, decent pipe, tiny compute: decode 23.4, prefill only 60–90
2× RTX 5090 + 512GB DDR5 offload · ~$10kmeasured ~12 tok/s · wired as physically true: GPU—CPU—DDRCPU3,35232GB3,35232GBDDR5512GB≈90 GB/sPCIe x8 ≈32 GB/sactivations only (tiny)GPU→VRAM 1.79 TB/s is fat, but feeds only 32GB (resident layers)DDR sits on the CPU's memory controller; there is no GPU→DDR wire;offloaded expert layers are read from DDR and computed by the CPUthe foundation is 90 GB/s of DDR + CPU compute → ~12 tok/s
2× AMD Strix Halo · ~$5.2kmeasured 7.2 tok/s · the thinnest interconnect in the fieldunified memory128GBunified memory128GBAI Max+ 395CPUGPUAI Max+ 395CPUGPUNICNIC256 GB/s256 GB/sllama.cpp RPC over ≤10 GbE ≈1.25 GB/sthinnest link here: an order below TB5 (drawn at the 1px floor)* platform total ~126 TOPS (incl. NPU; no FP4 metric) — a 256 GB/s memory pipebehind a 1.25 GB/s node link: fits 2-bit, can't make it fast
Reference: 2× GH200 · what NVLink looks like · ~$70k+measured 20–43 tok/s · interconnect on the same scale as memory pipesHBM 96GB · 4 TB/sH100≈4 PFLOPS*Grace480GBHBM 96GB · 4 TB/sH100≈4 PFLOPS*Grace480GBNVLink 900 GB/s (GPU↔GPU direct)900 GB/s* H100 has no FP4; counted at sparse FP8 ≈4 PFLOPS · HBM 96GB @ 4 TB/s (same cap-ellipsis rule)the links are 1–2 orders fatter than any home rig — why datacenter node-stacking works and home node-stacking doesn't
Interconnect magnifier ×20 — comparable inside this box onlyNVLink-C2C · 900 GB/s (GB300 CPU↔GPU) (still capped at ×20; ellipsis = thicker)PCIe 5.0 x16 · 64 GB/s (PRO 6000 card↔CPU)PCIe 5.0 x8 · 32 GB/s (consumer dual-card split)ConnectX-7 · 25 GB/s (Spark node-to-node)Thunderbolt 5 · 10 GB/s (dual Mac)≤10 GbE · ≈1.25 GB/s (Strix Halo RPC; still <1px at ×20)For reference: the GB300 HBM pipe in the main panels = the 48px cap — this box's fattest line, NVLink-C2C, is just 7.2px out there

Now the table. The two conclusions below are already visible in the drawings: decode speed is a function of memory-pipe width, and clustering is hard because the inter-node lines are one pixel thin. Raw specs (compute column uses each vendor's published sparse-FP4 AI figure; the 3090 has no FP4, so INT8; Apple/AMD use different metrics, as noted):

Platform Architecture Typical GPUs per build Per-GPU compute Per-GPU memory bandwidth Inter-GPU / inter-node link Jul 2026 price
RTX PRO 6000 Blackwell Blackwell 4–8 cards ~4,000 AI TOPS (FP4) 96 GB GDDR7 @ 1.79 TB/s PCIe 5.0 x16 (no NVLink) ≈ 64 GB/s $13,250/card [25]
DGX Station GB300 Blackwell Ultra + Grace 1 (superchip) ~20 PFLOPS (FP4) 288 GB HBM3e @ ~8 TB/s (+496 GB LPDDR5X @ 396 GB/s) NVLink-C2C (CPU↔GPU 900 GB/s); ConnectX-8 800 Gb/s external ~$100k+/unit [43][44]
DGX Spark / GB10 Blackwell (GB10) 4–8 units ~1 PFLOP (FP4) 128 GB LPDDR5x @ 273 GB/s ConnectX-7 200 GbE ≈ 25 GB/s $3,499–4,699/unit [20][21]
Mac Studio M3 Ultra Apple M3 Ultra (SoC) 1–2 units ~28 TFLOPS (FP32, GPU est.) 512 GB unified @ 819 GB/s Thunderbolt 5 ≈ 10 GB/s used ~$14.2k/unit [20][45]
AMD Strix Halo (AI Max+ 395) Zen 5 + RDNA 3.5 (SoC) 2 units ~126 TOPS (platform total) 128 GB unified @ 256 GB/s ≤10 GbE RPC ≈ 1.25 GB/s ~$2.6k/unit [20][46]
RTX 5090 Blackwell 1–2 cards ~3,352 AI TOPS (FP4) 32 GB GDDR7 @ 1.79 TB/s PCIe 5.0, no NVLink ~$3k/card (street) [20]
RTX 3090 Ampere 4–6 cards ~285 TOPS (INT8) 24 GB GDDR6X @ 936 GB/s NVLink bridge 112.5 GB/s (2 cards max) used ~$800/card [20]

Specs are from the vendors' official pages [44][45][46]; prices are July 2026 street, cited per row.

Two conclusions fall out of the table. First, decode speed is essentially a function of the per-GPU bandwidth column: the ~8 TB/s GB300 and 1.79 TB/s PRO 6000 pass, the 273 GB/s Spark and 256 GB/s Strix Halo fail — and the compute column barely matters. 20 PFLOPS vs. 1 PFLOP produces nowhere near a 20× gap at batch=1. LLM decoding isn't compute-starved; it's haul-starved.

Second, the interconnect column explains why "just add nodes" is hard: Spark cluster nodes talk at 25 GB/s — an order of magnitude below their own already-modest memory bandwidth. Adding nodes adds capacity, not bandwidth. Same story for the 3090's NVLink bridge, which only joins two cards — a four-card build still rides PCIe. That's the hardware reason the consumer multi-GPU lane hits a wall.

The Bottom Line

Tiered recommendations, by what you're willing to accept:

Your requirement Setup Expected speed Budget
Strict single-stream ≥30, full-model quality 4× RTX PRO 6000 (IQ4_XS) [6] 29–38 ~$50k
Strict single-stream ≥30, REAP pruning accepted 4× RTX PRO 6000 + NVFP4 [5] 45–60 ~$45k–55k
≥30 mostly for coding, tinkering accepted 8× ASUS GX10 + INT4/INT8 + MTP [8] 26–27 sustained / 33–54 peak ~$28k–30k
Aggregate ≥30 with real concurrent load 8× GB10 or 2× M3 Ultra 15–25 per stream ~$28k
~20 acceptable, long-context agent use 4× GB10 (ASUS) [7] 20–22, prefill 500+ ~$14k–16k
~17–23 acceptable, quiet and efficient used M3 Ultra 512GB ×1 or ×2 [11] 16–19 / 23.4 ~$14k / $28k
Just want to experience a 744B model running used EPYC 512GB + 1–2× 3090 [16] 6–11 ~$3k–4.5k
Homo economicus API, or rent GPUs by the hour [20][37] 44+ $3/M tokens or $2–6/hr

Three reminders — the easiest things in this data to miss.

First, the cost of 30 tok/s is nonlinear. 20 tok/s costs $14k, 23 costs $28k, and "a stable 30+" jumps to $45k. The last 7 tok/s cost more than the first 23 combined. Be sure you actually need 30.

Second, this is the worst possible moment to buy AI hardware. The memory crisis put 18% on the Spark [21], 55% on the RTX PRO 6000 [25], and killed the big-memory Macs outright [22]. Meanwhile the M5 Ultra is rumored for October, and if llama.cpp lands GLM MTP, every offload setup gains 50%+. If you can wait a quarter, wait.

Third, don't stare at decode alone — prefill kills. 30 tok/s only describes output. Coding agents routinely stuff 50–100K of context: Mac prefill runs 60–90 tok/s, so first token takes 10–20 minutes — unusable [10]; GB10 clusters do 500–1,200 [8]; RTX PRO 6000 does 5,000 [5]. The spread on this axis is wider than on decode.

And then there's the bill that embarrasses every rig. GLM-5.2's official API is $1.40/M input, $4.40/M output [37]; on OpenRouter, list prices go as low as $2.42/M output, and the cache-weighted average people actually pay is about $0.39/M in, $3.31/M out [38]. The purchase price of a 4× RTX PRO 6000 rig buys roughly 10 billion output tokens on the API.

Subscription vs. API vs. Self-Hosting

Put all three on one ruler. The reference workload is a fully-used Claude Code Max 20x month: official weekly limits are 240–480 hours of Sonnet plus 24–40 hours of Opus [40], and heavy users hit the weekly cap, not the 5-hour window — by ccusage-based token accounting, maxing the weekly limit ≈ $1,100/week of API value [41], and 4.33 weeks make ≈ $4,700/month (most people's felt number is around $4,000). The premise carries over from this whole survey: treat GLM-5.2 as the open-weights counterpart of the Opus 5 tier, and price the same basket of tokens three ways.

Subscription: Claude Code Max 20x API: GLM-5.2, cheapest providers API: GLM-5.2, official list price Self-host: 4× RTX PRO 6000 $200/mo ≈ $500/mo ≈ $1,000/mo ≈ $1,700/mo Monthly cost of the same token basket (a maxed Max 20x month ≈ $4,700 in Opus API value)
Same reference workload, monthly cost (July 2026, rough math). Subscription at list price; API converted at GLM-5.2 rates (Opus list prices are 4–6× GLM's); self-hosting includes electricity and 3-year depreciation, not your time as the ops team.

The math behind each bar:

Option Monthly cost How it's computed
Subscription: Claude Code Max 20x $200 List price. Roughly a 20× subsidy — Anthropic is paying the rest of your bill [41]
API: GLM-5.2, cheapest providers ≈ $500 The same basket at GLM list ≈ $1,000; OpenRouter's cheapest tier runs ~55% of list (CoreWeave $0.76/$2.42, StreamLake/NovitaAI promos ≈ $0.77/$2.41) [38]; with caching (network-weighted actual $0.39/$3.31) it drops further
API: GLM-5.2, official list ≈ $1,000 $1.40/M in, $4.40/M out [37], converted for an input-heavy agent workload (network-wide prompt:completion ≈ 100:1 for this model [38])
Self-host: 4× RTX PRO 6000 ≈ $1,550–1,830 Electricity: 2.2 kW at 24/7 ≈ 1,600 kWh ≈ $300/mo (US residential average 18.8¢/kWh [42]); depreciation: $45k–55k over 3 years ≈ $1,250–1,530/mo. The electricity alone beats the subscription. Running it less doesn't make the depreciation go away

Two electricity reference points: 8× GB10 at full load draws ~1.9 kW, about $260/mo at 24/7; dual M3 Ultra at ~0.5 kW is only ~$70 — Mac efficiency is real money on the monthly bill.

One-line conclusion: for personal use, the subscription wins, open-weights API is second (2.5–5×), and self-hosting is last (~8×, before you price in being your own ops team). The subscription is a ~20× subsidy, the API is the market price, and self-hosting can't even beat the subsidy on electricity. Local deployment was never about saving money — it buys privacy, offline operation, unlimited calls, and the joy of tinkering. I confess I'm mostly in it for the last one.

So, Did I Buy Anything?

Not yet.

I'm waiting on three things: a big-memory M5 Ultra, GLM MTP in llama.cpp, and memory prices climbing down from the crisis. Any one of them landing rewrites half the table above. That's local LLM hardware at its most fascinating and most discouraging: conclusions have a shelf life measured in quarters.

If you can't wait, buy from the recommendation table and skip the middle tier — two RTX PRO 6000s and two 5090s are the worst value positions in the entire field.

The table isn't finished either. If you have new measurements — especially Spark cluster runs with vLLM flags, quant, and context length — post them under the X thread and I'll keep merging new cases in.

References

Source of record: the full research report (23-row data table plus the concurrency re-evaluation) is here, compiled with Manus; community cases are in the X thread. Every entry below links to its original source (as accessible on 2026-07-28):

  1. Z.ai official blog — GLM-5.2: Built for Long-Horizon Tasks (744B total / 40B active, MTP drafting 5 tokens/step, 1M context, MIT license; official Hugging Face mirror)
  2. Unsloth docs — GLM-5.2 – How to Run Locally (memory needed per quant level)
  3. atcyrus.com — GLM 5.2 local hardware requirements: reported paths by quant (community measurement roundup; engine MTP support status and mixed configs)
  4. Hugging Face — 0xSero/GLM-5.2-504B (REAP pruned model card, Terminal-Bench 70.5)
  5. GitHub — 0xSero/glm-5.2-sm120 (4× RTX PRO 6000 REAP-469B NVFP4 vLLM recipe: ~60 tok/s, ~45 at 64k–100k, prefill ~5,100)
  6. Hugging Face — lukealonso/GLM-5.2-NVFP4 discussion (replies report 4× RTX PRO 6000 full-model IQ4_XS: 38 → 35 → 29 tok/s at 0/10k/40k context)
  7. NVIDIA Developer Forums — GLM-5.2 on a 4x GB10 cluster: ~22 tok/s decode, 256K ctx (recipe) (includes the 8× GB10 full-model AWQ-INT4 + MTP 26–27 tok/s replies)
  8. NVIDIA Developer Forums — GLM-5.2-Int4-Int8 on 8× GB10: ~1,200 t/s prefill, 33–54 t/s avg decode (coding-workload averages; deployment patch details)
  9. NVIDIA Developer Forums — GLM-5.2 (unpruned) at 800K max context on 4x DGX Spark (NF3 hybrid) (NF3 mixed recipe, 20–33 tok/s)
  10. Reddit r/LocalLLaMA — High-quality GLM-5.2 Quant on 4x DGX Spark – Guide, Results, and Comps (llamaCTO; DCP4 at 128K context 14.5–15.2 tok/s, MAX_NUM_SEQS=1; 8× RTX PRO 6000 ~106 at bs=1 / ~420 at bs=8; M3 Ultra long-context prefill data; author-mirrored on the NVIDIA forums)
  11. X @ivanfioravanti — MLX GLM 5.2 Distributed on two M3 Ultra 512GB (single machine 18.8 tok/s, dual 23.4; the 16 tok/s lower bound is the llama.cpp figure in [10])
  12. Reddit r/LocalLLaMA — GLM 5.2 on consumer hardware (2× RTX 5090 + 512GB, Q5_K_S, ~12 tok/s)
  13. NVIDIA Developer Forums — GLM 5.2 on 2x DGX Spark (GB10) nodes: 1-bit UD-IQ1_S, RPC llama.cpp, 256K context, ~8 tok/s (correction: earlier versions of this post, following the research report, attributed this to howtospark.com; the actual source is this forum thread)
  14. Reddit r/LocalLLaMA — GLM5.2 744B 2-bit at 7.3 tok/s on 4x3090 + 192GB, and why (the CPU-bottleneck finding: 1-bit runs no faster)
  15. Framework Community — GLM 5.2 on Dual Strix Halo (256GB): Worth it? (llama.cpp RPC, 7.2 tok/s; owner's verdict: not worth it)
  16. Hacker News — GLM-5.2 – How to Run Locally discussion (EPYC 512GB + 2× 3090 at ~6 tok/s, tunable to 9–11; electricity vs. API cost math)
  17. Reddit r/LocalLLaMA — CPU-only GLM 5.2: Epyc and 512GB RAM (EPYC 7452, UD-Q4_K_XL, 1.73 tok/s)
  18. NVIDIA Developer Forums — GLM 5.2 on 1x Spark (single unit via offload: 1 tok/s, 4 after tuning)
  19. vLLM official recipes — zai-org/GLM-5.2 (8× H200/H20/B200 and AMD ROCm reference deployments)
  20. llmrequirements.com — State of Local AI (July 2026 hardware prices, power draw, and rental panorama: ASUS GX10 $3,499, used M3 Ultra market, MI325X/MI355X prices, etc.)
  21. Tom's Hardware — Nvidia DGX Spark gets 18% price increase as memory shortages bite ($3,999 → $4,699)
  22. MacRumors — Apple Cuts More Mac Studio and Mac Mini RAM Options as Memory Shortage Worsens
  23. rentamac.io — Run GLM 5.2 on a Mac: The Memory You Need (256GB machines: 2-bit 3–9 tok/s, 1-bit ~21.6)
  24. X @antirez — GLM 5.2 4-bit on M3 Ultra: "usable — but you literally can't buy the hardware anymore" (quote-tweets [11]; follow-ups discuss the MTP implementation path)
  25. Tom's Hardware — Nvidia raises RTX Pro 6000 Blackwell GPU pricing to $13,250 — 55% increase over MSRP
  26. Reddit r/LocalLLaMA — GLM 5.2, what speeds are we getting locally? (OP's 6× RTX 3090 at 7.8 tok/s, among others)
  27. GitHub — Running GLM-5.2 with SGLang and KT-Kernel (ktransformers official tutorial) (CPU-GPU heterogeneous MoE offload; AMX kernels on the CPU side)
  28. X — @0xSero's reply (4× Spark, 30 tok/s)
  29. X — @JamesMcPherson's reply (4× Spark, 40–41 tok/s)
  30. X — @Dave_Charland's reply (running 4 units; the trade-offs)
  31. X — @siraustin's reply (4× Spark + MikroTik ≈ $20k)
  32. X — @taimurayaz's reply (the fp16 remark)
  33. X — @kristoph's reply (the framing summary)
  34. dnhkng.github.io — 2x GH200 for LLM inference, Part 3: GLM-5.2, expert offload, and the CPU question (batch 1/2/4 aggregate measurements; MTP-vs-concurrency trade-offs)
  35. Reddit r/MacStudio — you probably have no idea how much throughput your Mac Studio is leaving on the table… (M4 Max: MoE 30B 1.89× at 5-way, dense 3B 1.04×; LM Studio's MLX engine lacks continuous batching)
  36. arXiv 2601.19139 — Native LLM and MLLM Inference at Scale on Apple Silicon (continuous batching adds 21–87% throughput)
  37. ofox.ai — Run GLM 5.2 Locally (2026): 2-bit on a 256GB Mac or 4090 box (local vs. API cost; official $1.40/M in, $4.40/M out)
  38. OpenRouter — z-ai/glm-5.2 model page (32 providers' prices and P50 throughput, weighted effective prices, traffic mix; numbers and screenshots are a 2026-07-28 snapshot)
  39. OpenRouter — openai/gpt-5.6, openai/gpt-5.5 model pages (official-endpoint P50 throughput 37–38 / 47–48 tps, 2026-07-28 snapshot)
  40. Anthropic Help Center — About Claude Max plan usage (official description of Max 20x weekly and 5-hour-window limits)
  41. Botfarm — The True Price of a Claude Max Weekly Limit (in API Dollars) (ccusage-based token accounting: a maxed Max 20x week ≈ $1,100 in API value, ~20× subsidy)
  42. U.S. EIA — Electric Power Monthly (US residential average 18.8¢/kWh, April 2026)
  43. pi3g — NVIDIA DGX Station available now for purchase (GB300 channel pricing: from ~$100k pre-tax, OEMs at $99.5k–123k, 4–13 week lead times; 748 GB coherent memory and ConnectX-8 specs)
  44. NVIDIA — official spec pages for DGX Station (GB300), DGX Spark, and RTX PRO 6000 Blackwell
  45. Apple — Mac Studio tech specs (M3 Ultra: 819 GB/s unified memory bandwidth)
  46. AMD — Ryzen AI Max+ 395 spec page (the Strix Halo platform)