Cerebras answered the question it gets most often: how do you run a frontier model on a wafer when the chip has 44 GB of memory and the weights take terabytes. In a 22 October post the company walks through the layout on open Kimi K3 — about 3 trillion parameters — and promises 1,000–2,000+ tokens a second.
How layers fit on a wafer
In their scheme Kimi K3 has 69 linear-attention layers and 24 dense ones. Weights are counted at 4 bits, as on Hugging Face; the KV cache at 8. Three trillion parameters come to about 16 GB per layer. A WSE-3T wafer has 44 GB of SRAM: two linear layers sit side by side, a dense layer sits alone and leaves the rest for its KV cache. That is 35 plus 24 — 59 wafers.
The compiler maps layers onto wafers; the runtime ships activations over RDMA over Ethernet — about two microseconds a hop. Weights typically do not move: a layer fits in SRAM, and only activations cross between wafers.
A pipeline, not one request
Each wafer is a station on an assembly line. While station three works a token for request A, station two is already on B and station one on C. A station can take a micro-batch. The ceiling on concurrent requests is the KV-cache room on dense wafers, not a “GPU-style batch”. Cerebras writes separately: look at tokens per second per megawatt at the same generation speed, not batch size.
How to look at it
This is an engineering write-up of their own rack, not an independent bench. The Terminal-Bench and BrowseComp numbers for Kimi K3 in the post sit next to GPT-5.6 Sol — not your production. It matters if you already compare wafer-scale to a GPU cluster and hit weight movement. Everyone else can remember the triad: layer on wafer, activations over Ethernet, a pipeline of requests.
