Inside the transformer an interactive atlas of LLM architectures. Click any outlined block to zoom in

Model
click to zoom · esc zooms out · c compares
zoomable block live values / tokens frozen weights

Step through an illustrative next-token path

Click a token chip or press Step through to follow a single token through every stage. Token segments, scores, and percentages are illustrative; this page does not load model weights or a model tokenizer.

Number formats

Every weight in a model is stored in a numeric format. Fewer bits usually means less weight memory; speed depends on hardware and kernels, while quality depends on the quantization method. Select a format to see the trade-offs below.

sign (S) exponent (E): dynamic range mantissa (M): precision integer magnitude (I)

What happens to the weights

64 illustrative weights, snapped to the selected format's grid. Gray dots are the originals; purple dots are the stored approximations.

Distinct levels
-
values the format can represent here
Rounding error
-
RMS error vs. original weights

What happens to the size

Weight memory only (KV cache and activations come on top). The top option sizes whatever model you're exploring — paste a config.json to size any model. MoE bars use total weights, not the active subset.