3th Floor Studios  ·  CAST 2026
LLM Reproducibility Test
temperature · seed · determinism
— nondeterministic — reproducible
TEMPERATURE
Controls how the model picks the next token. At 0.8, it rolls weighted dice — the same question gets a different answer every run. At 0, it always picks the highest-probability token. No dice. No variance.
SEED
The starting number for the random number generator. The RNG isn't truly random — it's deterministic math. Same seed = same sequence of rolls = same output, every time. Without a pinned seed, every run is a fresh shuffle.
WHY IT MATTERS FOR TESTING
You can't file a bug you can't reproduce. temp=0 + seed=42 is the AI equivalent of a stack trace — it turns "sometimes it says something weird" into a test case you can re-run tomorrow and hand to a vendor.
NONDETERMINISTIC
Temp 0.8
Seed none fresh each run
temperature: 0.8  ·  no seed  ·  8 runs — watch for drift
Outcome Tally
REPRODUCIBLE
Temp 0 locked
Seed
temperature: 0  ·  seed: 42  ·  8 runs — all identical
Outcome Tally
This file is a reproducible bug report for an LLM. Clip it. File it. Hand it to a vendor.