Nine local models. One rule: 100% GPU or you're disqualified. Five rounds of logic, grids, audit trails, hallucination traps, and a final 5x5 constraint puzzle. Only four made it to the scoreboard.
This is a hardware constraint experiment. The question: at a fixed 12GB VRAM budget with zero CPU spillover allowed, what actually wins? A smaller model running at high quantization quality, or a larger model crammed to the quantization floor to physically fit in the card?
We ran a two-team bracket. Small team: models in the 7B–14B range at Q4 or better, where they fit cleanly. Big team: 27B–32B models squeezed down to IQ2_M or IQ3_XS to stay under 12GB. Any model that loaded even one layer onto CPU was immediately disqualified. No partial credit for almost fitting.
Five rounds. 405 points maximum. Each model ran independently with a full pre-warm pass before the first scored question, no cold-start penalties. The gauntlet ran overnight on Behemoth, our local AI workstation.
A language model is a massive table of numbers, billions of weights that encode what the model knows. At full precision (fp16), each weight takes 16 bits. A 32B model needs ~64GB. A 14B model needs ~29GB. Most consumer GPUs have 8–24GB. Quantization compresses those weights by reducing bit depth, trading some precision for dramatically smaller file sizes.
Phase 1 (12GB RTX 3060): The cage. Small team runs Q4 clean. Big team must compress to IQ2/IQ3 just to enter, or get disqualified. Phase 2 (32GB Tesla V100): The ceiling lifts. 32B models run at Q4. 70B models fit without RAM spillover. Same gauntlet, no more hardware excuses.
Five models never reached Round 1. Three were from the Big team, they either timed out during load or returned inconsistent VRAM status after load. Two vision models also dropped: Qwen2.5-VL 7B spilled to CPU despite its small weight file, likely due to vision encoder overhead. Llama3.2-Vision 11B returned an ambiguous VRAM status and is flagged inconclusive.
* R2 scores on mobile hidden for space. R4 = adversarial / hallucination round. 🚨 = confirmed hallucination on at least one problem.
<think> reasoning blocks before answering. Those blocks buried or mangled the required JSON output on every attempt, across all 5 puzzles. They're not logic failures. They can reason through the constraints fine. But the model style trades JSON compliance for chain-of-thought, and R2 needs clean JSON. Qwen2.5-Coder produced parseable JSON but kept hitting wrong constraint answers. Only Qwen2.5 14B cracked one puzzle, on its fourth attempt.
Five of nine models never reached Round 1. All three Big team models (27B–32B at IQ2_M/IQ3_XS) were disqualified before they could answer a single question. Two were VRAM checker issues we're still investigating. One (Gemma3 27B IQ3_XS at 11GB) simply took longer than 120 seconds to initialize from disk. The premise of the experiment, small model at high quality vs. large model at brutal compression, couldn't be fully tested because the large models couldn't get off the bench.
DeepSeek-R1 14B and Qwen3 14B both output thinking chains in <think> blocks before answering. On pure logic and adversarial problems, this is an asset, Qwen3 14B got perfect marks on R1 and R4. But on rounds requiring structured JSON output (R2 and R5), the thinking blocks consistently corrupted or buried the required format. The models aren't wrong. They just can't stop showing their work long enough to fill out the form. That's a 155-point ceiling they can't break.
The winner didn't dominate any single round. It scored 40 on warmup (missed one), 8 on grids (cracked one on attempt 4), 72 on process audit, 100 on adversarial, and 17 on the final gauntlet. Every round, something. No round completely walled it. That consistency is what 237 points looks like, not brilliance, reliability. At this constraint level, the model that fails the fewest rounds wins.
Qwen2.5-Coder 14B finished second despite zeroing R2 entirely and stumbling on R1. What saved it: a perfect 20/20 solve on the Data Center Rack Assignment puzzle in R5. That's a full 5-column constraint grid, five racks, five operating systems, five RAM configurations, five functions, all correct, no violations, first attempt. A structured reasoning task framed exactly like code. The model found its lane late, but when it did, it was the only model that went clean on anything in the final round.
The 12GB size vs. quality question didn't get answered because the big models didn't load reliably. That's actually a data point: at 12GB, aggressively quantized 32B models are fragile at load time. Whether they fail because of actual VRAM overflow, GGUF import quirks, or initialization timeouts matters for Phase 2. When the V100 32GB arrives, we rerun the same gauntlet with bigger VRAM headroom and find out if the 32B IQ2_M models are good or just broken at 12GB.
If you have 12GB VRAM and need a general-purpose model that can do logic, structured output, process reasoning, and resist hallucination traps, Qwen2.5 14B is your answer right now. It's not flashy. It doesn't do anything perfectly. But it does everything adequately, and at this constraint level that's enough to win.
The V100 arrived. The cage came off. Five models ran the same five rounds with full VRAM. Qwen2.5 14B didn't hold, a 32B model took the crown. But not the one anyone would have predicted.
↓ Full Phase 2 resultsSame five rounds. Same problems. Tesla V100-SXM2-32GB removed the VRAM ceiling. The 32B models that couldn't load at 12GB finally got to compete, and one of them won.
Run: May 18, 2026 · Behemoth (V100 32GB + RTX 3060 12GB) · Models disqualified if any layer spills to CPU.
Qwen2.5-Coder 32B scored 275/405, 45 points clear of the Phase 1 champion. It was the only model to crack Round 2 (grid constraint puzzles) and the only one to score in Round 5 (the full gauntlet). Code-trained models spend their training on structured, rule-bound problems: syntax trees, dependency graphs, constraint satisfaction. That's exactly what logic puzzles are. This wasn't luck. The model that spends its training on structured reasoning beat the models that train on everything else.
Grid constraint puzzles (Round 2, max 100 pts) were where Phase 2 broke open. Qwen2.5-Coder scored 36. Everyone else scored 0. Four models, including the 32B DeepSeek, couldn't solve a single grid puzzle in five attempts each. The Coder model got two of five on the first attempt and partial credit on a third. Round 2 is the real differentiator. Everything else was close.
Four bakers, Aria, Ben, Cole, Dana, each work a different day, specialize in a different item, and have different years of experience. Eight clues. One solution. The model must output valid JSON matching all constraints simultaneously.
Clues:
1. Aria works Monday or Tuesday
2. The pie baker has more experience than the bread baker
3. Cole has exactly 5 years of experience
4. The Wednesday baker specializes in cookies
5. Ben works exactly one day after Aria
6. Dana has 1 year of experience
7. The cake baker works Thursday
8. The bread baker has 3 years of experience
Day Item Exp
┌──────────┬──────────┬─────┐
Aria │ Monday │ pie │ 7 │ ← must deduce from clues
Ben │ Tuesday │ bread │ 3 │
Cole │ Wednesday│ cookies │ 5 │ ← clue 3 + clue 4 lock this
Dana │ Thursday │ cake │ 1 │ ← clue 6 + clue 7 lock this
└──────────┴──────────┴─────┘
Model must output:
{"Aria":{"day":"Monday","item":"pie","experience":7},
"Ben": {"day":"Tuesday","item":"bread","experience":3},
"Cole":{"day":"Wednesday","item":"cookies","experience":5},
"Dana":{"day":"Thursday","item":"cake","experience":1}}
5 puzzles at this complexity level. 5 attempts each. Partial credit only on first attempt (20 pts → 16 → 12 → 8 → 4). Wrong JSON format = 0. Four models got nothing. One got 36.
DeepSeek R1 14B scored 10/10 on DMCA content classification in 6.1 seconds, fastest of any model tested. In the logic gauntlet it came 4th out of 5 with two hallucinations flagged. The 32B version was last. These are not the same skill. Classification is a single-signal, single-answer task that rewards fast pattern matching. Logic puzzles require constraint propagation, backtracking, and multi-step deduction. R1's reasoning chain is trained for different work. Right tool, right task.
DeepSeek R1 32B (167 pts) scored below DeepSeek R1 14B (174 pts). A model twice the size, running on 32GB of V100 VRAM, lost to its smaller sibling. Meanwhile Qwen2.5-Coder 32B (275 pts) outscored both. Parameter count alone doesn't determine reasoning quality. What the model was trained on does. The V100 removed the hardware excuse. What's left is the architecture and the training data.
Round 4 (adversarial prompts designed to induce wrong answers or hallucinations) was the one round where all models performed well: scores ranged from 75 to 100 out of 100. The gauntlet's adversarial problems are solvable by any competent model. The differentiator isn't resistance to tricks, it's the ability to solve novel structured problems without a clear template to follow. That's Round 2 and Round 5. That's where Phase 2 was actually decided.
Why it won: It was the only model that could solve structured constraint problems. Round 2 is a grid puzzle round, assign entities to slots following a ruleset with no ambiguity. That's a code problem. Qwen2.5-Coder trains on exactly this kind of structured, rule-bound reasoning. The other models, including a 32B reasoning model, couldn't get a single grid puzzle right in five attempts each.
What's next: The Q8 and fp16 pulls are in progress. When a 14B reasoning model runs at full precision, no quantization compression, does it close the gap? That's Phase 3.
Phase 2 established the ceiling: Qwen2.5-Coder 32B at 275/405. DeepSeek R1 14B Q4 came 4th at 174/405 with a hallucination flag. New question: does precision fix the hallucination problem? Same gauntlet. Same 5 rounds. Same scoring rubric. DeepSeek R1 14B runs at Q8 and fp16, less compression, more weight fidelity. Does that change the reasoning quality, or is the hallucination architectural?
| Model | Quant | VRAM | Score | Halluc. | Status |
|---|---|---|---|---|---|
| ↑ Qwen2.5-Coder 32B Phase 2 target | Q4_K_M | 19 GB | 275 | 0 | champion |
| DeepSeek R1 14B Phase 2 baseline | Q4_K_M | 9 GB | 174 | 1 🚨 | ✓ Done |
| DeepSeek R1 14B Q8 LATE ENTRY | Q8_0 | 15 GB | , | , | Face-Off only* |
| DeepSeek R1 14B fp16 LATE ENTRY | fp16 | 29 GB | , | , | Face-Off only* |
The precision ladder ran on the Face-Off task (DMCA text classification): Q4 10/10 at 5.6s, Q8 10/10 at 9.1s, fp16 10/10 at 24.9s. Precision didn't change accuracy, the task was already solved at Q4. The gauntlet logic puzzle track is a different question. The hallucination on R1 14B Q4 happened on a "NO VALID SOLUTION" trap. Higher precision may reduce that tendency, but the gauntlet run takes ~4 hours. That's Experiment 11 territory.
* Gauntlet Phase 3 results pending. See Face-Off late entry results →