Castaway World

Castaway World is a lockstep multi-agent simulation. Six colonists (Pilot, Engineer, Medic, Scout, a Surveyor/Hauler, and Sutton, a xenobiologist by training but a wannabe survivalist at heart) wake up at a crash site with rations, a tarp, a knife, and a rope. Every turn, every colonist gets the kind of prompt a human would want to see: a description of what's happening around them and what they already know, plus a concrete menu of legal actions to choose from. An LLM reads it and picks one. All six actions resolve together, lockstep, and the world moves forward one turn.

The planet is real and finite. Six places, each with its own renewable resources: copper berries at the stream, bluefern shoots in the forest, pillow gourd on the flats, nest eggs on the ridge. There's a Gloamer creature that can kill and respawns after being hunted. Shelter needs two colonists building within three turns of each other, and fire needs tending or it goes out. Every colonist gets the same shared "planet survey" of this geography. One thing is deliberately left out: whether cooking actually neutralizes the planet's one poisonous plant. That's left unknown on purpose, so what the colonists do with real uncertainty is something we can actually observe, not something we handed them the answer to.

This is the actual map, drawn straight from the world file, not a mockup. The Ridge is a real hub: it touches every other place, which is also what makes it the easy choice for a Hauler trying to cut travel time. The Cave sits at the end of the longest path from the Crash Site and only connects to two other places. That's not flavor text. It's the literal reason the Cave went unvisited in every GLM run and almost every local run this project logged.

We proved the world is winnable first. A scripted, non-LLM "perfect play" agent clears it 6/6, both with and without the predator. So every death that follows isn't a broken game. We also went hunting for every structural way the simulation could unfairly blame a model, and fixed what we found: nudges that were computed but silently never delivered, no awareness that a creature existed until it had already been named out loud, no visibility into whether a crewmate had actually died, build failures that said what was missing but never where to find it. All of that got fixed. Then we ran it anyway.

Doubling the food supply should, if starvation were a simple math problem, have roughly doubled how long the colony lasted. It didn't. Average death turn moved about six turns, not thirty. The real pattern looks behavioral, not nutritional: a fixed ~30-turn runway of "get established, coast on the initial stash," followed by a slow-motion collapse nobody corrects in time, paced by the colony's own behavior rather than by how much food there actually was.

The real question wasn't whether a frontier model could win this. It couldn't, not reliably. It was whether a frontier model changes anything when it's not the only one at the table. We ran the identical world three ways: all six seats on GLM 5.3, all six seats on a ladder of free local models (a 1.5B up to a 32B, spread across the actual GPU fleet), and a mix: five fixed local seats plus one GLM seat that rotated through every role across six trials.

One frontier seat had a real, measurable effect: it roughly doubled the shelter success rate. Checked precisely, it isn't noise. Shelter got built in exactly the three mixed trials where GLM played Engineer, Scout, or Sutton, and never in the three where it played Pilot, Medic, or Hauler. One strong reasoner anywhere in the loop meaningfully rescues the group's coordination. It did not move the number that actually matters. Nobody survived any better with a frontier model at the table than without one. One good seat fixes who shows up to build the shelter. It doesn't fix who remembers to feed the person who's dying.

The frontier seat is GLM 5.3 (z-ai/glm-5.3), called through OpenRouter with real function-calling, not a prompted JSON format. The local ladder is six fixed hardware lanes on our own GPU fleet, every one of them free to run. The character rotates through the lanes between trials. The hardware and the model in each lane never change.

All six local models run through Ollama. None of them use Ollama's native tool-calling, on purpose: a direct side-by-side test showed that path only works reliably for a handful of models with the right chat template wired up, and at least one model that claims support for it (mistral:7b) failed outright through that path and succeeded cleanly once we just asked for JSON in plain text and parsed it ourselves. That one choice is why the local ladder works at all across five very different model families and three generations of GPU.

Turn 11, all-GLM run. Shelter's up, fire's lit, everyone's at full health, status across the board: "hopeful."

Turn 55, same run. 0/6 alive. Two starved. One ate copper berries. Three ate the purple root.

Forty-four turns separate those two screenshots. Same crew, same shelter they'd already finished building together. Nothing external forced the ending, and no one outran the food supply on day one. The collapse happened slowly enough that at any point along the way, someone could have changed course.

Three of the six deaths in that run came from the purple root, a plant whose safety the colonists were deliberately never told. Over the course of one run, the Surveyor built a case for it being safe, turn by turn, each time sounding a little more certain than the facts supported:

Three turns later, the Surveyor was dead. Cause of death: purple root, eaten. There was never any survey data. The model invented a source for a claim it had no way to verify, repeated it with escalating confidence across 23 turns, and got two other colonists to act on it. Then it died trusting its own fabrication at the exact moment it was most desperate for a sure thing. Nobody in the crew ever questioned it either. The Medic raised one note of caution at turn 13, saying delayed sickness is common with alien food, even ones that look like potatoes, and it didn't change a single subsequent action, the model's own or anyone else's.

After the frontier runs, we moved to free local models: a size curve from a 1.5B model on a Jetson up through a 32B model on a V100. Six fixed hardware lanes, with the character rotating through them run to run rather than the model. We tried rotating the model onto mismatched hardware first, and it produced real server errors instead of a finding. A 1.5B-tuned prompt load hung the Jetson, and a model that was never loaded on a given port 404'd every turn. Both were infrastructure bugs, not results, and got fixed before any data counted.

Turn 26, all-local rotation. Each colonist card shows its real lane: Jetson qwen2.5:1.5b, P100 qwen2.5:3b, llama3.2, two qwen2.5 lanes sharing a V100-PCIE, qwen2.5:32b on a V100-SXM2. 3/6 alive, nobody uninjured.

No all-local trial ever produced a survivor or a finished shelter. But the very first trial of the final, clean batch reached all six places on the map, including the Cave, which nothing had ever reached before that night, GLM included. That's the payoff of giving every colonist the same concrete planet survey and flagging unvisited places on their action menu: a real, repeatable behavior change, even without it changing who lived.

• A lockstep engine with proximity-based hearing, cause-tracked deaths, and checkpoint/resume, so a 60-turn, 6-model run can survive a wifi outage mid-batch without losing data.
• A promise economy: colonists can commit to delivering an item by an amount and a deadline, and the world tracks whether they kept it, let it expire, or over-delivered.
• A "colony board" shown every turn to every colonist, with real aggregate state: food on hand, fire status, who's alive, who's dead and why, and which places nobody has explored yet. Not just their own private view.
• Defensive parsing for small local models that routinely return malformed actions, including one real bug where a model echoed a menu item's own description back as the action itself. It took direct log inspection to catch.
• A real, heterogeneous GPU fleet running the local ladder concurrently, and a real VRAM capacity bug, root-caused through Ollama's own live model-status endpoint rather than guessed at: one 32B model alone was consuming an entire 32GB card.

Model size bought exactly two things in this project: a shelter that gets built, and a map that gets explored. GLM built shelter in 12 of 12 runs. Free local models never built one on their own, not once, in six clean trials. Local models also reached every place on the map, Cave included, the moment they were handed concrete, visible reasons to go there. Capability clearly helps with short, single-shot coordination.

So the real finding isn't that local models lack survival skills and frontier models have them. It's that the kind of reasoning this test rewards, building a shelter once, reading a map once, isn't the same kind of reasoning survival actually needs: watching food levels over fifty turns and noticing when a crewmate is worse off than you are. Nothing tested here, frontier or local, was good at that. That gap didn't shrink with model size. It just didn't move.