The wall is behavioral: one real success, then four proofs that refining the signal is not enough — and a fifth that confirms the diagnosis

Beginner track

Where we left off

Chapter 7 ended with a frustrating observation: sharpening the "am I lost?" signal (by giving the model more examples of situations where one is truly lost) did not change the final result at all — still zero logs chopped from a random starting point. This chapter tells the rest of the investigation: a real discovery that finally unlocks a first result (not huge, but real), followed by a series of additional attempts that each fail in a different and instructive way, until a clear diagnosis emerges — and then a brand new attempt that tests this diagnosis itself directly, with a negative but highly informative result: the wall is not in perception, it is in behavior.

The discovery: the planner was throwing its own good plan in the trash

While rereading the planner's code line by line (the one from Chapter 4, which imagines 512 stories of 12 actions and keeps the best one), a bug appeared — not a bug that crashes the program, but a silent bug in the logic itself. The planner correctly imagines a 12-action story, correctly identifies the best one... and then executes only the very first action, before starting all over from scratch on the next turn. Even when the imagined plan was excellent — "turn toward the trunk, step forward, strike, strike, strike" — the agent only executed the first step of this plan, threw away the next 11, and drew 512 completely new stories on the next turn. A good sustained gesture was thus systematically diluted into a sequence of isolated, independent decisions.

The fix is tiny: execute the first 4 actions of the best found plan, instead of just one, before replanning. Over a combined batch of 31 episodes (several grouped runs), this single fix produces 3 successes out of 31 (about 10%) — whereas all previous attempts from this chapter and the previous one combined (27 episodes in total) had produced zero successes. It is not a huge number, and it is not statistically solid yet at this sample size — but it is the very first non-zero result in the entire history of the project on this specific problem.

The lesson is amusing and a bit unsettling: a notation bug and a reasoning bug look the same from the outside. The previous chapters had spent a lot of time improving the quality of the signal used by the planner (sticky sampling, search reflex, data refinement) — all of this was real and necessary, but none of it touched the real flaw: the planner was choosing correctly, but then only acting on one-twelfth of its own choice.

A reinforcement attempt: curiosity, again, but "live" this time

Emboldened by this small success, the project tried adding curiosity (Chapter 7) back in, but in a different version: instead of training the committee of guessers once and for all on frozen recordings (which had caused its collapse), it is trained continuously, while the agent plays, on what it actually sees, moment to moment.

First launch: three episodes out of seven crashed due to a real technical bug (fixed along the way). On the four remaining valid episodes, zero successes — but with so few episodes, this is not surprising in itself (the expected success rate at this level, on only 4 tries, would be less than half a success). The real problem, discovered after the fact: the diagnostic that would have told us if the curiosity signal was working hadn't even been recorded. A simple lesson but worth repeating: a diagnostic tool that exists in the code but that you forget to enable is equivalent to having no tool at all.

Once this setting was fixed and the diagnostic enabled on a new batch of six episodes, the answer became clear — and negative, but in a precise and interesting way: the "novelty" signal starts relatively high at the beginning of each episode, and then slowly and steadily decreases until it becomes ten to sixty times weaker, and this regardless of what happens on screen. The most visually striking moment of an entire episode (a tree canopy scene, the "I see something important" signal at its highest) coincides with a curiosity signal at its lowest. The cause: the curiosity system's short-term memory fills up very quickly with visually similar images (the agent often stays in the same small visual area), and its predictor gets used to them in a few hundred instants — after which, nothing surprises it anymore, regardless of what actually matters. The signal follows elapsed time, not scene content. The project stopped this avenue without launching a larger batch, the mechanics of the problem already being clear.

Second attempt: reactivate the search reflex, but wired correctly

Chapter 7 had disabled the "turn your head when you are lost" reflex for the crafting mode, because its threshold didn't match the right model. But meanwhile, the two-brain agent had been introduced: the one searching for the tree still uses the old model, the exact one where the reflex had been correctly calibrated. This experiment therefore reactivates the reflex, exactly where the calibration should be valid.

Result over 7 episodes: still zero successes. And the most telling sign: in the episode where we can best observe what is happening, the agent goes through a long period (nearly 900 instants) where the "I see something important" signal is indeed high — comparable to the best canopy scene from Chapter 5 — and the search reflex correctly remains inactive during this whole time (it knows it is not lost). And yet, the agent still chops zero logs during this entire period. This is the clearest possible replica of Chapter 7's observation: correctly wiring the "I am lost" detector is useless if the real problem is converting "I see something" into "I act on it".

Third attempt: a more "intelligent" planning method

The project's bibliography recommends a more refined planning method than the simple random draw of Chapter 4: instead of drawing 512 stories once, draw a first batch, keep the best ones (the "elites"), then do a second draw centered on what the elites have in common, and repeat several times. The idea: progressively converge towards better imagined stories rather than settling for a single raw draw.

Result over 8 episodes: zero successes, and above all, a real visible regression in behavior: the agent starts repeating a single action much more extremely than usual — on average, the most frequent action takes up twice as much time as in normal episodes, with one episode where the agent strikes empty air, motionless, 89% of the time.

The explanation fits in one sentence: this method needs a signal that truly distinguishes good stories from bad ones in order to converge towards the good ones. When no tree is visible (the most frequent situation in a cold-start), the signal is almost flat — the small differences between stories are just statistical noise. A method that refines its draw over iterations mistakes this noise for a real signal and clings to it stronger and stronger at each iteration, instead of remaining varied like a simple random draw repeated at each turn does. The more "intelligent" the method is, the more confidently it gets it wrong when there is, fundamentally, nothing to learn from the situation.

Fourth attempt: train a real distance rule

Last avenue of this campaign: until now, the distance used to compare an imagined story to the goal was a simple raw mathematical distance between two embeddings — never trained to represent something precise like "how many actions actually separate me from the goal". This attempt trains an extra small function so that the distance in this transformed space more closely approximates the true number of necessary steps.

Tested offline first (before spending real gameplay time), this new distance rule passes the test with flying colors: truly close pairs of images get a distance much smaller than truly distant pairs, with a gap of about 8 times — well above the required minimum threshold. A good sign, very different from the flat collapse observed with curiosity.

But in real gameplay, over 6 episodes: still zero successes. And analyzing the images reveals a precise and novel explanation: the trained distance signal reacts very strongly to the brightness of the scene (daytime, twilight, inside a cave) rather than to the actual proximity of a tree — and not in a simple way: the darkest images of the whole episode yield, paradoxically, the lowest signal, not the highest. The cause: neither the expert recordings nor the random episodes used to train this distance rule contain real nighttime or cave scenes — this rule therefore never learned to distinguish "it is night" from "I am far from the goal", two very different things in the real game. A flat signal (like curiosity) fails cleanly, without breaking anything; a confidently wrong signal can actively mislead the planner towards what looks close without being close — worse, in a sense, than simply lacking a signal.

Fifth attempt: put good gestures on the menu, and a maneuver to cover ground

The diagnosis sketched after the fourth attempt offered a precise explanation: the problem is not that the planner judges poorly, it's that the right gesture is almost never proposed among the 512 stories it imagines. Two ideas to directly fix this were tested together, on top of the commit_length=4 fix already in place:

• Put ready-made good gestures in the list: instead of drawing the 512 stories entirely at random (with or without "sticky"), about 90 of them are now hand-written gestures — sustained forward-striking, continuous camera rotation, walking backward.
• A cruising maneuver: when the "I am lost" signal remains flat for too long, instead of turning its head in place (the reflex seen earlier, which cannot find anything where there is nothing), the agent sprints straight ahead and jumps over obstacles for a limited time, before handing control back to the usual planner.

Over 8 episodes: still zero logs, zero planks, zero reward. But the two mechanisms did work for real, not just get added without effect: the ready-made forward-strike gesture was indeed chosen by the planner in 3 out of 8 episodes (up to about half the playtime in one episode); the cruising maneuver triggered in one episode, where it occupied about a quarter of the time. So this is not a case where the new tools sit idle unused.

The really interesting result of this attempt is not the zero, it's this: in 3 episodes out of 8, the agent started repeating a single action between 83% and 100% of the time — an almost total lockdown on a single gesture. This strongly resembles what happened with the more "intelligent" planning method of the third attempt (the CEM), which also produced extreme lockdown on one action when the signal was flat. Caution, however: this resemblance is noted here as an observation, not yet as proof that it is exactly the same mechanism — deeper verification is underway before stating it as established fact.

This result refines the diagnosis once again: when the "how good is this imagined story" signal is flat (no tree in sight), the planner has nothing to correct itself with. Giving it pure random noise (previous attempts) just makes it fidget in place aimlessly. Giving it a concentrated menu of good candidates (the CEM) or ready-made continuous gestures (this attempt) makes it, conversely, lock blindly onto one of them — because nothing, in a flat signal, ever makes it change its mind. Hand-writing good gestures or refining the drawing method attack the problem from opposite ends, but neither gives the planner what it truly lacks: having learned, from real games, how a player actually searches when they see nothing interesting — rather than being handed a small frozen menu of gestures.

The diagnosis that holds, after these five attempts

Four independent attempts each tried to improve the quality of the signal used by the planner to judge where to look or go: live curiosity (flat), properly wired search reflex (useless in the face of an actually correct signal), a more refined planning method (counter-productive on a flat signal), a trained distance (wrong outside its training zone). A fifth attacked the generation of candidates directly rather than their judgment — two zero-cost ideas, tested together — and failed too, but in a way that confirms and refines the diagnosis rather than challenging it: the two new tools worked fine, and the agent still locked blindly onto a single gesture in several episodes. The only modification that ever produced a real non-zero result, in this entire chapter and the previous one, remains the Chapter 8 fix which changes nothing about the quality of the planner's judgment — it only changes how much of its own good plan it actually executes.

The now established diagnosis: the wall is not that the world imagined by the model is bad — it already knows how to judge a situation correctly, as proven by the offline test of the trained distance. The wall is that the stories imagined at each turn, whether drawn at random, sticky, refined by CEM, or partially pre-filled with hand-written gestures, almost never contain the right learned gesture to truly search and approach a tree — and when the proposed menu is too narrow or too concentrated, the planner locks onto it instead of continuing to explore. It is a problem of generating candidate actions, not of judging those candidates — and the most likely path forward is no longer "hand-write better gestures" but "learn these gestures from real games". Chapter 9 explains where this path stands, and the two complementary ideas still untested at this stage.