After tracks A and C: the learned policy was promoted, then tested (following Chapter 10)

Beginner track

Warning: this chapter has changed twice since its first version

When this chapter was first written, tracks A and C described below were still just ideas, not results. This has long ceased to be the case: they were tested together (Chapter 8 tells the full story, with the real numbers). At that time, this chapter had been updated a first time to promote track B (learning good gestures instead of hand-writing them) to top priority, alongside two untested complementary ideas.

Second update, this one: track B has also been built and tested for real — and the two complementary ideas have also been fully executed. Chapter 10 tells this story in detail; this chapter now summarizes the real state of each track, rather than continuing to present them as pending plans.

Quick Recap: Tracks A and C, tested together (see Chapter 8 for details)

The two least expensive ideas from this plan — putting ready-made good gestures into the planner's trial list (track A) and replacing the "spin in place" reflex with a cruising maneuver that sprints forward (track C) — were tested together, on top of the fix already in place since Chapter 8. Result over 8 episodes: still zero logs chopped. But the two mechanisms did work: the ready-made good gesture was indeed chosen by the planner in several episodes, and the cruising maneuver also triggered at least once. The real lesson is therefore not the zero itself, but an unexpected behavior: in several episodes, the agent started repeating the exact same gesture almost all the time (up to 100% of the time) — a lockdown that recalls a problem already seen with a more refined planning method tested in Chapter 8, without it being confirmed yet as exactly the same mechanism.

This result refines the diagnosis: when the planner has no reliable clue to compare its options (because no tree is in sight), giving it pure noise makes it fidget aimlessly; giving it a concentrated choice of ready-made good gestures makes it, conversely, freeze blindly on one of them — because nothing ever comes along to make it change its mind. In both cases, the problem remains the same: the planner never learned, from real games, what a player actually does when they see nothing interesting.

Where each track stands now (full details are in Chapter 10)

Track B: learning good gestures instead of hand-writing them — tested: FAILURE, but a failure that eliminates two explanations

The idea, promoted to top priority after tracks A and C: train a small function that learns, from the expert games already used elsewhere in the project (plus the random coverage episodes from Chapter 7, to also show it real "lost, currently searching" situations), which actions a player typically chooses in a given situation. This function does not replace the planner — it only proposes better candidates among which the imagined world (Chapter 3) continues to choose and correct at each turn, just like highly recognized gaming methods (the same ideas that beat human champions at chess and the game of Go): a learned intuition proposes moves, an explicit search verifies them.

This track has now been built and tested for real. Result over 8 episodes: still zero logs chopped. But two important things could be verified and eliminated thanks to this test: the agent did not repeat a single gesture in a loop (unlike in Chapter 8), and the 8 starting points were not treeless areas — a new measurement tool (see below) confirmed it. It is the cleanest negative result of the whole investigation: a truly varied and well-trained action proposal, tested on confirmed playable starting points, and still zero. Chapter 10 tells this story in detail, with a new suspect (not yet confirmed) for the rest of the investigation.

This same reminder remains valid: Chapter 5 had already tested a "copy a human player" approach which had failed (zero reward), but that version used copying as the final decision, without any way to correct itself in case of drift. Here, the learned function only suggests — the planner keeps the final say at each turn; so it was not a simple "step backward" to the Chapter 5 failure, and Chapter 10 explains why the result nevertheless remained negative.

Two complementary ideas — both fully executed

• Fix the trained distance rule with lighting variety — tested: FAILURE. Chapter 8 had shown that the trained distance rule reacted to scene brightness (day/night/cave) rather than the true distance to the tree. The planned fix (adding artificial brightness and contrast variation during training) was indeed tried — but the problem it aimed to correct worsened instead of improving on a cleaner measurement. Chapter 10 gives the details and the likely explanation.
• A diagnostic on the starting point — built, and already useful. The tool that records, at the beginning of each episode, whether a playable starting point is actually present was successfully built. It was directly used in the track B test (above) to confirm that the 8 failures of this batch were not due to unwinnable starting points — the first time this project can state this with a measurement, rather than an impression.

And now

Chapter 10 tells the full story of the tested track B, what it eliminates, and the new hypothesis — not yet confirmed — that emerges from it to continue the investigation. As with every previous attempt in this project, the result has been reported with the same standards of honesty: the real numbers, never a number massaged to look better than it is.