Even a detector that cannot learn anything falls into the same shortcut: the problem is in the eye, not in the learning

Beginner track

Where we left off

Since Chapter 11, an uncomfortable fact has been sitting at the centre of this whole campaign,

never repaired: the planner's compass — the one comparing an imagined story against a memory of

success — points backwards once it leaves its training ground. Every attempt since

(Chapters 12 to 16) has built something around that problem — a memory of visited places, an

anti-drowning reflex, an exploration prediction — without ever tackling the compass itself. This

chapter tells the story of the first two attempts aimed directly at the heart of the problem.

Neither works. But together, they teach something important about where this defect really

hides.

First idea: don't repair the compass, learn not to trust it

Instead of correcting the compass's biased judgement, this first idea tries to route around it

differently: build a separate detector whose only job is to spot "this image looks like something

the model never saw properly during training" — and if so, ignore the compass and hand over to the

memory of visited places (Chapter 12), which has already proved itself.

What is distinctive about this detector is that it does not train at all, in the sense machine

learning usually means. No neural network, no gradient descent (the usual method by which a model

gradually adjusts its internal settings to improve a score). It simply measures, once and for all,

what the typical statistical shape — the average and the spread — of the training images looks

like in the compressed space (the embedding, the compact representation the model builds from

an image) that the model already uses. Then, for any new image, it simply computes how far that

image sits from this typical shape. Further away = more suspicious. It is a fixed mathematical

formula, not something that "learns" in a way a misleading shortcut could quietly slip into.

This detector was put through three tests, on the same frames already used in Chapter 11 to prove

the compass was inverted:

1. Can it tell known ground from unknown ground? Almost, but narrowly missed — the difference

between the two categories was not sharp enough.

1. Does it specifically spot the frames where the compass got it wrong, rather than just "any

somewhat different image"? Missed more clearly — the detector was reacting to "this is a

different setting", not to "this is where the judgement is false".

1. Does it get distracted by an unrelated detail, like the plain brightness of the image?

Clearly missed, and this is the most important of the three results: the detector tracked

brightness almost as much as the previous failed attempts in Chapters 10, 14 and 15.

Why this third failure matters more than the five before it

Here is this chapter's most important point. Five times before this one (Chapters 10, 11, 14

counting both its halves, and 15), a trained system was caught out by scene brightness instead of

genuinely judging whether a tree was present. One could have thought the problem came from the way

those systems learned — an easy shortcut that training would always end up finding, like a path

of least resistance. But this new detector learns nothing at all. It is a fixed formula, with

no adjustable settings, with no way to "cheat" by finding a shortcut during training — because

there is no training.

And yet it falls into exactly the same trap.

That completely changes the most likely explanation. The problem is probably not "our training

methods always find the same bad shortcut". The problem runs deeper: the confusion between "there

is a tree close by" and "this is dark" appears to be already present in the very way the base

model compresses an image into a compact representation — before any additional learning is

involved at all. Any calculation built on top of that representation, however careful, inherits the

same defect, because the defect is in the raw material, not in how it is worked.

Second idea, more modest: does the problem come from a shortage of dark photos?

This chapter's second line revisits an old hypothesis never directly tested, dating back to the

very start of this investigation (the brightness shortcut fixed — unsuccessfully — in Chapter 10):

what if the model confuses darkness with the absence of a tree simply because it had almost never

seen dark scenes during its very first training?

The check is simple, with nothing trained: count, in the original training data, the proportion of

dark or underwater frames. Result: barely 1% — so the lack-of-diversity hypothesis was

well-founded, for that original model.

But Chapter 14 had already retrained a version of the model on a much broader data mixture,

including many scenes from the real crafting environment — and this check shows those data already

contained 16 to 22% dark or underwater frames, even counting generously. The odd frame spotted in

Chapter 14 (the one whose score got worse after retraining) was therefore not an extreme case never

encountered: it resembles many other frames already present in the training data. Conclusion: for

this precise case, missing dark photos is no longer a plausible explanation. If anyone ever picks

up repairing the base model, the most promising direction is probably not "more data", but

something in the way training compares different scenes with one another — a point nobody has

tested yet.

What this changes for what follows

These two negative results, taken together, close one question and clarify another:

• Sixth independent confirmation of the same brightness trap — after a small trained module,

that same module retrained, an artificial lighting variation, a giant off-the-shelf model, a

hand-built colour calculation, and now a plain statistical formula with no learning at all. Six

completely different ways of going about it, one and the same wall.

• The "not enough dark data" direction is closed for the precise case that motivated it — not

because more data would never help in general, but because, in this precise case, there was

already enough.

• The project therefore faces a clear choice, not yet settled: either retrain the very core of the

model with a different training objective — a heavy piece of work, never attempted so far — or

accept this defect as a known project limitation and keep building on what already works

(executing plans for longer, the memory of visited places, the anti-drowning reflex). This is not

yet decided, and this chapter does not decide it on anyone's behalf.

As with every chapter on this site: two real attempts, two honest failures, and a result that, even

though negative, teaches something solid about the nature of the problem.