Metacognition is the unlock

The big challenge for traditional LLMs is that they are path-dependent; while they can consider the puzzle as a whole, as soon as they commit to a particular guess they are locked in, and doomed to failure. This is a fundamental weakness of what are known as “auto-regressive large language models”, which to date, is all of them.

That’s Ben Thompson. Getting out of that trap is what each stage of model progress has chipped away at, and where you are in that progression sets the ceiling on what a model can do without you standing over it.

The first stage was reactive. Early chat models answered in one shot: prompt in, text out, nothing in between. They were fluent and often right, and they couldn’t catch themselves mid-thought because there was no middle to the thought. The whole answer arrived at once.

The second was reasoning. Models learned to work step by step before answering, and to lay out the working rather than just the conclusion. That’s a real jump. A model reasoning through a problem catches errors that a reactive model just commits, because the steps are visible to it as it produces them. Reasoning buys you reliability on anything with a chain to it.

The third is agentic. Harnesses and agents break a goal into steps and sequence the work: do this, then that, check the result, move on. That’s what turns a model that answers questions into a system that finishes tasks, holding a goal across many calls without a human threading each one.

The next one

The next stage is metacognition, which is thinking about its own thinking. Reasoning about the approach rather than through the task. Noticing that a line of attack is failing and changing it. Noticing it’s hit the same blocker three times and trying something else. Treating a failed attempt as information about the strategy instead of something to retry word for word.

The difference is easy to see once you’re looking for it. An agentic loop that hits an error retries the step, maybe with a tweak, and retries again. It’s working hard inside a frame it never questions. A metacognitive agent asks whether the frame is wrong. It can judge the strategy that produced the step and pick a different one. The first can loop forever. The second can notice that it’s looping.

In people this is most of the game. Two people make the same mistake. One files it under bad luck and makes it again next week. The other works out the rule, changes how they operate, and never hits that class of problem the same way again. The second person is better at learning from the moment, rather than smarter in it.

Where you trigger it

An agent loop usually has an observe step, a decide step and an act step. Metacognition adds a fourth: learn. That’s the seam it lives in, and the useful part is that you can trigger it deliberately rather than waiting for it to emerge on its own.

After an attempt, before the next one, make the agent answer a different kind of question: “what did that attempt tell me, and should I change my approach?” instead of “what’s the next action”. It’s cheap to add, since it’s a prompt and a place in the loop to run it, and the effect is out of proportion to the cost. The agent stops treating every failure as a reason to retry and starts treating some failures as a reason to rethink. That gives it a moment to be honest about whether the current plan is working, and permission to drop it.

Current models don’t have much of this yet. They’ll cheerfully retry a doomed approach, declare success on work that failed, and miss the pattern sitting in their own errors. It’s arriving rather than here. You can see the shape of it, though, and you can scaffold towards it in the loops you build today. The learn step is where you reach for it before the models get there on their own.

A reactive model answers, a reasoning model works through, an agentic one sequences. A metacognitive one notices it’s about to make the same mistake again, and stops.