Several Breakthroughs Away

The closest I came to the future in 2021 was a waiting list I never joined. I was new to the field then, and a colleague told me about a model you could reach through an API that would, he said, answer almost any question you put to it. There was a queue for access; he may have signed up, but I told myself it would take a while and never got around to it. I did skim the announcement post, and I remember the thought that crossed my mind before I filed it away: once the masses find out about this, it is going to be huge. Then I lost interest and went looking at other things. That model was GPT-3, and what stays with me now is not the model but the shrug: I saw a corner of what was coming, labelled it correctly, and still felt nothing move. It would be nearly two years before I understood that the ground had shifted anyway, and that the thing which shifted first was not the machine but what a small number of people had let themselves believe was possible.

Read More

How Robots Learn: From Classical Control to RL to Robot Foundation Models

In 1988 the roboticist Hans Moravec noticed something embarrassing about artificial intelligence: it is “comparatively easy to make computers exhibit adult level performance on intelligence tests or playing checkers, and difficult or impossible to give them the skills of a one-year-old when it comes to perception and mobility.”1 Nearly four decades later that paradox still defines robotics, but the way the field attacks it has been completely rebuilt, twice. This post traces that rebuild: how hand-engineered control gave way to reinforcement learning, why reinforcement learning then lost the manipulation battle to a much older idea (copying people), and how the last four years fused everything into robot foundation models that look strikingly like their language-model cousins. The goal is a self-contained tour: enough math to see why each transition happened, real numbers from the papers that drove it, and a running experiment you can reproduce.

  1. Hans Moravec, Mind Children (Harvard University Press, 1988). Steven Pinker later compressed the same observation into “the hard problems are easy and the easy problems are hard.” The usual explanation is evolutionary: perception and motor control encode roughly a billion years of optimization, while chess-style abstract reasoning is a recent, shallow layer. 

Read More