Can a machine be surprised? A reader wrote in to ask

A reader wrote in after the last piece to ask a question I have been turning over since: can one of these systems be surprised? Not startled, not interrupted, but surprised in the way a person is when the world does something they genuinely did not expect. It is a better question than it looks, so let me try to earn an honest answer rather than a clever one.

Start with what surprise is in us. You are surprised when reality and your prediction come apart. You expected the top stair and there was no top stair. That jolt is your model of the world being corrected against the world itself, and it carries a feeling, which is the part that matters to us. The feeling is what makes you remember the missing stair for the rest of your life.

Now the machine. A large language model is, under the hood, a prediction engine. It is forever guessing the next piece of text, and it keeps a number for how probable it thought each piece was. When the actual next word is one it rated as very unlikely, there is, in a narrow and real sense, a gap between what it predicted and what arrived. There is even a word for how badly a model was caught out by a stretch of text. It is called perplexity, which is a nicely honest name. So if surprise just means a prediction missing its mark, then yes, something in there is being caught out all the time.

But notice what is missing. The gap is measured, not felt. There is no jolt, no catch of breath, nothing that makes this particular miss matter more than the ten thousand other misses in the same paragraph. The model does not go quiet afterwards and think about the stair. It does not change the way it walks. The number updates and the sentence rolls on. What we call surprise is not really the gap at all. It is what the gap does to the rest of your day, and the machine does not have a rest of its day.

This is the thing that usually gets left out of these conversations, so I will say it plainly. These systems are extraordinary at the measuring and have nothing whatever of the feeling. It is tempting, when a model produces something you did not expect, to say it surprised itself. It did not. You were surprised. The surprise was yours, borrowed for a moment and handed back across the screen, and the machine was exactly as untroubled by the brilliant answer as by the dull one.

I find that worth holding on to, and not as a put down. A tool that can be caught out without ever being troubled is a strange and useful thing. It means you can push it, test it, feed it the odd case all day long, and it will never flinch or sulk or decide it has had enough of you. The flinching is still our department. So is the remembering. When the machine does something that makes you sit up, the part worth paying attention to is not happening in the machine. It is happening in you, and it is yours to keep.

Whether or not machines can be surprised, people will be, and Life After AI imagines five households in 2035 and what AI changes in their homes, jobs, money and friendships.