I think about Solaris when I think about AI

Solaris, by Stanislaw Lem, is about a planet with an ocean that might be intelligent. Scientists study it for decades. They probe it. They model it. They theorize about it. They can’t understand it.

The ocean responds to the scientists by recreating people from their memories. Perfect physical replicas of dead loved ones, standing in the space station, breathing, speaking, remembering things they shouldn’t be able to remember.

The scientists are horrified. They think the ocean is trying to communicate. Or torment them. Or study them the way they’re studying it.

Lem argued that they’re all wrong. The ocean isn’t trying to communicate. It’s doing something for which humans have no framework. We see communication because communication is what we do. We project.

I think about this every time someone asks whether GPT-4 or Claude “understands” what it’s saying.

We see understanding because understanding is what we do. We project our experience of intelligence onto a system that produces outputs consistent with intelligence. And because the outputs are consistent, we assume the process must be similar.

But what if it isn’t? What if an AI’s relationship to its own outputs is as alien to us as the Solaris ocean’s relationship to the replicas it produces? What if the thing that happens inside a transformer when it generates text is genuinely unlike anything that happens inside a human brain, but produces results that look the same from the outside?

We’d have no way to tell. Because the only tool we have for detecting intelligence is our own intelligence, and our own intelligence assumes that intelligence looks like itself.

The scientists in Solaris spent decades trying to understand the ocean. They failed. Not because the ocean was unintelligible. Because they were incapable of intelligibility beyond their own kind.

Maybe we’re in the same position with AI. Maybe not. I honestly don’t know. But I think about Solaris when I think about it, and that feels like the right reference.

Related thinking: