Sam Altman at Davos talking about AGI and I

Sam Altman went to Davos and told a room full of billionaires that artificial general intelligence is coming. Maybe soon. And it’ll be great.

I watched the clip on Bloomberg three times.

The first time, I was impressed. He was articulate, composed, and specific about what OpenAI is building. He didn’t hedge the way most tech CEOs hedge. He said AGI. He said it could be the most important technology in human history.

The second time, I felt uneasy. The confidence was absolute. No uncertainty. No “we think” or “it’s possible.” Just “AGI is coming and it will be good.” Said to a room of people whose primary interest in AGI is how to profit from it.

The third time, I was just confused. Because I genuinely don’t know if he’s right.

The confidence problem

Visionaries are always confident. That’s what makes them visionaries. Steve Jobs was confident about the iPhone when the phone industry laughed. Elon Musk was confident about electric cars when Detroit said they were toys. Confidence in the face of skepticism is sometimes the mark of genius.

It’s also sometimes the mark of delusion.

The problem is that you can’t tell the difference in real time. Confidence looks the same whether it’s built on insight or wishful thinking. The evidence arrives later. Sometimes much later.

Altman is confident about AGI. Maybe he’s seen something in the labs that justifies the confidence. Maybe the next version of GPT does something that would change my mind. Or maybe he’s a CEO promoting his company’s product at a conference full of potential investors.

Both can be true simultaneously. That’s the thing about humans: we can be sincere and strategic at the same time.

What AGI means (and doesn’t)

Altman uses “AGI” to mean an AI system that can do any intellectual task a human can do. That’s a common definition. It’s also vague enough to be unfalsifiable. When does a system cross the line from “very capable AI” to “general intelligence”? Who decides?

GPT-4 (not yet released as I write this, but rumored) can supposedly write code, analyze images, pass exams, and reason about complex problems. If it can do all that, is it AGI? Some will say yes. Others will say it’s just a very good next-token predictor that simulates understanding without having it.

The definition fight will consume years of discourse. I’m already tired of it, and it hasn’t started yet.

The part I keep thinking about

Here’s what I keep circling back to: Altman told a room of billionaires that AGI will be great. He didn’t tell the truck drivers, the accountants, the translators, the customer service reps. The people who Davos attendees employ. The people whose jobs are most likely to change.

That’s not Altman’s fault, necessarily. You go to Davos and you talk to the people at Davos. But the optics matter. The person building the most powerful technology in history announcing it to the wealthiest people in the world, at a ski resort in Switzerland, feels like a scene from a movie where the audience is supposed to feel uneasy.

I feel uneasy.

Not because I think AGI is impossible. I don’t know if it is. Not because I think it’ll be bad. I don’t know that either. I feel uneasy because the people making the decisions about this technology are very confident and very rich and very far from the people who’ll be most affected by it.

That pattern has played out before. It usually doesn’t go well.

But then again, every visionary sounds delusional until the evidence arrives. Maybe the evidence is coming. I’ll be watching.

Related thinking: