GPT-3 can write code. I have feelings about this.

I need to talk about GPT-3.

OpenAI just published the paper. 175 billion parameters. Trained on a significant fraction of the internet’s text. It can write essays, answer questions, translate languages, and do basic math. It’s a language model, which means it predicts the next word in a sequence, except the sequences it produces are coherent enough to pass for human-written text in short bursts.

But that’s not what I want to talk about.

I want to talk about the code.

The thing that got me

Someone gave GPT-3 a plain English description of a program. Something like “create a simple webpage with a button that changes color when you click it.” And GPT-3 wrote the HTML, CSS, and JavaScript. Working code. Not perfect code, not production code, but code that you could paste into a browser and it would do the thing.

I sat with that for a while.

Then I tried more examples from the paper. Describe a function in English, get Python back. Describe a layout in words, get HTML. It doesn’t always work. Sometimes it generates nonsense, or code that looks right but has subtle bugs, or code that works for the simple case but falls apart at the edges.

But it works often enough that I’m having an identity crisis. And I don’t think I’m being dramatic about this.

What programming was

I learned to program the way most people did. You learn syntax. You learn data structures. You learn algorithms. You memorize the weird parts of whatever language you’re using. You develop an intuition for how to decompose a problem into pieces small enough that a computer can execute them.

Programming was translation. You had an idea in your head, and you translated it into a language the machine understood. The skill was in the translation. Knowing the destination language. Knowing its quirks, its idioms, its failure modes.

GPT-3 is, potentially, the end of that translation step. Not today. Not reliably. Not for anything complicated. But the direction is clear. If you can describe what you want in English and a model can produce working code, then the skill shifts from “knowing the language” to “knowing what to ask for.”

Which is a very different skill.

The gap between demo and reality

I should be honest about the gap. It’s large.

GPT-3 can write a simple webpage. It can write a basic Python function. It can generate SQL queries from natural language descriptions. These are real capabilities that work right now.

It cannot build a distributed system. It cannot debug a race condition. It cannot architect a codebase that will be maintained by a team for five years. It has no understanding of what it’s writing. It’s producing sequences of tokens that statistically follow from the prompt. When those sequences happen to be valid code, it looks like magic. When they’re not, it looks like a confident person who has no idea what they’re talking about.

I’ve seen both in the same session.

Where this goes

Here’s what I keep thinking about at 2am.

Today, GPT-3 can write a button that changes color. It’s a parlor trick. Amusing, impressive as a demo, not useful for real work.

But GPT-2 couldn’t do this. GPT-1 definitely couldn’t. The improvement from each generation to the next has been dramatic. Not linear. Each step makes the previous version look primitive.

So what does GPT-4 do? GPT-5? At what point does “write a function that validates email addresses” turn into “build me an app that manages inventory for a small business”?

I don’t know the answer. Nobody does. OpenAI’s API is in private beta and the early experiments are already generating things that would have been science fiction five years ago.

What I’m actually feeling

I’ll be direct.

Part of me is excited. The idea that natural language becomes a programming interface is genuinely thrilling. It makes creation accessible. Someone with a great idea but no coding background could describe what they want and get a working prototype. That’s powerful.

Part of me is scared. Not for my job specifically, but for the whole concept of expertise. If a model can approximate what a junior developer does, and each year it approximates a slightly more senior developer, then what does a career in software look like in fifteen years?

And part of me, the part that sits on the roof at night looking at stars, finds the whole thing awe-inspiring in a way that overrides the worry. We built a mathematical model that turns English sentences into computer programs. The model doesn’t understand what it’s doing. It has no concept of “code” or “function” or “button.” It just knows what words tend to follow other words. And from that, it produces something that works.

That’s strange and beautiful and a little frightening all at once.

I’m going to keep watching this. I think we’re at the beginning of something, and I think most people don’t realize it yet.

The paper is worth reading, if you can get through the technical parts. The demos are worth watching. And the question is worth asking: what happens when anyone can program, and programming is just talking?

I don’t have an answer.

I just have feelings about it.

Related thinking: