How ChatGPT resets the levels

On a recent Scriptnotes, I showed off ChatGPT to Craig and Scriptnotes guest Rian Johnson. We asked it to write up a plot summary for a future Knives Out movie set in space.

We got an answer similar to this query I just ran:¹

In the far-flung future, wealthy space entrepreneur Harrison Thorne is found dead in his luxurious space station, the victim of an apparent suicide. However, Detective Benoit Blanc, a renowned investigator known for solving even the most complex cases, has a hunch that foul play is at work. As he begins to investigate, he finds that every member of Thorne’s eccentric and contentious family has a motive for murder.

As the detective delves deeper into the mystery, he must navigate the tangled web of lies and deceit spun by the suspects, all while trying to stay one step ahead of the cunning killer. With the help of Thorne’s loyal assistant Marta, Blanc must unravel the truth before the killer strikes again.

As the body count rises and the tension mounts, it becomes clear that no one is above suspicion in this high-stakes game of space-age crime and deception. Will Detective Blanc be able to solve the case before it’s too late, or will the killer get away with murder in the cold vacuum of space?

None of us were blown away by the plotting. It felt like it was copy-pasting ideas from the first movie, with a lot of generic pitch filler. Still, we all agreed that the sentence-by-sentence writing was surprisingly good. It didn’t feel like it was written by a bot.

When we asked it to write a scene, it was functional but weightless. Here’s one based on the summary above:²

To be generous, it’s not nonsense. Unlike earlier attempts at algorithmically-generated screenplays, you can follow what happens, and characters do feel like they’re talking to each other. But there’s no nuance or character motivation despite the endless parentheticals.

It’s not a scene, but a simulacrum of one.

Listener JR, who makes his living using tools like these, wrote in with an interesting perspective:

Language models are built on “training data,” which is the text you feed into a learning process to produce the output. For very sophisticated models, the training data is vast: for something like ChatGPT, it includes something like all the text you can scrape off of the last twenty years of the Internet, or so.

But this means ChatGPT is about as smart as the average writer on the Internet has been over the past twenty years — and indeed, the models that comprise GPT drag the results toward the average, not the extraordinary, because the average has much nicer statistical properties than the extraordinary for companies that seek to produce a marketable, scalable product from their models, which requires the ability to tweak, diagnose, and defend what you’re selling.

ChatGPT isn’t optimized for breathtaking originality because that’s not its goal. It’s meant to be conversational and approachable, not wild and daring. It’s giving you hamburgers rather than Japanese wagyu sando with white truffle shavings because 90% of the time, that’s what you want.

JR continues:

Ultimately what these models mean is that with the click of a button you can now be just as good as the average writer who posts content to the Internet, and so the old “average” is now the new “zero.” If you wrote at the average level of the Internet in 2022 you now write at the zero level.

That’s a big idea, so let’s turn it over in our hands a bit.

Let’s imagine we can pick something that represents the average quality of English writing on the internet. It might be an article about holiday lobster prices or a piece of Peloton marketing copy.

Could any English speaker write at that level? No.

You probably don’t have to think that hard to come up with several people in your life who can’t write up to the standards of “Yahoo News article” or “Peloton blog post.”

Many folks don’t write, won’t write, can’t write.

In fact, if we want to consider it “median” writing quality rather than “average,” then approximately half the English speakers on the internet couldn’t hit this level of sophistication.

Only now, with tools like ChatGPT, they can. It’s very good at taking awkward, ungrammatical writing and making something decent out of it.

I fed ChatGPT this text —

if youre stove won’t light here’s what you do 1) check that that the gas is on turn the knob do you hear it 2) lisen if it’s clicking that’s the igniter. if it’s not clickign you could light it with a match 3) lift off the burner cover cuz sometimes that the issue 4) try another burner

— with the instruction to “Rewrite this as a good blog post.” Here’s the result:

If Your Stove Won’t Light: A Step-by-Step Guide

If you’re trying to cook a meal and your stove won’t light, it can be frustrating and time-consuming. Here are some steps you can try to troubleshoot the issue:

1. Check that the gas is on. Turn the knob and listen for the sound of gas flowing. If you don’t hear it, the gas may be turned off or there may be a problem with the gas line.

2. Listen for the igniter. If you hear a clicking sound, that means the igniter is trying to light the gas. If you don’t hear a clicking sound, you can try lighting the burner with a match.

3. Lift off the burner cover. Sometimes, food or debris can accumulate underneath the burner cover and prevent the gas from lighting. Removing the cover may resolve the issue.

4. Try another burner. If all else fails, it’s possible that there is a problem with a specific burner on your stove. Try using a different burner to see if it lights properly.

By following these steps, you should be able to get your stove working again in no time. If the problem persists, it may be a good idea to call a professional for assistance.

Imagine you’re a technician asked to document how a system works. You could describe the steps and let ChatGPT do the writing. Would you still need to proofread it, to make sure it’s not hallucinating something wrong? Absolutely. But most reluctant writers are competent readers. They should be able to spot when it’s gone off track.

Where I think these tools have even more benefit is for people who don’t feel confident in English. Type in your native language and let the AI do its work, translating not just the words but the intent.

Which brings us back to JR’s thesis, that a tool like ChatGPT resets the levels. In a world in which everyone can write at the old “average” level, what happens?

I’m not sure. There’s not a great comparison in recent human history. Calculators enabled ordinary people to do a lot more number-crunching, but were they transformative? (For the human calculators of Hidden Figures, perhaps so.)

A world filled with AI-polished text might push us in unexpected directions. To err is human, and we might find language incorporating mistakes to porve a hooman wrote it. Or we might use these tools like Instagram filters to make us look like someone we aspire to be.

Personally, I don’t think screenwriters are particularly endangered, at least not in the short term. Future tools may be trained and tuned for scriptwriting, but they’ll be working off a corpus of all the screenplays that have come before. And that’s how it’s always been.

JR notes:

All writers are competing against the cumulative past of our species and its semi-automatic weapons of creative destruction, and are competing to stand out from an average defined over the past couple of decades or so.[…] Writers should only worry about automated writers as much as they are already worried about the agglomerated capacity and output of our species up until the present moment, which, eh, we can do better.

1. The prompt is “Give me a plot summary for a Knives Out set in space.” ↩
2. Prompt: “Write me a scene from this movie where Blanc first meets Marta, in screenplay format.” ↩