John August

Words on the page

Which of these answers was written by AI?

Tools, Words on the page, Writing Process

GPTZero is a new service created by Edward Tian designed to sniff out AI-generated text.

I’m dubious, in part because I’m not sure I could reliably spot the differences between sentences assembled by humans versus machines. But maybe that’s just my human bias.

As an experiment, I decided to run a quick, not-at-all-scientific test. I asked ChatGPT to “write three paragraphs explaining what a manager does for screenwriters.” I also looked up the answer for “What does a manager do?” on screenwriting.io.

Here are the two responses, one written by a human1, the other by an AI. See if you can spot which is which.

SAMPLE A

In brief, a manager manages your career.

This includes tangible acts, like setting up meetings, giving notes and feedback, and assisting with deal negotiation, as well as the more abstract, like helping steer your career’s direction.

Managers, unlike agents, are not required to be licensed by the state. They usually have fewer clients than agents, and often take more personal and less business-focused roles in their clients’ careers. Managers, are allowed to take producer credits on their clients movies, which agents are not.

Like agents, a manager typically receives 10% of a client’s earnings. A manager should never ask for an upfront or flat fee.

SAMPLE B

A manager for a screenwriter plays a vital role in helping the writer navigate the film and television industry. They can assist with finding and developing new ideas, as well as helping the writer shape their existing ideas into marketable concepts. This often involves providing feedback and suggestions to help improve the story and make it more appealing to potential buyers.

Managers also help screenwriters with the business side of things, such as identifying potential buyers and pitching the writer’s work to them. They can also help negotiate deals and contracts, as well as manage any legal issues that may arise. In addition, they can help the writer navigate the many different players in the industry, such as agents, producers, and studio executives.

Finally, a manager can also help screenwriters with their long-term career goals by helping them identify opportunities for growth and advancement. This might include working on larger projects, transitioning to other forms of media, or developing a personal brand to increase their visibility and credibility in the industry. Overall, a manager plays a critical role in helping a screenwriter achieve success in the competitive film and television industry.

For each answer, GPTZero measured the text’s Perplexity — the randomness of the text — across several ranges.

Total Average Highest
Sample A 41 136 219
Sample B 11 36 49

That 219 is an example of “Burstiness,” which GPTZero defines as “spikes in perplexity.” For Sample A, the opening sentence has a perplexity of 219, whereas highest perplexity in Sample B is just 49.

That’s the tell, apparently.

Some human written sentences can have low perplexities, but there is bound to be spikes in perplexity as the human continues writing. Contrastingly, perplexity is uniformly distributed and constantly low for machine generated texts.

After crunching the numbers, GPTZero renders its verdict.

SAMPLE A (score 66.44): Your text is likely human generated!

SAMPLE B (score 9.93): Your text is most likely to be AI generated!

On both counts, GPTZero got it correct.

But how hard would it be to fool the system? I spent two minutes editing the AI-generated version into the following:

Sample C

Managers can play a vital role for screenwriters. Basically, they help the writer navigate the film and television industry. They can assist with finding and developing new ideas, or shaping their existing ideas. They provide feedback and suggestions.

Managers also help screenwriters with the business side of things: who the buyers are and how to pitch to them. They can help negotiate deals and contracts, as well as manage any legal issues that may arise. They help the writer navigate the many different players in the industry, including as agents, producers, and studio executives.

Good managers think about screenwriters’ long-term career goals. This might include working on bigger (or smaller!) projects, transitioning to other forms of media, or developing a personal brand. Overall, a manager can play an important role in helping a screenwriter achieve success in the competitive film and television industry.

Total Average Highest
Sample A 41 136 219
Sample B 11 36 49
Sample C 19 106 439

The verdict:

SAMPLE C (score 118.77): Your text is likely human generated!

Some takeaways

  • The ChatGPT answer was well-written and potentially helpful for someone who didn’t know anything about managers.
  • GPTZero is looking for patterns a human likely wouldn’t notice, which makes sense. But an AI model trained to provide responses with high perplexity and burstiness would likely evade detection. It’s interesting to see this arms race play out, because adversarial networks are already a fundamental tool in AI.
  • It’s currently very easy to fool GPTZero with just minimal rewriting. If this tool meant to detect AI-generated term papers, students will quickly learn how to evade it.
  1. Stuart Friedel wrote most of the early screenwriting.io answers. ↩

Clarifying scene geography

Follow Up, Formatting, Scriptcast, Words on the page

Based on an earlier Three Page Challenge entry — and with the blessing of writer Andrew McDonald — I’ve posted a new video in which I revise a scene in real time. Along the way, I discuss how small choices on the page can help (or hurt) a reader’s ability to understand where characters are and what’s happening.

As a fun bonus, Andrew has also filmed a demo version of his vision.

Two transitions in a row

Formatting, QandA, Words on the page

Don Donahue writes:

How would you format two transitions in a row?

I’m experimenting with a transition where a drone gets shot down in Alaska. Right now I have…

CUT TO STATIC:

FADE TO BLIZZARD:

Is this acceptable? Is there a better way to do this?

This looks awkward. It feels like there’s something omitted between the two transitions. After all, the point of a transition is that you’re transitioning to something. That something is missing here.

It’s important to understand that by “transition” screenwriters are really talking about two distinct things:

  1. What we’re seeing on screen as we move from one scene to the next.
  2. The handoff from one idea to the next.

The first kind of transition happens on the right-hand margin, uppercase, often with a colon at the end. It’s unique to screenplay grammar.

In your example, FADE TO BLIZZARD is an example of this kind of transition. It’s strictly cinematic. You’re telling us what we’d see on screen, and giving specific instructions to an imaginary editor.

The second kind of transition happens in all kinds of writing. You see it novels, plays, non-fiction and journalism. I’d argue that at least half of the craft of writing is transitioning smoothly to the next thing.

In your example, CUT TO STATIC is describing what we’re seeing and hearing, but it’s the kind of thing the characters in the scene are probably experiencing firsthand. So I’d leave it in scene description.

Obviously, I don’t know exactly what’s happening in your scene, but let’s imagine it something like this:

Malcolm leans close, pointing to a dark smudge on the screen.

MALCOLM

There! What do you think that is?

DAHLIA

A bunker? Something’s moving inside.

CLOSE ON SCREEN, a FLASH. Something flies straight for the lens. Bright. Hot.

MALCOLM (O.S.)

Shit! A rocket!

Impact! The screen cuts to STATIC.

CROSS-FADE TO BLIZZARD.

EXT. ARCTIC STATION – NIGHT

HOWLING WINDS. The domes of the compound are just barely visible. Yellow work lights wobble.

Fade? Cross-fade? Your call. To me, FADE generally implies that we’re going to black or white.

You’ll notice I put a period after BLIZZARD rather than a colon. Blizzard is the thing we’re cross-fading to, so it feels like a complete thought. I’d give it a period.

The colon makes sense when the object of the transition is in the next line, like this:

DA’VONNE

I know where he’d hide it.

CUT TO:

INT. TRUCK GARAGE – NIGHT

With flashlights, Da’Vonne and Sarah make their way amid the massive semis and trailers.

Here, the CUT TO: is doing the work of both a cinematic transition (a blunt cut to the next place) and a narrative transition (answering the question “Where does Da’Vonne think it’s hidden?”).

Uppercased transitions are a useful tool for screenwriters, particularly when they can do this double-duty. But you’ll read many screenplays that hardly use them at all, and that’s fine too. They’re never strictly necessary.

Dots, Dashes and Parentheticals

Formatting, Words on the page

I recorded this video to illustrate the different ways screenwriters can indicate dialogue is being interrupted. It’s a situation that happens quite often in most screenplays. The choice of how you show it can impact the read.

There aren’t hard-and-fast rules, but there are definitely conventions, and in this video I cover most of them.