AI use is seeping out of business and science writing and into the world of literature. And that, says
Malin Hay in a post on the London Review of Books blog, is a problem, because literary editors may be the worst equipped to spot text generated by it:
Experimenters in the US last year showed nine subjects a series of articles, half written by humans and half generated by ChatGPT, Claude and other large language models. Asked to guess which of the texts were human, the four subjects who rarely or never used ChatGPT in their daily lives scored "at a similar rate to random chance", while the five who used chatbots almost every day at work collectively misidentified only one in three hundred texts.
So how do you spot AI-generated fiction? Hay says familiar tells include a fondness for em dashes and for the formulation "not x, but y", which it has favoured since GPT-3. But at the start of her post Hay quotes three passages that use neither of these but still have a distinct whiff of AI.
She offers suggests some more sophisticated indicators of the use of AI fiction:
Some of the markers seem to be lexical: AIs like talking about sweetness, loudness, quiet, age and beauty. There is a lot of insisting in AI-generated texts, as well as a lot of promising, a lot of permitting and a lot of filling up.
Another sign is the overuse of tricolons ("something neat, something pleasing, a quiet violence made beautiful"). And bots often leave out definite articles from phrases where they’re not strictly necessary: "Coffee and cocoa leaned wild", "rain in teeth" or, from later in The Serpent, "Sita became obstacle by existing."
There is a flatness or evenness to AI-generated texts: Wikipedia's guide to detecting AI says that LLMs "tend to omit specific, unusual, nuanced facts" and "replace them with more generic, positive descriptions".
The strange thing about this evenness is that it isn’t usually couched in neutral language: "a flood that chokes but insists upon being swallowed" is violent, but not vivid. Even if you had your face shoved into a cake, it wouldn’t "flood" your mouth. The prosody is so smooth that you feel a lack of pressure despite the description of gross or vile acts; it rings hollow. It fills you with its nothing.
The Serpent is a short short by Jamir Nazir called The Serpent in the Grove which recently won the Commonwealth Short Story Prize, only to give rise to suspicions that AI played a part in its production. You can read it and judge for yourself.
Incidentally, I have long been fond of parenthetic dashes myself, but use en dashes for the job on this blog. That's because em dashes tend to look clumsy in online text.
This fondness dates back to the days when I wrote for David Boyle at Liberal Democrat News. I noticed that whenever I used brackets in my writing he struck them out, but he let dashes through.