An AI text detector doesn’t “know” who wrote a passage — it makes a statistical guess based on how predictable the word choices are, then reports that guess as a score or percentage. Independent testing has repeatedly found these scores to be unreliable enough that OpenAI shut down its own detector within four months of launching it, and researchers have documented consistent false accusations against specific groups of writers. Treat any single detector’s verdict as a hint, not a verdict.

How detectors try to spot AI writing

Most tools analyze two things: perplexity — how surprising or predictable each word is given what came before — and burstiness — how much sentence length and structure varies across a passage. Large language models tend to pick the statistically likely next word, producing text that is smoother and more uniform than typical human writing, which has a more erratic rhythm. A detector trains a classifier, often another neural network, on large sets of known human and AI text, then scores new text by how closely its patterns resemble the AI side of that training data.

A newer, more direct approach is watermarking, embedded by the AI company at the moment text is generated rather than guessed afterward. Google DeepMind’s SynthID for text works by slightly biasing the odds a model assigns to candidate next-words in a pattern only the company’s own detector can recognize; it’s now built into Gemini’s outputs. Because it’s embedded at generation rather than inferred from style, SynthID is more reliable than after-the-fact classifiers — though DeepMind itself notes its confidence score drops sharply once text is heavily rewritten, machine-translated, or very short.

Why detectors get it wrong

Detection is fundamentally probabilistic, and every method above has documented failure modes:

  • OpenAI abandoned its own classifier. In 2023, OpenAI released, then discontinued, an AI-text classifier after acknowledging a “low rate of accuracy” — it correctly flagged only about a quarter of AI-written text while mislabeling roughly 9% of human writing as AI-generated.
  • Non-native English writers are disproportionately flagged. A Stanford study ran seven popular detectors against TOEFL essays written by non-native English speakers alongside essays by native speakers. The detectors misclassified the majority of the non-native essays as AI-generated while scoring native-speaker essays almost perfectly — because the tools read simpler, more predictable vocabulary as a sign of machine authorship, and non-native writers tend to use more predictable word choices for reasons that have nothing to do with AI.
  • Light editing breaks the signal. Rewriting a few sentences, blending AI drafting with human revision, or translating text weakens both style-based detectors and watermarks. Since most real-world writing today mixes AI drafting with human editing, a clean “AI” or “human” verdict increasingly fails to capture how a text was actually produced.
  • No detector publishes an error rate low enough for high-stakes decisions. Because a false positive can mean a wrongly accused student or a rejected application, vendors including Turnitin now advise treating a detection score as one input for a human reviewer, not as automatic proof.

Can you trust a detection score?

Treat a detector’s output the way you’d treat a single witness: useful as a prompt to look closer, not sufficient as sole evidence. If you want to check a piece of text yourself, a tool like GPTZero lets you scan up to 10,000 characters free without creating an account — handy for a first read, but subject to the same caveats above (longer documents require a free account or a paid plan). For anything with real consequences — a grade, a hiring decision, a legal dispute — pair a detector’s score with other evidence, such as drafts, edit history, or a conversation with the writer, rather than relying on the number alone.

FAQ

Can AI reliably detect its own writing? Not on its own. Even watermarking systems like SynthID, embedded at the moment of generation, lose confidence once text is edited, translated, or shortened.

Do universities still use AI detectors? Many do, often through Turnitin, but a growing number of institutions have disabled the feature or instruct instructors to treat scores as advisory rather than conclusive, following documented false-positive cases.

Can you make AI text “undetectable”? Rewriting, paraphrasing, or light editing reliably lowers detection scores — which is exactly why detector vendors warn against using their tools as definitive proof.

Sources: TechCrunch on OpenAI’s detector shutdown, Stanford study on detector bias against non-native English writers, Google DeepMind on SynthID text watermarking, Wikipedia: Turnitin.