How AI Detectors Actually Work (and Why They Get It Wrong)

An AI detector is not reading your text for meaning. It is doing statistics: measuring how predictable your writing is (perplexity) and how much it varies (burstiness), then comparing that against what it expects a language model to produce. The output “87% AI” is a correlation score, not a confession. The detector never knows whether a model was involved. It knows whether your text statistically resembles the text models produce. That is a much weaker claim than any confident-looking percentage suggests, and it is the reason honest human work still trips the wire while a lightly edited machine draft slides through.

Start with how the model writes

A large language model does not plan a sentence and then write it. It predicts the next token based on everything that came before. Same statistical prediction behind how AI creates images and videos. Each token is chosen because it scored as the most probable continuation given the preceding text.

That process leaves a fingerprint. Because the model keeps reaching for the highest-probability next word, its output has a certain smoothness. Sentences settle into similar lengths. Word choices stay inside a safe common range. Rhythm holds steady from paragraph to paragraph. This is exactly the pattern detectors are built to spot.

The two measurements that do most of the work

Nearly every detector, regardless of the marketing around it, leans on some version of perplexity and burstiness.

Perplexity measures how surprising your word choices are to a language model. Feed a sentence in and, if the model thinks “yes, that is exactly what I would have written,” perplexity is low. If it thinks “I would not have predicted that word there,” perplexity is high. AI-generated text sits at low perplexity because it was literally produced by chasing the most probable next word. Human writing usually scores higher, because people make choices that are contextually fine but statistically odd: a bit of slang, an unusual turn of phrase, jargon, a word picked on instinct rather than probability.

Burstiness measures variation across the document, mostly in sentence length and complexity. Humans are inconsistent. We stack clauses until a sentence nearly buckles, then answer it with three words. Graph the sentence lengths and you get a spiky, uneven line. Model output tends to be flatter and more regular, with sentences clustering around a similar size and complexity holding steady start to finish. Detectors read that flatness as a signal.

Those two numbers rarely act alone. They feed into a trained classifier, usually a neural network fitted on large collections of human and machine text, alongside features like vocabulary diversity, transition patterns, and paragraph structure. The classifier weighs all of it and returns a single probability. That probability is the “87%” you see. Correlation score, not confession.

This is the practical hinge for anyone shipping AI-assisted work. Since the signal is statistical rather than semantic, it responds to structure. That is why guidance on how to remove AI detection from your writing only holds up when it works at the level of sentence rhythm and word-choice distribution rather than swapping a few synonyms and hoping. Surface edits leave the underlying statistics roughly where they were. Structural changes shift the numbers the classifier is reading.

Thresholds: where a probability becomes a verdict

A detector outputs a number between 0 and 1. Then someone (or a default setting) draws a line. Above the line, flagged. Below, clear.

That line is a policy choice, not a fact of nature, and it quietly controls everything. Set the threshold low and you catch more machine text but sweep in more genuine human writing as false alarms. Set it high and you protect the humans but let more AI-assisted text through. There is no setting that gives you both, because human and model text overlap in the statistical space the detector measures. You are always trading one kind of error for the other.

A vendor can advertise “99% accuracy” and still, at the threshold an institution actually runs, flag a meaningful share of clean human writing. The accuracy figure and the real-world error rate are answering different questions. Accuracy figures are usually earned on a curated test set, pitting text straight out of a model against carefully written human prose with a wide gap between the two piles. Real submissions do not look like that. They are edited, revised, blended, translated, written by non-native speakers, and sitting right in the gray zone where the two distributions blur.

The Australian Catholic University case

The gap between benchmark accuracy and field behaviour stopped being abstract when a large organisation ran detection at scale.

Australian Catholic University became a reference case after internal figures surfaced showing the machinery under load. Across 2024, the university logged on the order of 6,000 academic-misconduct referrals. Reporting on the internal data indicated roughly 90% of them related to suspected AI use. AI, in other words, had become the overwhelming driver of its integrity caseload almost overnight.

The outcomes are the revealing part. Roughly a quarter of the referrals were dismissed on review. Cases that rested solely on the detector’s report did not hold up. The university later stepped back from that AI-detection tool. Read that sequence carefully: an automated signal generated an enormous volume of accusations, one referral in four did not survive human review, and the institution eventually concluded the detector could not carry the weight being placed on it.

None of that means detection is worthless. It means the score is an input, not a verdict. The failure at ACU was not that a classifier produced probabilities. It was that a probability got treated, at least at intake, as if it were proof.

Why honest work still trips the wire

There is a comforting story people tell themselves: if you did the work, you have nothing to fear. The research does not support it.

Kofinas and colleagues, writing in the British Journal of Educational Technology in 2025, examined whether “authentic” assessment (real-world applied tasks meant to be hard to fake) could protect academic integrity in the age of generative AI. Their answer was blunt. Authentic assessment alone does not safeguard integrity, and institutions cannot lean on it as a defence against AI misuse. The implication runs deeper than any single detector. If even carefully designed applied tasks cannot cleanly separate human from AI-assisted work, a statistical classifier squinting at perplexity certainly cannot. The authors argue the durable answer is a shift toward process, live and interpersonal assessment like oral exams and reflective discussion, rather than trying to catch AI after the fact from the text alone.

That maps directly onto why false positives happen. Certain human writing is naturally low-perplexity and low-burstiness. Non-native English speakers often write with simpler vocabulary and steadier sentence structure, which is exactly the fingerprint detectors associate with machines. Formal, disciplined academic prose, the kind writers are explicitly trained to produce, reads as uniform. Anything on a heavily documented topic pulls toward the common phrasings that dominate training data. In every one of those cases, a real person writing honestly produces the statistical profile the detector was built to flag. The tool is not malfunctioning. It is measuring exactly what it claims to measure. The measurement just does not mean what people want it to mean.

The trend runs against the detector

Every new generation of language model writes with more variation, better vocabulary distribution, and more natural paragraph structure than the last. In detector terms, newer models produce higher perplexity and more burstiness, not because anyone is trying to dodge detection but because the models are simply getting better at writing. Higher perplexity plus more burstiness is, by definition, harder to distinguish from human text. The upgrades that make a model more useful to you also make its output quieter to a detector.

Detector vendors respond by retraining classifiers on fresh model output. They are structurally behind. Each model release opens a window where detection rates drop before the classifiers catch up. Under the hood, the two distributions the detector is trying to separate (human writing and machine writing) keep drifting toward each other. As that overlap grows, even a theoretically ideal detector gets closer to a coin flip in the blurry middle. This is not a bug a smarter classifier fixes. It is the shape of the problem.

What to actually do with this

Treat any single detector score as one weak signal, never as a decision. It is a probability derived from statistics, tuned by a threshold someone else picked, and it cannot tell assistance from authorship. If your work will pass through a detector you do not control, run your own pass first so nothing surprises you. If what you are shipping is coursework, it makes sense to check your essay for AI before submitting rather than finding out after the fact. Knowing the number in advance turns a gate you are guessing at into one you can plan around.

Write, or rewrite, for genuine variation. The qualities detectors reward are the same ones that make writing good: varied sentence length, specific and occasionally unexpected word choice, a real voice instead of the safe smooth default. If you lean on a model for drafting, the editing pass that lifts quality also happens to be the pass that shifts perplexity and burstiness. Surface swaps do not move the score. Structural rewrites do.

Keep perspective on what the score is. A detector reads how predictable your writing is and compares it to a model’s habits. That is a real and sometimes useful signal. It is not a verdict on whether you did the work, whether the writing is good, or whether it is even AI-assisted at all. The people who handle this well are not the ones chasing a magic phrasing trick. They are the ones who understand the machinery, treat the score as a gate to clear rather than a truth to fear, and keep their attention on the writing itself.

Detection will be part of how AI-assisted work moves through the world for a while yet. Imperfect, occasionally unfair, and steadily losing ground to the models it is trying to catch. But it is also a gate that plenty of writing has to pass through, and understanding how the gate reads your text is the difference between hoping you clear it and knowing you will.