AI Detection in 2026: What Actually Works, What's Theater

AI detection is the most over-claimed corner of the AI writing space. Vendors advertise 99% accuracy. Universities treat detector scores as evidence. Students get expelled. Freelancers get their work rejected.

And most of that confidence is theater.

This site sells an AI detector, so read the next paragraph as a disclosure rather than a boast.

We publish our detector's measured error rates, and they are bad: a 96% false negative rate, a false positive rate of 4.2% with a 95% confidence interval running from 0.7% to 20.2%, and an ROC-AUC of 0.407 where 0.5 is a coin flip. The corpus, the method and the code that produced those figures are at /ai-detector/accuracy. Several competitors publish figures orders of magnitude better than ours, on corpora orders of magnitude larger, and at least one publishes confidence intervals too. What this post is about is not that we are more transparent than they are — we are not. It is that a number without its uncertainty, its corpus and its failure mode is not usable by the person it is shown to, and most of the market still ships exactly that.

An earlier version of this page did the thing it criticises. It presented per-vendor accuracy and false-positive rates for GPTZero, Originality.ai, Turnitin, Copyleaks and this product, plus three case studies with detector scores, as though someone had run them. Nobody ran them. Those sections are gone and what replaced them is marked as either measured, published by someone else, or unknown.

AI detection is useful. It is not authoritative. The difference matters.

What AI Detection Actually Is (vs What Marketing Claims)

The Marketing Claim

"Our AI detector identifies ChatGPT, Claude, Gemini, and other AI-generated content with 99% accuracy. Protect your institution from academic dishonesty."

You've seen this pitch. Every detector runs it.

The Reality

AI detectors are statistical classifiers that predict, with some probability, whether a given text was generated by a language model. They do this by measuring features of the text (perplexity, burstiness, token distributions, syntactic patterns) and comparing them against training data of known human writing and known AI writing.

Key words: predict, probability, statistical.

Detectors do not inspect a text's history, check it against any database of AI outputs, or read the author's mind. They look at the text alone and produce a guess, with confidence. That guess is sometimes right and sometimes wrong, and the error rate depends on the text type, the AI model used to generate it, the writing style of humans in the training data, and half a dozen other factors.

When a vendor says "99% accuracy," they usually mean: "In our internal test set, using the benchmark we chose, with the AI models we decided to include, we got 99% accuracy." Real-world accuracy on content the detector hasn't seen is almost always lower.

The Fundamental Uncertainty: Why Perfect Detection Is Impossible

Here is the mathematical reality most articles avoid.

Language models are trained to produce output that is statistically indistinguishable from human writing. That is the training objective. Any detector works by finding statistical differences between human and AI text — the features the model hasn't yet managed to match.

As models get better, those differences shrink. GPT-3 output was easy to detect because it had obvious repetition and formulaic hedging. GPT-4 output is harder. GPT-5 output is harder still. Claude 4 output is extremely difficult to detect when prompted carefully.

Meanwhile, humans are not a single homogeneous group. Human writing includes:

• A first-year student writing a careful, over-structured essay (low burstiness — looks AI-ish)
• A professor dashing off a conference email (highly bursty, very human)
• A non-native English speaker writing formally (uniform structure — looks AI-ish)
• A novelist deliberately varying sentence length (high burstiness, very human)

The overlap between "formal, careful human writing" and "current AI writing" is large and growing. The fundamental problem: any feature that flags one will also flag some of the other. Reducing false positives raises false negatives, and vice versa.

The 2023 Stanford study by Liang et al. ("GPT detectors are biased against non-native English writers") is the best-known result: Liang et al. (2023) found that GPTZero and four other detectors misclassified 61% of human-written TOEFL essays as AI-generated. Even for native-English TOEFL essays, the false positive rate exceeded 5%.

And that was with GPT-3.5 era detectors and tests. Current detectors are better tuned, but the underlying problem hasn't gone away. It can't.

Statistical Methods vs ML Classifiers

Broadly, two families of detection approaches exist.

Statistical Methods

These compute interpretable features of the text and apply thresholds. Classic examples:

• Perplexity scoring — run the text through a reference language model (e.g., GPT-2), compute the average log-likelihood of each token. AI text tends to have lower perplexity.
• Burstiness scoring — compute the standard deviation of sentence lengths and syntactic complexity. AI text tends to have low variation.
• Token distribution analysis — look for overuse of specific tokens (em-dashes, 'moreover', 'in conclusion', 'various'). AI models have biases toward certain vocabulary.

GPTZero uses a combination of perplexity and burstiness. Our own Coda One AI Detector does this too, and we show you the raw perplexity and burstiness scores alongside the AI probability — because the numbers are interpretable and you can reason about them.

Strengths: Transparent. You can see why text was flagged. Generalizes reasonably well to new AI models because perplexity is model-agnostic (it's a property of the text relative to a reference model, not a specific signature of a specific generator).

Weaknesses: Easier to game. A humanizer that increases perplexity and burstiness defeats this class of detector consistently.

ML Classifier Methods

These train a machine learning model (usually a transformer classifier) on labeled examples: "this is human writing," "this is AI writing from GPT-3.5," etc. The classifier learns complex patterns that distinguish the two.

Originality.ai is primarily an ML classifier approach. So is the internal Turnitin system. So are most of the more expensive commercial detectors.

Strengths: Can catch subtleties that statistical methods miss. Often scores higher on internal benchmarks.

Weaknesses: Opaque — you cannot see why a text was flagged. Training-set bias is a huge problem (see the TOEFL-essay issue above). Struggles with AI models the classifier wasn't trained on. A new frontier model drops and the classifier's accuracy can collapse until retraining.

The best detectors today use hybrid approaches: statistical features as interpretable baseline, plus a classifier for subtle patterns. Coda One's detector is hybrid. Originality.ai is primarily classifier-based. GPTZero leans statistical.

Accuracy Numbers That Exist

There are three kinds of number in this area, and only two of them are worth reading.

Measured, by us, on us

48 labelled samples, 24 human and 24 AI. Full corpus and method at /ai-detector/accuracy.

• False negative rate 96%. Handed AI-written text, our detector usually does not flag it.
• False positive rate 4.2% — one human sample in 24 — 95% CI 0.7% to 20.2%. On 24 samples that interval is the honest answer; a single figure would be a guess with a decimal point.
• ROC-AUC 0.407, below chance, and below two of the detector's own components measured separately.
• The single human sample we wrongly flagged is an academic abstract. That is the pattern, not the exception: see the registers our detector scores highest, where careful formal human prose outranks AI that was asked once to vary its rhythm.

The corpus is small and skewed — 21 of its 24 AI samples come from one model — and those limits are published alongside the numbers rather than under them.

Published, by researchers, on detectors generally

Liang et al. (2023, Patterns) ran seven detectors over human-written TOEFL essays and found more than half misclassified as AI-generated, while essays by native-English writers were classified correctly far more often. This is the most-cited result in the field, it is independent of any vendor, and it is the reason the bias question below has a real answer.

Claimed, by vendors, about themselves

Every commercial detector publishes an accuracy figure for itself. Some publish more: GPTZero publishes a false positive rate, a recall figure, its corpus size and the models it tested against; Pangram publishes a model card and a technical report. Both disclose more than we do and both report better numbers.

We publish a 95% interval, which at 48 samples is less a virtue than an admission. So does Pangram, on its Pangram 4 model card: 95% bounds on four separate error rates, Wilson-named on two, over evaluation sets of 190,000 to 1,000,000 examples. An earlier version of this paragraph said no competitor published an interval; it was checked against Pangram's blog posts, which state point estimates, and not against its model card, which does not. GPTZero's published figures are point estimates.

An earlier version of this section said no vendor's figure had ever been independently reproduced, and that reproducing one was impossible because the vendors' terms forbid it. That was wrong, and the way it went wrong is worth naming. We cannot run such a test — those terms do forbid the automated submission it needs, and Turnitin has no public API at all. But "we cannot" is a fact about our position, and "nobody has" is a claim about the world. Nobody checked the second one.

Independent evaluations exist. Jabarian and Imas (2025, University of Chicago Booth / Becker Friedman Institute, working paper 2025-116) evaluated Pangram, Originality.ai, GPTZero and an open-source RoBERTa baseline over 1,992 passages across six genres, matched human text against four frontier models, and tested the humanizer StealthGPT alongside. The authors declare no conflict of interest and the paper is free to read. Its finding: Pangram achieves "essentially zero FPRs and FNRs" on medium-to-long passages within that sample, with Originality.ai and GPTZero a secondary tier.

What we do not know

Where our detector would place. We have never measured it against another and cannot, for the reason above — and it is not in the independent study either, which evaluates detectors an institution might actually deploy. What that study does show is the shape of the field, and our published 96% false negative rate on 48 samples sits nowhere near it.

How to Use Detectors Responsibly: As Signals, Not Verdicts

If you're a student, teacher, editor, client, or QA reviewer, here is the honest framework.

Detector Scores Are Probability Signals, Not Proof

A 95% AI score does not mean 95% probability that the text is AI. It means the detector's internal model assigns this text a 95% probability of belonging to its "AI" class, given its training distribution. The model could be wrong about the distribution. The text could be out-of-distribution. The model could be biased against the author's writing style.

High Scores Justify Follow-Up Questions, Not Accusations

If a teacher sees a 90% AI score, the appropriate next step is a conversation: ask the student to discuss the essay's argument, explain specific choices, expand on a point. Not an accusation of cheating. A student who wrote their own essay can engage substantively. A student who submitted unedited AI output usually cannot.

Use Multiple Detectors

No single detector is reliable enough on its own. Running text through three and requiring agreement across all three cuts false positive rates meaningfully. Our rule of thumb: 80%+ agreement across three detectors is suspicious; a single detector reading 95% is inconclusive.

Record Draft History

If you're a writer subject to detection, keep evidence of your process. Google Docs version history, draft files with timestamps, notes you took while drafting. If you get falsely flagged, this evidence is more persuasive than any counter-detection test.

Institutions Should Publish Policy Around Uncertainty

Schools using AI detection should have written policy acknowledging that detectors are fallible, defining what a positive detection means procedurally, and specifying that detection alone does not constitute evidence of academic misconduct. Most schools don't do this yet. The ones that do have fewer wrongful-discipline cases.

Comparison: Major Detectors

Prices are absent on purpose. They move, and a price nobody re-checks is a wrong price with a delay — the one competitor price on this site that was re-checked turned out to be roughly double the real one. Check the vendor's page.

Approach descriptions come from each vendor's own documentation. Nobody outside these companies can confirm what runs inside them, this product included, which is why the third column is the only one that matters.

What we do differently

Two things, and neither is accuracy.

We show which sentences drove the score. A single percentage is unfalsifiable — you cannot argue with it or learn from it. Our detector marks the specific phrases and structures it reacted to, so you can check its reasoning and disagree with it. The tells it looks for are documented at /ai-detector/tells.

We publish the error rates, including the embarrassing one. The 96% false negative rate means the tool misses most AI text. We publish it because a detector whose failure rate is unknown cannot be used responsibly by anyone, and because the alternative — a 99% figure with no interval and no corpus — is the thing this post is about.

What we do not do, contrary to what an earlier version of this page said: compute perplexity or burstiness. The scoring is explicit statistical heuristics and phrase patterns. Those two words appear on a lot of detector marketing, ours included until recently, and in our case they were never accurate.

The Honest Positioning

AI detection is useful for:

• Screening large content pipelines for obvious AI dumping
• Providing a signal in combination with other evidence
• Helping writers identify regions of their own text that might be flagged elsewhere
• Research into language model outputs

AI detection is NOT sufficient for:

• Making disciplinary decisions against individual writers
• Proving AI use in legal proceedings
• Rejecting work without conversation or review
• Replacing human judgment about authorship

Any vendor telling you otherwise is selling you an easy answer to a hard problem. The hard problem — how do we handle AI in writing — doesn't have an easy answer. Detection is one imperfect tool among several. Use it accordingly.