How Do AI Detectors Work? | Techniques & Accuracy
One consequence of the AI boom has been a surge in AI detectors—sometimes called AI writing detectors or AI content detectors. AI detectors work by analyzing features of a text and evaluating if they better match human samples or AI-generated samples.
These tools are becoming increasingly popular, as people want to know when AI has been used in the writing they work with.
However, just because AI detectors are popular doesn’t mean they are always reliable. Therefore, it’s important to understand how AI detectors work, so as to better understand the results they give us.
How does generative AI work?
To understand how AI detectors work, it’s important to understand how generative AI works, too. Generative AI is any AI that “generates” content, such as ChatGPT, Gemini, or Claude.
These models work with two primary principles: machine learning and natural language processing (often abbreviated to NLP).
Machine learning is a way for computers to learn and make decisions without being explicitly programmed to do so. Machine learning allows computers to recognize patterns in large sets of data, and later, use those patterns to make predictions or decisions about new data.
NLP is similar, but focuses specifically on getting computers to understand human language. Computers break down language into its smallest units (e.g., nouns, verbs, prepositions, determiners) and use algorithms to analyze the relationship between those units. With enough data, AIs can interpret and generate human-like text.
I also asked it to explain the machine learning and NLP processing behind its response, keeping both the response and explanation simple enough for a child to understand.
Its response to my query: “Cats purr because it makes them feel happy and calm! It’s like when you hum a favorite tune. They also purr to show they feel safe with you or even to help themselves feel better if they’re hurt or scared. It’s their special way of talking without words.”
How it used ML/NLP to respond: ChatGPT “read” my query and used NLP to understand that I want to learn something about cats and that I’m looking for a reason (evidenced by the interrogative “why”). It then used machine learning to find patterns in information it has learned about cats, select the best facts, and arrange them in an easy-to-understand reply.
How do AI detectors work?
Like generative AI, AI detectors work thanks to machine learning and NLP. There are four main concepts involved in understanding how AI detectors work:
Classifiers
A classifier is a machine learning model that categorizes input data into predetermined classes. For AI detectors, these classes are “AI-written” and “human-written.”
Typically, data engineers or “AI trainers” feed classifiers large datasets that have already been labeled as being written by AI or by a human.
The classifier then analyzes features like word usage frequency, sentence length, and complexity in the different classes of data. Once it identifies patterns that vary between the sets, it establishes a boundary between the classes.
When a new text is presented to the classifier, it evaluates the same features and looks for these patterns again. Depending on the patterns it finds, it decides if the text falls on the AI or human side of the boundary it previously established.
Embeddings
Embeddings translate human language into numerical representations that computers can process. Computers can’t understand the meaning of words, but they can understand numbers.
These numerical representations, called vectors, capture the meaning, structure, and context of words in a way that machines can understand. By mapping words into a multi-dimensional space, embeddings allow AI to detect relationships and patterns between words. For example:
- The vector for the “king” may be [0.9, 0.6, 0.3] and “queen” [1.0, 0.5, 0.3]. The proximity of these vectors helps AI understand that these words also share proximity in human meaning.
- The word “bear” would have different vectors when referring to an animal than when used in a phrase like “bear in mind.” In some cases, embeddings change depending on the context.
Perplexity
Perplexity is a feature analyzed by classifiers that measures how predictable a text is. Perplexity looks at how likely the text is to perplex—or confuse—the average reader, whether because it doesn’t make sense or doesn’t sound natural.
| Example sentence continuation | Perplexity level | Explanation |
|---|---|---|
| I would love a bowl of soup. | Low | This is a very likely continuation. |
| I would love a bowl of creamy and tangy tomato soup. | Medium-low | This continuation is less likely, but makes grammatical and contextual sense. |
| I would love a bowl of that tomato soup that your mother made for us last year when I slept over at your house and we had a snow day. | Medium | This makes sense, but it’s unusually structured and difficult to follow. |
| I would love a bowl of grasshoppers jumping. | High | This doesn’t make sense and is grammatically incorrect. |
Generative AI strives to produce texts with low perplexity. Its goal is to generate content that is easy to understand for most people. On the other hand, human writing usually has higher perplexity. Humans make more creative choices when writing, and human error means that typos or misspellings are more common.
Classifiers use the patterns they’ve previously established and the relationships captured in embeddings to analyze perplexity. If an AI detector detects that a text has low perplexity—that it was mostly able to predict what words came next—it may decide that the content was probably created by AI.
That said, common sentences are common because people use them often. So perplexity alone isn’t a strong enough feature to determine if AI or a human created a text.
Burstiness
Burstiness is another feature analyzed by classifiers. It refers to variation in sentence length and structure:
- A text has low burstiness when it has little variation in sentence structure, complexity, and length.
- When the text shows more variation across sentences, it has high burstiness.
| Sample paragraph | Burstiness level | Explanation |
|---|---|---|
| I woke up late. The sky outside was a mix of orange and gray, a sign of the storm approaching. Groggily, I made my way to the kitchen, grabbing a cup of coffee to shake off the sleepiness. It was going to be a long day, I could already tell. | High | The length and complexity of sentences vary. The text uses adverbs and present participles as adjectives to describe and establish flow. |
| I woke up late in the morning. The sky outside was a mix of orange and gray. I made my way to the kitchen. I grabbed a cup of coffee. I could tell it would be a long day. | Low | Sentences are consistently short or medium in length. Most sentences are simple sentences. |
AI content tends to have lower burstiness than human text. Since AI language models are working to predict what word should come next, they’re likely to choose predictable words and structures when writing sentences. So, low burstiness is one feature that leads AI detectors to predict that a text was written by another AI and not a human.
How reliable are AI detectors?
No AI detector is 100% reliable. All of them have a margin of error, and there are many accounts of people who got false positives—that they put their human-created content through an AI detector and were told that AI created it.
AI detectors can also give false negatives. Detectors can fail to identify sophisticated AI-generated text, especially if the AI was prompted to mimic human writing closely. Generative AI is also evolving so quickly that some AI detectors simply can’t keep up.
Humans have also caught on to how AI detectors work and know how to edit AI-created content to make it appear human-made.
Is the QuillBot AI detector accurate?
Even if no AI detector is always right, QuillBot’s free AI Detector is advanced. Instead of simply choosing between a human or AI writer, QuillBot’s tool classifies text the following ways:
- AI-generated
- AI-generated & AI-refined
- Human-written & AI-refined
- Human-written
This gives users a more in-depth understanding of how a text may have been constructed.
If you want to determine if a text was produced by AI, we recommend using AI detectors in tandem with other forms of analysis before reaching a decision.
What are AI detectors used for?
AI detectors are used across fields and industries by anyone who would like to check if a text may have been produced by AI.
| User | Reason |
|---|---|
| Educators | To check that student work is original and not generated with AI. |
| Publishers | To ensure they only publish human-created material. |
| Editors | To check that the content their journalists submit is human-written. |
| Recruiters | To make sure cover letters are applicants’ own writing. |
| Content writers and copywriters | To publish AI-generated content to the web without Google detecting that their articles or copywriting was created by AI. |
| Social media and forum moderators | To fight misinformation through detection of spam and fake news generated by AI. |
| Forensic analysts | To verify the origin of content in legal disputes or investigations. |
| AI trainers | To see how AI detectors work so as to better train their generative AIs. |
AI detectors vs plagiarism checkers
There are some differences between AI detectors and plagiarism checkers, although both are used to determine dishonesty in educational or work contexts.
| Question | AI detector | Plagiarism checker |
|---|---|---|
| What does it try to detect? | If a text was written by generative AI | If a text (or parts of a text) were copied from other sources |
| How does it do this? | Evaluates the features of the text and compares it to patterns they recognized in the data sets their trainers fed them | Compares text input to large database of research journals, student dissertations and research papers, and so on |
Plagiarism checkers will sometimes identify chunks of AI-created text as plagiarism. This is because generative AI often draws on sources that it doesn’t cite. So while AI usually creates original sentences, it may also occasionally include sentences directly copied (or barely paraphrased) from other texts.
Detecting AI writing manually
There are ways to detect AI writing that don’t rely on AI detectors. Manual methods, while not foolproof, can complement AI detectors and provide a clearer indication of whether a text was AI-generated.
One manual approach is reviewing a document’s version history. If large chunks of text appear suddenly, it might suggest AI use. However, keep in mind that the author may have drafted elsewhere and pasted it in.
Presentations and oral assignments can also help. Talking to someone about their writing can reveal if they truly understand it. If they’re unfamiliar with their own work, AI could be involved.
Finally, you can analyze writing similarly to how AI would—looking at perplexity and burstiness—and supplement this analysis by examining other features of writing, too.
| Criteria | How AI-generated content usually looks |
|---|---|
| Perplexity | Text is generally predictable and has few surprise word choices. |
| Burstiness | Text reads monotonously and employs little variation in sentence length or structure. |
| Repetition and redundancy | AI may repeat the same words or state the same idea in different ways. |
| Overly polite language | Generative AIs are designed to be “friendly assistants,” so their default language is often overly formal and polite. They do not sound conversational unless prompted to do so by users. |
| Unconfident language | AI typically steers away from bold, confident claims. Look for passive phrases like “It’s important to note that …”, “Some might say …”, or “X is commonly regarded as …” |
| Inconsistency in voice | If you know the voice of the person whose work you’re analyzing (e.g., a student), you might be able to tell if it feels inconsistent in a text. AI usually struggles to mimic an author’s tone of voice unless prompted extremely well. |
| Inconsistency in themes | AI often struggles with themes, particularly with incorporating them throughout a writing and including thematic closure at the end of a text. |
| Underuse of stylistic elements | AI generates text based on predictability, which means it often underuses stylistic elements like metaphors, similes, and analogies. If you see a lot of these, especially uncommon ones, it might be a sign the text was written by a human. |
| Logical or factual errors | AI can “hallucinate,” meaning it makes factual or logical errors. Look out for contradictions, implausible “facts,” or nonsensical arguments. |
| Incorrectly cited or uncited claims | With academic writing, it’s important to cite sources. AI tends to not do this, or to cite them incorrectly. Therefore, evaluating sources can help you see if a text was written by AI. For example, AI might cite a source where the cited information doesn’t even appear. |
AI image and video detectors
AIs that have typically been used for text generation, like ChatGPT and Gemini, are now capable of creating images, too. And other generators specific to image and video, like DALL-E or Synesthesia, are also becoming more popular.
As such, it’s increasingly important to be able to detect when an image or video (also known as “deepfakes”) has been created by AI. Some AI detectors for videos and images already exist, like Deepware or FakeCatcher, though it’s unclear how reliable they are.
Thankfully, due to AI’s current limitations with generating images and videos, there are some easy-to-spot features that can tip us off as to AI-created media, like:
- Anatomical errors like hands with the wrong number or proportion of fingers
- Nonsensical text or text that looks blurred or distorted in some way
- Unconvincing faces that may be overly idealized
- Overly smoothed or overly sharpened skin, hair, or fur
- Unnatural light sources, inconsistent shadows, or bizarre reflections
- Unnatural or jarring movements
- Strange or disrupted patterns or surface details
- Metadata and watermarks
Frequently asked questions about AI detectors
- What do teachers use to detect AI?
-
Teachers use a mix of strategies and tools to detect AI.
One method teachers use to detect AI is manual analysis. Teachers look at grammar, style, tone of voice, and the themes present in writing to see if it feels human.
Teachers also know how AI detectors work and how to use them to analyze writing. AI detectors can check—for example—essay writing to see if the qualities of the essay match human samples or AI samples.
QuillBot’s free AI Detector is one of the tools that can help teachers detect AI.
- How can you check if an image is AI generated?
-
There are a few ways you can check if an image is AI-generated.
First, review the image for anything that doesn’t look quite right. AI-generated images often have distorted text, patterns, or human features (especially faces and hands).
Second, check for metadata information. Some AI image generators use specific filenames or imprint a watermark on their images.
Third, understand how AI detectors work and how you can use them to analyze the probability that an image was generated by AI.
And if you need help detecting texts generated by AI, QuillBot’s free AI Detector is one option.
- Can AI write a book?
-
Yes, AI can write a book. However, a book written by AI probably wouldn’t be a very good book.
Unlike humans, AI is limited when it comes to creativity (e.g., with figurative language), themes, and tone of voice.
Despite this, AI can be helpful to human authors. For example, AI can suggest writing strategies to put writers in the right mindset.
Many publishers also use AI detectors to check that what they publish is high-quality, human-written work.
If you’d like to see how AI detectors work, you can try QuillBot’s free AI Detector.
- Can AI write code?
-
Yes, AI can write code. That said, it is still quite limited in the quantity and complexity of code it can write.
AI can be helpful for writing basic code or for helping developers check for errors. But if you want to create an entire program or complex website, AI probably isn’t the best choice.
Keep in mind how AI detectors work: by evaluating how surprising and varied writing is. Since code is a very structured language and doesn’t contain these factors, it’s harder to tell if it was written by AI.
QuillBot’s free AI Detector can help you check if text was written by AI, but may not be good at detecting if code was written by AI due to the reasons above.
- What are the rules on how to cite AI?
-
There are different rules on how to cite AI depending on your specific context.
You shouldn’t cite AI as a source of factual information. But if you’re studying AI, you may be able to cite it as a primary source. Understand how to cite sources in the style relevant to your industry or project, and then look at how the manual says to cite AI.
If you use AI in other ways (e.g., developing research questions), some institutions require you to cite it. Check your institution’s guidelines.
Educators understand how AI detectors work (like QuillBot’s free AI Detector) and can detect AI writing, so it’s important to be clear in your citations.
