Burstiness & Perplexity | Definition & Examples

Was this written by a real person or an AI? As generative AI tools become better and better at mimicking human writing, this question is getting increasingly difficult to answer.

Measures like burstiness and perplexity, which capture patterns in a piece of writing, are one way to assess whether text is written by generative AI. AI detectors like QuillBot’s AI detector can use these metrics to quantify how human-like a piece of writing is.

What are burstiness and perplexity?

“Perplexity” and “burstiness” describe two components of a piece of writing: how predictable it is (perplexity) and how varied the overall structure is (burstiness).

Perplexity definition

Perplexity measures predictability—how easy it is to guess what comes next.

Consider the sentence “I went for a hike in…” A predictable, or low-perplexity, outcome might be something like “the woods.” “Muskoka” is also a valid ending to this sentence, but it’s harder to predict, so it would have moderate perplexity. An even more unlikely, high-perplexity answer would be “a distant galaxy inhabited solely by sea sponges.”

Errors like typos and misspellings occur more frequently in human-written text than in AI-written text, and because they are unpredictable, they also increase perplexity.

The idea of predictability is core to how AI writers work. These large language models (LLMs) are trained on massive amounts of text, and they use this training to predict which words should come next. As a result, AI-written text tends to have lower perplexity than human-written text.

Burstiness definition

While perplexity considers how predictable text is at the level of individual words, burstiness considers the bigger picture—how much variation exists between sentences and throughout the overall text.

Burstiness measures how factors like sentence length, sentence structure, and word use shift throughout a document. People tend to write and speak in long and short bursts—we alternate between short, punchy sentences and more complex ones. Generative AI, on the other hand, will often use more uniform, repetitive sentence structure. Its burstiness is therefore generally lower than that of a human.

Consider the two examples, and see if you can guess which one is higher in burstiness.

Burstiness vs perplexity

Burstiness and perplexity can both be used to assess whether a piece of text appears more AI-like or human-like, but they consider separate elements of writing. Perplexity measures predictability at the level of words, whereas burstiness measures predictability at the level of sentences and text structure.

For instance, a text with low burstiness might contain sentences that are all the same length and style, whereas a text with low perplexity would use highly predictable words. Consider the following examples of how different levels of burstiness and perplexity can change how a piece of writing sounds.

Burstiness, perplexity, and AI detection

Because burstiness and perplexity measure how human-like writing is, they’re a great way to evaluate AI text generators. Developers and researchers can use them to evaluate and enhance model outputs.

These measures can also capture certain styles of writing. For example, an academic paper—which has distinct sections, each with unique structure and terminology—has higher burstiness than a poem, which usually follows a more predictable rhythm. Recognizing these differences can help models generate more realistic text.

Burstiness and perplexity can also be used to determine whether a piece of writing is AI-generated. AI detection tools will combine these measures with other indicators to determine the likelihood that something was written by an AI. However, as AI text generators become increasingly sophisticated, distinguishing human from AI will become a much harder task.

Frequently asked questions about burstiness and perplexity

What is an example of burstiness?

Burstiness is a metric that captures how varied the structure of a piece of writing is. Elements like sentence length, sentence structure, and changes in word use impact burstiness. 

An example of writing with low burstiness is as follows: “I have a dog. My dog’s name is Tim. Tim looks a bit scruffy. Tim’s ears stick out. Tim likes to go for walks.”

Compare this with an example of writing with high burstiness: “I have a dog. His name is Tim, and he’s a scruffy little character with ears that stick out sideways like broken antennae. Tim lives for walks—whether it’s a quick trot around the block or a long ramble through the park.”

AI-generated text often has lower burstiness than human writing. Burstiness is therefore used by tools like QuillBot’s AI detector to guess whether something was written by a person or an AI.

What is an example of perplexity?

Perplexity measures how predictable the words in a sentence are. 

Consider the phrase: “I sat at the bar and ordered a glass of red…” A low-perplexity (i.e., highly predictable) ending would be “wine.” Conversely, a high-perplexity (less predictable) ending might be “jelly beans.”

AI-generated text often has lower perplexity than human writing. Perplexity is therefore used by tools like QuillBot’s AI detector to guess whether something was written by a person or an AI.

What are burstiness and perplexity used for?

Burstiness and perplexity are two metrics that capture patterns in a piece of writing. Perplexity measures how easy it is to predict which words come next, and burstiness measures how easy it is to predict sentence structure and length. 

These measures are used by tools like QuillBot’s AI detector to guess whether something was written by a person or an AI. They can also be used to evaluate the quality of an AI text generator.

How do I check if a document is AI generated?

The simplest way to check if a document is AI generated is to upload it to an AI detector. These tools quickly analyze the text and determine whether it is more likely to have been generated by a human or AI.

You can also examine the document yourself for typical signs of AI writing, such as predictable sentences and word choice or a generic sound.

When you’re trying to detect AI writing, remember that no method has perfect accuracy. QuillBot’s free AI Content Detector offers a detailed report that differentiates between AI-generated and AI-refined text, which can help you understand more clearly how a document was created.

Are higher perplexity and higher burstiness always better?

Perplexity measures how predictable the words in a piece of writing are, while burstiness reflects how predictable its structure is.

Because human writing often scores higher for these measures than AI-generated text, it may seem like maximizing them is always better. However, in many cases, lower perplexity and burstiness can make a text easier to read and understand.

If word choice is predictable (perplexity), text becomes easier to read. Likewise, if the structure is predictable, it may be easier to skim. 

In fact, even poor writing (like spelling errors or run-on sentences) can inflate burstiness and perplexity.

For these reasons, the ideal balance depends on the context and purpose of writing. Still, these measures help tools like QuillBot’s AI detector determine whether something was written by a human or by AI.