Bots writing for bots: 35% of new web pages use AI

The internet is rapidly becoming a closed loop of machines talking to machines. Over one-third of web pages published since ChatGPT arrived on the scene show significant signs of AI authorship, according to TechCrunch AI. This finding comes from a new Pew Research study, painting a stark picture of how modern web content is produced. Combine this with recent Cloudflare data showing that bot traffic has officially surpassed human web traffic, and a clear reality emerges. Much of the internet now consists of automated bots reading text written by other automated bots.

To figure out exactly how much AI content is floating around, researchers didn’t just guess. Pew used the Common Crawl web archive to pull nearly half a million English-language web pages spanning the last five years, starting a couple of years before ChatGPT’s debut. They then ran this massive dataset through Open Pangram’s AI detection technology to identify pages that were likely written or heavily edited by generative models.

When looking at a recent random sample of 10,000 pages, researchers found AI authorship markers on about 10% of the URLs. However, that baseline number includes older pages published before modern AI tools even existed. Once Pew filtered the data to only include pages published after ChatGPT’s November 2022 release, the number spiked dramatically. A massive 35% of these newer web pages showed significant signs of AI editing or authorship.

Where is this synthetic content living?

  • .com domains: Commercial sites show AI authorship at roughly 10 times the rate of academic or government sites.
  • .org domains: Non-profits and organizations sit much lower, maintaining a 4.6% rate of AI authorship.
  • .edu and .gov domains: These institutional sites remain largely human-driven, hovering around just 1% AI authorship.

Beyond relying on detection software, Pew researchers noticed specific stylistic shifts across the web that heavily correlate with AI usage. If you are reading a post with a sudden spike in the use of em dashes, strict adherence to the Oxford comma, or formulaic phrasings like “it’s not X, it’s Y,” you’re highly likely looking at AI output.

The researchers acknowledge a key limitation with this methodology. AI detection tools like Pangram aren’t flawless and can occasionally misclassify human writing as AI-generated. However, when applied at this massive scale, the data provides a highly accurate directional trend of where the web is heading.

For content creators, SEO professionals, and digital marketers, this rapid saturation matters deeply. If 35% of new web content is AI-generated, simply publishing generic AI text is no longer a competitive advantage. It is merely the new baseline. Standing out in search results now requires original research, a distinct human voice, and unique data that language models can’t easily synthesize. Furthermore, for AI developers scraping the web to train new models, this study highlights the growing necessity of aggressive data filtering to avoid training AI on synthetic data.

The fabric of the web is fundamentally changing. As synthetic content floods commercial domains, finding the human signal in the AI noise will become the internet’s next great challenge. Readers can review the full study details and methodology at the original source.

Scroll to Top