New number worth sitting with: on the technical writing metrics alone, Fable ranks #2 out of 18 models for sounding the least like AI. Add in actual human preference, and it drops to dead last. That gap is the whole story here.
A founder building an email tool (slashy.com) got tired of guessing which model writes the least like a robot clearing its throat before every sentence, so he built an open source benchmark for it: theslopindex.com.
How the benchmark works
He didn’t just eyeball outputs and vibe check them. He built a real baseline: a corpus of actual human writing across email, Slack, social, and essays. Then he wrote 112 identical scenarios (cold emails, schedule changes, launch tweets, argumentative essays) and ran all 18 models through the same prompts, several samples each.
Scoring runs on five axes: conciseness, templating (reusing the same phrasing across unrelated tasks), rhythm (sentence and paragraph variance), tells (the “delve,” “it’s not just X, it’s Y” stuff), and human preference. No LLM judges involved. He kept AI out of the grading on purpose, on the reasoning that having models score models is asking the fox to grade the henhouse.
Three ways to actually use this
- Model selection for client-facing writing: if a model scores well on tells and templating but tanks on human preference, that’s your sign it optimizes for benchmarks, not readers
- Prompt and harness debugging: a model with strong mechanical scores but weak human preference is a candidate for better system prompts, not a model swap
- Building your own slop detector: the method (baseline corpus, fixed scenarios, no LLM judging) is copy pasteable for grading your own agent’s output against a house style
Tips and pitfalls
- Don’t trust “least slop” rankings that skip the human preference axis. Mechanical scores can be gamed by the same benchmark chasing that makes writing worse in the first place
- 🎯 Test with your own scenarios, not just the generic ones. A model that’s clean on cold emails can still be a mess on technical docs
- 🎯 Recheck rankings after a model update. This is a snapshot, not a permanent scorecard
- 🎯 If a model’s rhythm score is flat, that’s the fastest fix: vary sentence length in your prompt before you blame the model itself
If you’ve been picking models by vibes, this gives you an actual framework instead. Go run your own writing through it and see where your favorite model lands!
Frequently Asked Questions
Q: What specific signs show a piece has “AI slop”?
Watch for five telltale patterns: verbose writing that takes paragraphs to say what should take sentences, templated responses where the same structures repeat across different scenarios, monotonous rhythm with predictable sentence lengths, over-used phrases like “delve” or “it’s not just X, it’s Y,” and the overall sense that it’s obviously AI-generated. Readers often notice the verbosity first, that’s the biggest giveaway.
Q: Why does Fable bomb on human preference after scoring second-best mechanically?
Simple: models that over-optimize for benchmark metrics end up sounding more AI-generated, not less. Fable nails the mechanical stuff but creates such rigid, predictable patterns that humans instantly clock it as AI. One commenter nailed it: “a model so optimized for benchmarks it circles back to being useless in practice.”
Q: How do I use this to pick a better AI writing tool?
Skip the mechanical rankings and focus on human preference scores, that’s what actually matters. Head to theslopindex.com, filter by your use case (email, Slack, social media, essays), and pick the model that real humans rated as most authentic. The creator’s key insight: human judgment is “most important as everything else are just heuristics for this.”
I benchmarked which of 18 AI models writes the least like “AI slop”
by u/penguinothepenguin in PromptEngineering