Simon Willison just published a neat prototype for a problem that trips up a lot of developers: how do you store every past version of an edited document without your database ballooning? His answer, worked out on a dog walk and prototyped with an AI coding agent, is almost embarrassingly simple. Compress the whole history as one blob. According to Simon Willison, 1,000 simulated revisions that added up to 20.4 MB of raw text squeezed down to 80.3 KB using Zstandard compression.
What stands out here is how little machinery this takes. No diffing engine, no delta chains, no clever versioning library. Just compression doing what it does best on highly repetitive data.
Quick Start
You’ll learn a lightweight scheme for storing full revision history in a single SQLite database. You need SQLite, a compression library (zlib or Zstandard), and a way to serialize JSON. That’s it. The payoff: near-complete history at a fraction of the storage cost.
Why the naive approach fails
Start by understanding the problem it solves. The easy way to keep history is one row per version, storing the full previous value each time. That works, but it’s brutal on storage. If your document is 20 KB, every single edit adds another 20 KB to the database. A thousand edits, and you’re carrying megabytes of almost-identical text.
That redundancy is the key insight. Each version overlaps heavily with the last. Compression algorithms feast on repeated strings.
The scheme, step by step
- Bundle every version into one JSON array. Take the full text of every prior version and put it in a big JSON array of strings, from the start of the document’s life to now. This is the raw material compression will collapse.
- Compress the whole array. Apply zlib or, better, Zstandard (ZSTD) to the entire JSON array. Because every version repeats most of the text before it, the compressor wipes out the redundancy. This is where the 20.4 MB to 80.3 KB reduction comes from.
- Store it as a BLOB column. Add a history column to your table typed as a BLOB, so it holds binary data. Drop the compressed JSON array straight in.
- Keep timestamps in a second, uncompressed column. Store a parallel JSON array of timestamps as plain Unix integers. Simon Willison notes these don’t need compression at all. They’re small, and keeping them separate means you can read the timeline without touching the compressed blob.
That’s the entire design. Two columns: one magic compressed text array, one array of timestamps.
The optimization that makes it practical
Here’s the catch, and the fix. If your whole history lives in one blob, every single edit forces you to decompress the entire array, append the new version, and recompress it all. That gets slow as history grows.
Simon’s AI agent (he prompted GPT-5.6 to build Python prototypes, which it churned on for 38 minutes) suggested a clean solution: break the history into multiple rows. Cap each row at either 128 revisions or 3 MB of uncompressed JSON, whichever comes first. When a chunk fills up, start a new one. Now each edit only touches the current, smaller chunk instead of the full history.
Why this matters
This is a reminder that some storage problems don’t need specialized tooling. Compression on redundant text is a well-understood, decades-old trick, and pointing it at revision history is the kind of idea that’s obvious only in hindsight. For anyone building note apps, CMS platforms, wikis, or collaborative editors, it’s a cheap way to keep full history without a dedicated version-control layer.
It’s also a small case study in how AI tools are changing prototyping. Simon talked through the idea out loud using ChatGPT’s voice mode, then handed the concept to a coding agent that returned working Python. Idea to tested prototype in an afternoon.
Next steps
- Prototype it against your own data. Pull a real document’s edit history and measure your actual compression ratio. Zstandard usually beats zlib on both speed and size.
- Tune the chunk limits. 128 revisions or 3 MB is a starting point, not gospel. Adjust for your document sizes and edit frequency.
- Benchmark reads. Decide how often you actually need to reconstruct old versions, since that drives whether chunking helps or hurts you.
Full details, including the prototype files, are available at the original source.