Local AI isn’t ready to replace your ChatGPT subscription. Not even close, and the numbers show it. I just watched a breakdown from Matthew Berman, an AI creator who covers this space full-time, and he ran the actual math.
Here’s what surprised me. The author should be the easiest person on earth to convince. He has a ton of compute and deep technical skills, and he even invested in LM Studio. He wants local AI to win. But when he needs real work done, he still opens ChatGPT or Claude.
🧠 The big idea
The expert draws a line most people blur. “Open-source” (or open weights) means a lab released its model for free. “Local” means you run that model on your own machine, and it works even with the internet unplugged.
The best open models are fantastic, just one step behind the frontier. The catch is size. They’re so huge they need racks of GPUs, not a home desktop. And as he puts it, most local AI talk online is “look what I’ve got running,” not “look what I actually got done.”
💸 The math that stings
The creator priced out real hardware and measured the speeds he got:
- RTX 4090 (~$5,000): a compact 27B Qwen model at 25 to 40 tokens per second. That’s barely usable.
- RTX 5090 (~$8,500): the same model at 40 to 70 tokens per second, but only for one task at a time.
- RTX Pro 6000 Blackwell (~$16,000): 95 to 245 tokens per second on a coding model. Top open models like GLM 5.3 Flash still need heavy “quantizing,” which shrinks the model and costs it some smarts.
- Mac Studio, 256 to 512GB ($10,000 to $15,000): it runs the big models, but only at 20 to 55 tokens per second.
Then comes the knockout punch. Spread a $4,000 rig over three years and it costs about $144 a month for a model scoring around 34 on the Artificial Analysis Intelligence Index. Claude Haiku 5.5 costs about $27 a month, runs at 235 tokens per second, and scores about 10 points higher. Every model on the “best quality for the lowest price” curve is closed.
🔍 Three things that stuck with me
- Parallel agents break local setups: One GPU splits its speed across every agent you run. Modern AI assistants spin up several agents for a single task, so this hurts fast.
- Data centers made your GPU pricier: Big labs are buying up chips and memory, so a 5090 jumped from roughly $3,000 to $4,000 to about $7,000 to $8,000. The speed stayed the same.
- Local AI is the new overclocking: For most fans, the fun is in the setup, not the results. You need strong tech skills, spare cash, lots of time, and real interest in tinkering.
🚀 Where it could actually work
This part got me excited. The author thinks local AI wins through a hybrid setup built into the operating system by Microsoft, Apple, and Google. Simple or sensitive tasks run locally, and the heavy work goes to the cloud. You shouldn’t even notice which one is doing the job.
Why does this matter? Three reasons:
- Data sovereignty: your private data stays on your device.
- No lockouts: labs can change the rules anytime, like when Anthropic blocked subscriptions from working with OpenClaw.
- Less concentrated power: owning your own intelligence spreads control around.
Quick tip: before you spend thousands, try LM Studio on hardware you already own. Then look at your tokens per second, not just whether the model loads.
The full video has the cost-versus-quality chart and the complete hardware breakdown. Go watch it before you buy that GPU!