You get a video from a friend or a parent. “Look at this crazy thing someone invented.” You watch it twice. Something feels off. Is it real, or is it AI slop? I keep landing in that exact spot, and I hate the guessing.
So I got hooked on this experiment from Matt Wolfe, the creator behind Future Tools. He set out to build a simple website where you paste a link from YouTube, TikTok, Instagram, or X, and it tells you if you’re looking at an AI video. The catch is that he did the whole build using AI coding tools, and the results say a lot about where we actually are with this tech.
Before you go trying to build your own detector or just eyeballing every clip, here’s the fast way to think about it, based on what this AI professional found.
What actually matters when detecting AI
The creator set up a few criteria worth stealing:
- Accuracy on obvious fakes. If a human can instantly tell, the tool should too.
- No false confidence. It should never scream “real” when it has no clue.
- Cost you can predict. A detector nobody can afford to run isn’t a public tool.
- It has to work on real links, not just uploaded files.
The approaches he tested (pros and cons)
He tried a few paths, and the comparison is the useful part.
🟢 Google’s SynthID + Gemini video understanding
- Pros: Gemini is one of the only models that can actually watch a video and describe what’s happening. SynthID can flag content made with Google’s own AI.
- Cons: In his tests, Gemini almost never caught AI on its own. It kept coming back with “no clear indicators,” even on clips any person would spot in a second.
🔴 Pure LLM coding agents doing the logic
- Pros: They wrote the whole app, ran their own tests, and even graded themselves over and over.
- Cons: One run went over 8 hours and still came back “goal stalled.” Early versions handed out “high confidence” verdicts pulled straight from a raw number, which is exactly the kind of false certainty you don’t want.
Third-party detection API (SiteEngine)
- Pros: This was the piece that actually worked. It uses computer vision to flag AI frames, and it correctly caught the obvious fakes he threw at it.
- Cons: It’s paid, and the pricing is brutal to predict. More on that below.
Where it landed
Here’s the recommendation I’d pull from his whole journey: don’t trust a single model, and don’t trust a slick verdict. The build only started working once he leaned on a dedicated detection API instead of asking a general AI to “just tell me.”
Once SiteEngine took priority in the logic, the tool finally behaved. It correctly cleared real talking-head videos from creators he knows. It flagged the clearly fake stuff, like a woman jetting across water in an inflatable suit and a plane that shrinks so a guy can walk around inside it. That’s the outcome he was chasing.
But the honest part is what makes this worth sharing. Even after roughly 10 hours of prompting, the app still choked on a video that was obviously AI, calling it “inconclusive” because the API said fake and Gemini disagreed. When two systems argue, you get a shrug.
The cost reality
This is the deal-breaker for a free public site. The API pricing is measured in “operations,” and nobody can tell what one operation costs you up front. In his own light testing, he burned over 12,000 operations across just six videos. One recent scan alone ate 440. On a $100-a-month tier that reportedly gives 40 operations, letting the internet run wild with it makes no sense. That’s why he chose to publish the code instead of hosting it.
If you want to try it yourself
The person who built it put everything on GitHub so you can run it locally. Rough steps based on what he described:
- Grab the project code from the creator’s GitHub repo.
- Sign up for your own detection API key (he used SiteEngine).
- Drop your API user and secret into the environment variables file.
- Run it locally and paste in a link to scan.
- Watch your operation usage closely, because it climbs fast.
The bigger takeaway
Here’s the point that stuck with me. We keep hearing that AGI has arrived. This savvy professional isn’t buying it, and his reasoning is sharp. If these models are supposedly as capable as any human, why can a regular person spot AI slop in seconds while the best available models say “inconclusive”? The one model that can even watch video, Gemini, kept missing the fakes entirely. The thing that actually worked was an older-style computer vision tool, not the flashy generative models getting the AGI headlines.
He also flagged the cat-and-mouse trap: the better detectors get, the better the fakes get at hiding. So this problem isn’t getting simpler soon.
My honest read? A dedicated detection API beats asking a chatbot “is this real,” but neither is a magic shield yet. Treat any verdict as a hint, not a ruling.
Want to see the full build, the failed 8-hour run, and every awkward “inconclusive” moment? Check out the full video for the whole story, and grab the code if you want to run your own detector.