Google just open-sourced a compiler that lets AI models run predictions on data they can never actually read. It’s called HEIR, and according to Hacker News, it’s the newest addition to Google’s Private Computing Toolkit. The pitch is simple to state and hard to build: run AI inference directly on encrypted inputs, return encrypted results, and never expose the underlying user data at any point in the process.
The technology underneath is homomorphic encryption. Normally, encryption forces a trade-off. You can lock user data down with end-to-end encryption, but then the service provider can’t do anything useful with it, like spam filtering or fraud checks. Homomorphic encryption breaks that trade-off by letting servers compute on ciphertexts without decrypting them. A cloud service can recommend content without ever seeing what you like. As Hacker News reports, one of Google’s demos does exactly that.
What stands out here is the usability angle. Homomorphic encryption has existed for years, but converting a program to use it efficiently required a team of cryptographers. HEIR, which stands for Homomorphic Encryption Intermediate Representation, is a compiler toolchain that automates the hard part. It takes pre-trained AI models built for plain data and converts them to run on encrypted inputs. Google says its long-term vision is a one-click tool that lets non-experts add encrypted inference to production apps.
What HEIR actually does
- Compiles standard models to encrypted ones. You bring a pre-trained model that expects unencrypted data. HEIR rewrites it to operate on ciphertexts, so you don’t hand-tune the cryptography yourself.
- Keeps the guarantees purely cryptographic. Unlike hardware-based approaches such as secure enclaves, HEIR’s privacy comes from math, not trusted chips. That’s a meaningful distinction for sectors that don’t want to rely on a vendor’s hardware promises.
- Doubles as a research platform. Cryptographers can build on HEIR’s testing and benchmarking infrastructure instead of reinventing it. Google says four peer-reviewed papers already build on the project, with collaborations spanning Georgia Tech, Carnegie Mellon, Purdue, Tsinghua, and the University of Edinburgh.
- Works with hardware accelerators. Google has partnered with Belfort, Niobium, Cornami, and Optalysys, companies building chips to speed up homomorphic encryption. Latency benchmarks on those accelerators are promised in the near future.
Four working demos
Google shipped four private inference applications, all compiled with HEIR, with source code on GitHub and latency measured on a single-threaded CPU:
- Private content recommendations using a Deep Learning Recommendation Model, built with Belfort Labs, LG, and NYU.
- Credit card fraud detection, compiled with Niobium and hardshell.ai.
- Threat intrusion detection using the Kitsune anomaly system, which flags suspicious encrypted network traffic without the provider seeing packet contents.
- Hotword detection, built with Belfort Labs, so an audio-triggered agent can recognize wake words while keeping the recording private.
Why does this matter? Healthcare and finance sit on data they legally and ethically can’t share, which blocks a lot of otherwise useful AI. Local processing is one workaround, but it’s limited by device power and risks leaking the provider’s proprietary model. HEIR points at a third path: send encrypted data to a powerful server, get useful results back, and expose nothing in between.
The honest caveat, and Google states it plainly, is cost. Homomorphic encryption carries a real compute overhead. The company’s argument is that the trade-off has shifted from “can we do this at all” to “how much does it cost,” and that cost is dropping fast. That framing matters. It turns a hard cryptographic limit into an engineering and economics problem, which is the kind of problem the industry knows how to grind down over time.
The demos run single-threaded on a CPU today, so this isn’t drop-in production speed yet. But the direction is clear. If accelerator partners deliver the latency gains Google is teasing, private inference stops being a research curiosity and starts becoming a practical option for regulated industries. Anyone curious about the technical details and demo code can find the full write-up and repository at the original source.