OpenAI CFO Sarah Friar has published what she calls a scorecard for the AI age, and it’s a direct shot at how most companies currently measure AI. According to OpenAI, the framework rests on four metrics: useful work delivered, cost per successful task, dependability, and return on compute. Not model benchmarks. Not token counts. Not how many seats you bought.
What stands out here is who’s saying it. Friar runs the finance function at the company selling the compute. When the CFO of the biggest AI vendor tells you to measure output rather than usage, that’s worth reading twice.
The measurement problem nobody solved
Most AI budgets right now are justified with vibes. A team buys licenses, people report feeling more productive, and finance signs off on the renewal. Meanwhile the CFO has no line item that connects spend to anything a board would recognize as a return.
Friar’s four metrics attack that gap directly:
- Useful work. Tasks actually completed and shipped, not prompts sent.
- Cost per successful task. The failed attempts count against you. A cheap model that needs five retries isn’t cheap.
- Dependability. Does it work the same way on Tuesday as it did on Monday? Inconsistent systems can’t be built on.
- Return on compute. Value produced per dollar of inference, treated the way you’d treat return on capital.
That third one is where most deployments quietly die. A system that works 85 percent of the time sounds impressive in a demo and is nearly useless in a workflow where someone has to check every output anyway.
Why this lands now
The timing isn’t accidental. Enterprise AI spend has scaled fast, and the questions coming from finance departments have gotten sharper. The honeymoon phase, where “we’re experimenting with AI” was a sufficient answer, is closing.
There’s also a structural shift underneath. As models get cheaper per token, the interesting cost question moves from price-per-token to cost-per-completed-outcome. Those are very different numbers. An agentic workflow burning through a long reasoning chain might cost more per task than a single call while still being dramatically cheaper than the human alternative. Token pricing alone can’t tell you that.
And OpenAI has a commercial interest in this reframe. If buyers measure cost per successful task instead of raw token price, expensive-but-reliable models look better than cheap-but-flaky ones. That doesn’t make the framework wrong. It does mean you should apply it with your own eyes open.
What to do about it
If you’re running AI inside a business, three moves are worth making in the next quarter:
- Pick one workflow and instrument it properly. Track completed tasks, failure rate, retry cost, and human review time. One well-measured process beats ten unmeasured pilots.
- Establish the human baseline first. You can’t compute return on compute without knowing what the task cost before. Most teams skip this and then can’t prove anything.
- Treat dependability as a gate, not a metric. Below a certain reliability threshold, a system doesn’t get deployed regardless of how good the cost numbers look.
For vendors, the implication is different. Benchmark scores are losing their sales power. Buyers who adopt this kind of scorecard will start asking for reliability data under their own conditions, and marketing claims won’t survive that conversation.
Where this goes
Expect scorecards like this to become standard procurement language within two years. The pattern mirrors what happened with cloud spend: a wave of enthusiastic adoption, then a correction where FinOps teams appeared and started attaching numbers to everything. AI is entering its FinOps phase.
The companies that build this measurement discipline early will make better calls about where to expand and where to cut. The ones that don’t will keep renewing contracts on the strength of anecdotes until someone in finance asks a question they can’t answer.
Friar’s framework won’t be the only scorecard that emerges. But it’s the first credible one from inside the industry, and that gives it a head start on becoming the default. Full details are available in OpenAI’s original piece.