When OpenAI introduced its advanced “reasoning” AI model, o3, in December, the organization collaborated with the developers of ARC-AGI, a specialized benchmark aimed at evaluating highly capable artificial intelligence systems, to showcase o3’s performance. However, months after the initial presentation, recent revisions present a somewhat less impressive image of the original results.
The Arc Prize Foundation, which oversees and administers the ARC-AGI benchmark, recently updated its estimates concerning the computing expenses associated with operating o3. Initially, the foundation calculated that the best-performing variant, known as o3 high, incurred approximately $3,000 in computing costs to solve one ARC-AGI challenge. However, according to the latest revisions, the Arc Prize Foundation now believes this number is significantly higher—potentially reaching around $30,000 per individual task.
This adjustment holds significance as it highlights just how costly some of today’s most advanced AI models may become for certain applications, particularly in their initial phases. OpenAI has not yet set or disclosed the official pricing scheme for the o3 model, nor has it made the model generally available. Still, the Arc Prize Foundation suggests that the pricing for OpenAI’s existing model, o1-pro, serves as a suitable point of reference.
To provide additional context, the o1-pro model currently represents OpenAI’s most expensive offering available to date.
“We consider o1-pro a more accurate comparison for calculating the realistic costs of o3, given the substantial amount of compute resources used during testing,” explained Mike Knoop, co-founder of the Arc Prize Foundation, in an interview with TechCrunch. “Nonetheless, this remains a provisional estimate; thus, we’ve indicated o3 as ‘preview’ on our official leaderboard, reflecting existing uncertainty pending OpenAI’s announcement of definitive pricing details.”
Given the extensive computing power that o3 high reportedly consumes, the elevated cost estimates shouldn’t be entirely surprising. According to data from the Arc Prize Foundation, the o3 high configuration required 172 times greater computing resources compared to the least demanding setup, o3 low, to solve the same set of ARC-AGI tasks.
Furthermore, speculation around premium-tier services that OpenAI may introduce for enterprise-level customers has circulated for some time now. Earlier in March, reports appeared in The Information suggesting that OpenAI might charge enterprise customers as much as $20,000 monthly for advanced, specialized AI “agents,” such as those tailored specifically for software development tasks.
Some observers could argue that despite their seemingly lofty price tags, even the most expensive OpenAI models might still prove to be more cost-effective compared to engaging human employees or external contractors to handle similar tasks. On the other hand, AI researcher Toby Ord highlighted in a post published on the platform X that these AI systems may lack efficiency relative to human workers. Illustrating this, Ord pointed to the fact that the o3 high configuration required an average of 1,024 separate attempts per problem in the ARC-AGI benchmark before obtaining its optimal result.