The Economics of AI Inference at Scale

A founder DM'd me last week. His agent product was doing about 400 million output tokens a month on Claude Opus 5 and the bill had just crossed $10,000. He wanted to know if he should self host. I asked him three questions: what's your average prompt length, what percentage of your traffic hits the same system prompt, and are you using prompt caching. He didn't know the answers to any of them. That is the actual state of inference economics in 2026 — a bill nobody reads, sitting on top of a stack nobody instruments, priced against benchmarks nobody trusts. The interesting thing about inference cost is not the sticker on Anthropic's pricing page. It's the delta between what you get charged and what the workload actually costs the provider to serve — because that delta is where DeepSeek eats OpenAI's lunch, where prompt caching turns a $10K bill into a $2K bill, and where "we should just self host" goes from a founder fantasy to a real decision. This post is the math I wish somebody had shown me the first time I stared at a five figure API invoice and wondered whether I was being ripped off. The per token price is not what your workload costs The number on the pricing page is a menu, not a bill. Claude Opus 5 is $5/M input and $25/M output. GPT 5.5 is $5/M input and $30/M output. DeepSeek V3.2 is $0.28/M input and $0.42/M output. Those are the numbers everyone quotes. They are also the numbers everyone gets wrong to reason from, because real workloads are 70 90% input tokens and the input side is where the real levers live. An agent product with a 6,000 token system prompt calling Claude Opus 5 20,000 times a day is…