The Cheaper Tokens Get, the Bigger Your AI Bill Gets

An unfamiliar unit is becoming an industry's output

Steel is measured in tons. Electricity in kWh. Oil in barrels. Telecom in gigabytes.

In the AI industry, the token is starting to play that role.

A token is the small unit of information a model uses to process text. When a user types a sentence, the model breaks it into tokens to read it, and it generates its answer one token at a time. Developers have known the concept for years. Ordinary users almost never saw it.

Then AI grew into an industry, and the situation changed.

API prices are quoted per million tokens. Data centers compete on how many tokens per second they can produce. How many tokens you generate from the same electricity has become a measure of efficiency. How well you use caching changes the unit cost of the same job.

NVIDIA calls this, bluntly, the AI Factory. If the factory of the industrial age turned raw materials into products, the AI factory turns power and compute into tokens. The core metrics NVIDIA points to are tokens per second, tokens per watt, cost per token, utilization and uptime.

There is a reason this does not read as pure marketing.

The deeper AI goes into real corporate work, the less it matters how clever a model was on a single answer. What matters is how much "intelligence workload" it processed reliably over a full day.

From the age of training to the age of inference

In the early years, the market's attention went almost entirely to training.

How many GPUs does it take to train a GPT-class model. How many parameters. What does the training run cost. How much better does the model get when you double its size.

Training is a huge project. But it happens a limited number of times.

Inference is different.

Inference happens every time a user runs a search. It happens when Copilot summarizes a document, when it writes code, when a support desk handles an inquiry. The computation repeats every time an ad is personalized, a document is classified, a contract is reviewed, a security alert is analyzed.

If training is building the factory, inference is the factory producing goods every day.

The deeper AI reaches into the economy, the more likely it is that industry scale is set not by a handful of enormous training runs but by inference repeating daily across billions of users and enterprise systems.

NVIDIA itself now describes the AI industry as moving, through 2026, from model development to production inference. Large-scale AI infrastructure stops being a one-off training facility and becomes equipment that produces tokens 24 hours a day.

Agents break the idea of "one question"

Then agents arrive, and the inference economy gets another layer more complex.

Picture a person asking a chatbot one question. They type it. The model answers. The exchange ends.

Agents do not work that way.

Hand an agent a task like "find the causes of customer churn last quarter and write a report," and it can run through many steps.

Find the relevant data. Read the documents. Query the database. Form an interim hypothesis. Go back for more information. Call another model or tool. Verify the calculations. Return to an earlier step if something is wrong. Write the final output.

To the user this is one job. Inside, it can be dozens of model calls, and for complex work, hundreds.

A Microsoft Research study of agentic coding tasks released in 2026 put numbers on the gap. The researchers found that agentic coding tasks can consume up to 1,000 times more tokens than ordinary code reasoning or code chat.

The more interesting finding is that token usage is not stable. Give the same task to the same model and total token consumption varied by as much as 30 times from run to run. Nor did spending more tokens keep improving accuracy.

The result shows that an AI agent is not simply a longer prompt. It is software with a new cost structure.

Jevons, in AI form: cheaper means more, not less

Here is the industry's most important paradox.

If models get more efficient and token prices fall, does GPU demand drop?

Intuitively it seems it should. If the same job costs a tenth as much, surely you need less compute.

The history of technology says otherwise, often.

The 19th century economist William Stanley Jevons observed that as steam engines got more coal-efficient, British coal consumption went up rather than down. Efficiency lowered the cost of use, and the range of things worth doing with it exploded.

Something similar can happen with AI.

Work that people did by hand because AI was too expensive starts moving to AI. Data that was analyzed once now gets analyzed hourly. Personalization once reserved for key accounts is offered to every customer. AI that only ran when a person asked becomes an agent working around the clock.

If the price per token falls to a tenth and usage rises a hundredfold, total spending goes up tenfold.

So there is a wide gap between the fact that inference is getting more efficient and the conclusion that total compute demand will shrink.

In practice, model vendors are cutting unit costs through faster models, caching, batch processing and dedicated inference chips, while simultaneously expanding features that consume more tokens: longer context, deeper reasoning, multi-agent architectures.

AI looks less like an industry where efficiency reduces consumption and more like one where every efficiency gain drags in a new set of tasks that were never economically viable before.

But a token boom is not a boom for every AI company

There is an important trap here.

Rising token consumption does not lift every AI company's profits.

A million tokens is not a million tokens. The economics differ completely.

As of September 2026, Anthropic prices Claude Sonnet 5 at $2 per million input tokens and $10 per million output tokens. Google Cloud sets different prices for standard, priority and batch processing depending on the model and the mode. Within the same model, cached input can be far cheaper.

Look only at the price lists and tokens seem like a fully standardized raw material.

But unlike a ton of steel or a kWh of electricity, AI tokens are not uniform in quality.

One model's 100,000 tokens can produce a better result than another model's million. Conversely, a more expensive model can run reasoning far longer than necessary and simply raise the bill.

OpenAI says the same thing in its own documentation: do not compare models on the headline price per million tokens. Tokenization, output volume and reasoning usage differ by model, so what you have to measure is the total cost of completing a representative task.

From that point on, a token is not a simple commodity like oil. It is a unit of compute with variable productivity.

The real unit is cost per outcome, not cost per token

Think about why companies buy AI at all, and the answer gets clearer.

No company adopts AI because it wants to consume tokens.

They want to handle support tickets more cheaply. They want to cut software development time. They want to raise sales rep productivity. They want to shorten contract review. They want to detect security incidents faster.

So the real economic unit of AI is not the token. It is the completed job.

Microsoft has recently described this as the cost-to-outcome curve.

The company is co-designing its own MAI models with silicon, and says running MAI models on Maia 200 improved performance per watt by 40 percent. But the more important goal, it says, is not simply cutting the price of a token. It is lowering the total cost of producing one work outcome.

The examples are telling.

According to Microsoft, MAI-Code-1-Flash, used in GitHub Copilot, achieved a higher code acceptance rate while cutting median token usage by 10 percent. On common Excel tasks, the company says it delivers quality comparable to GPT-5.6 at lower cost.

You can see the coming basis of competition.

Model A costs $1 per token but needs 10 million tokens to finish the job. Model B costs $3 per token but finishes in 1 million.

On the surface A is cheaper. In actual work cost, B is far cheaper.

Enterprise AI pricing competition is therefore likely to move from "dollars per million tokens" to "what does it cost to close one support ticket, one code fix, one contract review."

In the agent era, cost control becomes a software feature

Once agents are widely deployed, a new problem appears.

AI can spend a great deal of money on its own.

When a person uses a chatbot, costs are roughly predictable. But an agent that cannot solve a problem will search again, call the tool again, hand the work to another agent, and repeat its reasoning.

In a TokenOps post published in August 2026, Microsoft noted that a single agent run can generate hundreds of model calls. Each call looks cheap, but add up a full run and you can end up consuming more in tokens than the task itself is worth.

That creates a new management layer in enterprise software.

Agents will need more than permission management.

A budget per job. A cap on model calls. A cap on reasoning time. Per-model cost ceilings. A retry limit on failure. Cache hit rates. Cost measured against the value of the job.

These controls become necessary.

Just as people have travel policies and purchase limits, AI agents get a compute budget.

This matters for platform competition too. Companies will not simply shop for the smartest agent. They will want agents whose costs are predictable, whose runaway computation can be stopped, and whose failures can be traced.

Falling token prices may be the condition for market expansion, not a crisis

For investors there are two ways to read falling token prices.

The first reading is deflation.

Competition intensifies, inference chips improve, token prices keep sliding. Revenue and margins at AI companies come under pressure.

The second reading is market expansion.

As prices fall, work that was never economical moves to AI. More companies deploy agents, and existing users call AI more often. Unit prices drop, but volume grows much faster.

Which force is stronger will determine the size of the industry's profits.

That is why looking only at token prices tells you little about an AI company.

Is usage growth outpacing the decline in unit prices. Is the fall in inference cost faster than the cuts passed to customers. Does switching models improve gross margin. How is total token usage per agent run changing. How far do caching and batch processing actually pull down unit cost.

These are the numbers that matter.

Tokens will probably get cheaper. Total consumption of intelligence can grow far faster than that.

The biggest market in AI may be work, not questions

So far the generative AI market has mostly been measured in human questions.

How many people use ChatGPT. How many searches they run. How much code gets generated.

In the agent era the unit of the market can change.

Instead of one person asking AI 20 questions a day, 50 agents can spend the night working through customer data, code, security logs, inventory and sales material.

Human time is capped at 24 hours a day.

Software agents have no such constraint.

This is why the inference market's potential can be far larger than the training market's.

There is a natural ceiling on the number of questions humans ask. The ceiling on the volume of work a company can automate is much higher.

The largest source of demand in AI may end up being not "time humans spend asking AI questions" but "the total volume of work companies hand to machines."

Tokens are the compute consumed to process that work.

Seen this way, agents are not just a new AI product. They are the turning point that changes the unit of AI compute demand from a human click to a corporate business process.

What it really means to say tokens are becoming an industrial commodity

It does not mean tokens will become as fully standardized as steel or electricity.

The important part lies elsewhere.

As AI matures into an industry, you have to measure how much intelligence was produced, at what cost it was supplied, and how much economic result it created.

NVIDIA watches tokens per second and tokens per watt. Cloud providers watch inference cost per token and utilization. Model companies design pricing for input, output and cached tokens. Enterprise customers calculate AI cost per job. Investors watch whether all of it comes back as revenue and cash flow.

The tighter that chain gets, the more AI resembles a factor of production rather than a software feature.

From that point, "which model is smartest" stops explaining the industry.

The more important question is this.

Put in the same dollar of electricity and the same dollar of compute: who produces more useful intelligence?

And does that intelligence create far more than a dollar of economic value for the customer?

The next round of AI competition is likely to be fought on that productivity equation.

Sources

• NVIDIA, AI Factories: The New Infrastructure of Intelligence
• NVIDIA, Unlocks AI Compute at Scale
• Microsoft FY2026 Q4 Earnings Call
• Microsoft Research, How Do AI Agents Spend Your Money?
• OpenAI, Understanding and counting tokens
• Anthropic, Claude Sonnet 5
• Google Cloud, Agent Platform Pricing

Insight Times Editorial Desk