Intelligence Falls to $2 a Million Tokens While Micron Posts an 87% Margin

Frontier model prices are converging at $2 per million input tokens, yet the memory that runs them is getting scarcer. Investors may want to ask what becomes scarcer as AI usage grows.

The economics of AI are inverting. API prices for frontier models are converging at $2 per million input tokens, yet the memory and context storage that run them are becoming a bottleneck. The question for investors is less how smart AI has become than what grows scarcer as AI usage rises.

Frontier AI converges on $2

On Sept. 22, OpenAI launched GPT-6 Sol at $2 per million input tokens and $10 for output. On Sept. 28, Anthropic priced Claude Sonnet 5.5 the same way. On Sept. 30, Google set the introductory price of Gemini 4 Argon at $2 and $10 as well.

Price is not the only thing that moved. According to Google, Argon ranked first on the long-horizon software engineering benchmark DeepSWE v1.1 at 77.9% and on Zapier AutomationBench at 51.3%, and it raised the maximum output limit to 1 million tokens. Anthropic says Sonnet 5.5 keeps the same per-token price as the prior generation but needs fewer tokens for the same task, cutting cost per task by up to 30%.

The unit of AI price competition is shifting from "cost per token" to "cost to finish one piece of work."

Epoch AI, which tracks the cost of reaching a given level of performance, finds that costs have fallen about 47% per quarter since 2023, or roughly 13x a year. The direction looks less like a temporary promotion than a structural trend.

Meanwhile, Micron posted an 87% gross margin

Software prices are falling, but physical infrastructure is moving the other way. Micron's fiscal fourth-quarter 2026 revenue was $54.229 billion, about 4.8 times the $11.315 billion a year earlier. Non-GAAP EPS was $33.42, and non-GAAP gross margin was 87.0%.

  • $54.23B: FY4Q26 revenue, up about 379% from a year earlier
  • 87.0%: FY4Q26 non-GAAP gross margin
  • $61.5B: midpoint of FY1Q27 revenue guidance

The contract figures stand out more. According to Reuters, customer financial commitments tied to long-term supply agreements rose from $22 billion in June to $32 billion, mostly as cash deposits. Micron's remaining performance obligations grew to about $150 billion. The company expects memory and storage supply to be tighter in fiscal 2027 and 2028 than in 2026, and says it has already secured contracts for most of its 2027 HBM output.

Reading this as "the memory cycle is gone" would be risky. Long-term contracts improve revenue visibility and strengthen the supplier's bargaining power. They do not rule out customer capex adjustments, technology shifts or a downturn once new supply arrives.

Jevons paradox: cheaper AI may need more infrastructure

The mechanism is simple. When intelligence gets cheaper, the same budget buys more calls. A chatbot answers once. An agent plans, searches, runs code and fixes failures in a loop, so token use per task rises sharply.

If token prices halve and usage more than doubles, total infrastructure demand does not fall. That is why the Jevons paradox, in which efficiency gains raise total consumption, could show up in AI. It is an economic hypothesis, not an automatic law. What decides the outcome is how far usage growth outruns the pace of price declines.

The next bottleneck is not only HBM: KV cache and SSDs

As long contexts and multi-turn agents spread, GPUs must do more than compute. They must keep prior context available. That is the job of the KV cache. Its size varies widely with model architecture, precision and the number of KV heads, so it is wrong to say that 1 million tokens always needs 320GB. Still, one research example calculated that a 1-million-token context for Llama 3 70B needs about 320GB of KV cache, far beyond a single 80GB H100.

That pushes designs toward tiering context across CPU DRAM and flash storage rather than holding it all in HBM. NVIDIA presents its BlueField-4-based CMX as a shared context layer that extends GPU memory, and claims up to 5x the throughput and power efficiency of conventional storage.

Micron's earnings materials are notable here. The company says its PCIe Gen 5 7600 SSD and Gen 6 9650 SSD are already shipping to major customers for KV-cache applications. It is a real commercial signal that the investment map for "AI memory" could widen from HBM to server DRAM and enterprise SSDs.

Who benefits across the value chain

LayerPositivesKey risk
Memory and storageHBM shortage, expanding long-term contracts, demand for server DRAM and SSDs for KV cachePrice spikes may prompt customer optimization and capacity additions
Cloud and big techCheaper models drive more usage and cloud trafficHigher memory, power and network costs squeeze inference margins
Vertical AIFalling model costs for a key input; room for higher-value, outcome-based pricingFoundation models expanding features and bundling
General-purpose model providersScale economies and higher usageIntensifying price competition; token margins compress without differentiation
AI prices are falling, but the memory that runs AI is selling at very high prices right now.

Insight Times Editorial Desk

More by topic

Starting with this story's topics, then what readers are following now

Investor psychology, decision-making and books on investing

HBM

This story's topic78 storiesSee all →

Memory stacked next to the GPU.

Memory Is Still Short, but Less Short: Checking for the Peak This Week

Citi trimmed its 2027 DRAM and NAND shortage forecasts together. Server pricing remains strong, while consumer markets show weakness. This week's numbers show where the memory boom may peak.

Investing

A 3x P/E is not a discount. It is a verdict on how long memory profits last

Nomura's buy case on Samsung Electronics and SK Hynix does not stop at higher memory prices. It argues that if AI has turned memory from a cyclical component into strategic infrastructure, the formula the market uses to value both companies has to change too.

Investing

AI ran out of memory before it ran out of GPUs

The HBM bottleneck is spreading into DRAM prices, server costs and the price of the next laptop or phone.

Tech

2027 Isn't the End: Citi's Case for a Memory Supercycle Running to 2031

Citi says the memory shortage timeline is stretching again. If AI shifts from "train once" models to continual learning, HBM, server DRAM and enterprise SSDs could all fall short at the same time.

Tech

Has Memory Peaked? Prices Are Cooling, but the Shortage Looks Set to Run to 2028

Price growth has slowed and consumer demand is already breaking. Server and HBM supply remains short, and Micron says more than 75% of its 2027 volume is already committed to customers.

Tech

AI Chip Peak Check: Demand Still Rising, Risk Building in 2027-2029 Supply and Funding

DRAM and NAND contract prices are still climbing and HBM suppliers hold the edge. Power shortages, AI financing costs and simultaneous memory expansion are the risks for later years.

Investing

The Real ASML Number Isn't the 3% Pop, It's the 30% Capacity Hike

ASML's rally isn't just an AI trade. Customers are already locking in 2027 tool slots, and ASML is raising Low-NA EUV capacity by roughly 30% to keep up.

Investing

A building that houses servers and networking gear and supplies them with powe…

Why Nebius Is Keeping Some 2027 AI Capacity Off the Shelf

Nebius mixes long-term contracts, short-term premiums and auctions. Holding back capacity pays only if the servers stay busy, and it is not yet clear the pricing will last.

Tech

Is Space Cooler for AI? Why Musk Wants Data Centers in Orbit

The next limit on AI may be power and cooling, not GPUs. Elon Musk is looking to Earth orbit for an answer, but the test is cost per hour of compute.

Tech

White House Swaps AI for "SI." The Real Investment Signal Is Infrastructure, Not the Name

An executive order renames artificial intelligence "Super Intelligence" and six frontier labs sign a voluntary safety accord. What investors should watch is the data center, power, nuclear and chip policy now gathered under one national-competitiveness frame.

Tech

Musk Sees the Next Phase of AI Infrastructure: Who Can Switch GPUs On, Fast

If neoclouds struggle under financial pressure, AI demand may not shrink. It may simply favor the few operators with the capital and execution to serve it, a shift that could matter for SpaceX and Tesla.

Tech

Weekly Check: The AI Bull Market Isn't Over, It's Just Gotten Harder

A 5% 10-year yield, 470 gigawatts stacked up in Texas, Chinese DRAM, and whether data centers actually get switched on are starting to decide who wins the AI trade through 2027.

Tech

Is Memory Peaking? In the AI Era, Watch the Bottleneck, Not the Demand

Demand still outruns supply. But memory stocks can peak before demand does. The real question is how long HBM4, server DRAM, CXL and Big Tech capex keep the cash flowing.

Tech

Marvell's $115 Billion Lumentum Deal: Is AI's Next Bottleneck Light, Not Chips?

Marvell's purchase of Lumentum points to a contest beyond GPU speed, in data transfer and power efficiency. Whether the price is right is not yet confirmed.

Tech

NVDA

101 storiesSee all →

Who Hits $1 Trillion Next: Reading AMD and ASML by Different Rules

One company is chasing Nvidia's GPU empire. The other sells the machine that draws the chips for every advanced chipmaker, including Nvidia. Both get floated as 2028 trillion-dollar candidates, but the numbers investors need to watch are entirely different.

Investing

Box Shows AI Its Manuals, and the AI Starts Redesigning the Workflow

Box Architect, built with NVIDIA, moves AI from reading documents to designing how work gets done. It has not yet reached beta, so the real-world payoff remains to be seen.

Tech

Tech Selloff: Is the Bull Market Over? Wall Street's Four-Year Chart Says Not Yet

The uptrend has survived five double-digit pullbacks in four years. This time, AI companies' debt and cash flow are the new variables.

Investing

Nvidia and Palantir Put AI Inside the Supply Chain - The Real Shift Is That It's Learning Judgment

A single Vera Rubin rack holds 1.3 million parts. The real breakthrough in Nvidia and Palantir's new supply chain AI isn't automating that complexity, it's capturing the judgment calls that used to live only in planners' heads.

Tech

If You Only Watch Tesla, You May Be Missing Half the AI Story

Nvidia builds the AI brain. Who supplies the electricity to run it? Six power companies Tesla investors should know, and why Tesla's storage business is just one link in the chain.

Investing

Nvidia Moves From GPUs to CPUs as AI Agents Reshape a $200 Billion Chip Market

A customer's early test found Nvidia's Vera CPU handled about twice the AI agent workload of its existing chips. The bigger story is where the money in semiconductors is heading as AI shifts from answering questions to doing work.

Tech

Nvidia's $500 Billion Experiment: Wall Street Asks Who Backs the GPU, Not Just the Chip

The real issue in Nvidia's financing platform is not chip sales. It is how far Wall Street will trust GPU resale prices and rental income three to four years out.

Tech

GPU

73 storiesSee all →

A chip built to draw graphics, whose ability to run thousands of identical cal…

The Next AI Chip Race Is About How Chips Connect

Applied Materials will lift its stake in Dutch equipment maker BESI from about 9% to 20%. The bet is less about control than about a new way to link chips.

Tech

US AI Data Centers Are Headed for 90GW. Gigawatts Now Matter More Than Dollars

Goldman Sachs expects US data center capacity to grow from 64GW at the end of 2026 to 90GW at the end of 2027. The AI race is moving from buying GPUs to building power plants and transmission lines alongside them.

Tech

The Cheaper Tokens Get, the Bigger Your AI Bill Gets

As AI shifts from training to inference, the token is becoming an industrial commodity. Falling prices are not cutting demand, because agents and automation consume tokens at a scale no human chat session ever did.

Investing

Dark navy cover. Flow diagram: Nvidia to Nscale (credit guarantee, in talks) to Anthropic (~$45B compute lease signed in August plus a separate ~$45B, ~460MW deal in talks). Title: Why Would Nvidia Back a Customer's Loan?

Why Would Nvidia Guarantee Its Customers' Data Center Loans?

Nvidia is reportedly in talks to back financing for Nscale's West Virginia data center. Nothing is signed, and the terms are not public.

Tech

Two Companies Drive 34% of S&P 500 Earnings Growth as AI Pays Sellers First

Nvidia and Micron generate a third of expected earnings growth in US large caps. Apollo says the financial effect of AI is not yet visible in the S&P 493.

Investing

Amazon Goes After Nvidia's Strongest Moat, and It Is Software, Not GPUs

Amazon's open-source TorchNeuron aims to let existing AI code run on its Trainium chips. The contest comes down to the total cost of AI compute and the return on Amazon's $220 billion in annual capital spending.

Tech

AI's Next Bottleneck Is Power, Not GPUs

Jensen Huang and Elon Musk sent the same warning in Washington. The AI race is widening from building better models to supplying power and chips at industrial scale.

Tech