Part 3 of 3: CPU economics for agentic AI.
The market is finally facing some political uncertainty. On November 5, the Philadelphia Semiconductor Index fell more than 12% on fears that Democratic election wins would bring tighter rules on AI data centers. Many of the companies we cover at BEP Research have had significant drawdowns, even though the businesses and the technologies are doing well. I don’t think making investment decisions based on politics is a wise long-term solution or way to go about it.
I keep hearing that you can just use the cheapest CPU to run AI agents. Others say the chip should be optimized to keep the GPUs busy and the cost per task down. Two parts into this series, we couldn’t settle that without first pinning down what an agentic CPU even is. So we wrote five tests and ran every contender through them. The headline numbers shrank, Vera’s included.
We’ve made the cheap-CPU case ourselves. In May, in The Last x86 Island, we wrote that “at the GPU-heavy ratios of a Vera Rubin rack, the head-node CPU is a small share of cost and power.” The head node is the CPU that hands work to the GPUs, and in agent work it spends a lot of its time waiting on a model or a tool.
In October, OpenAI’s compute chief said in the NVIDIA and Oracle announcement that OpenAI is bringing Vera, NVIDIA’s own CPU, into production for agent sandboxes at scale. No timeline, no volume. (A sandbox is the sealed-off workspace where an agent runs code.) Modal, a cloud that runs AI code in sandboxes, put out an open benchmark that only counts a run when the task’s own test passes. Almost every agent result so far was run by a vendor or with one’s help, from Vera’s NVIDIA partners to AMD’s and Intel’s own tests and the study Arm put out.
An Agentic CPU Is the Host That Runs Everything Around the Model
An agentic CPU is the processor that does an agent’s work between model calls: running tools and code, spinning up sandboxes, checking results and keeping the context the next call needs. On Amazon’s October 29 call, Andy Jassy said “post-training reinforcement learning and agent tool use is mostly done on CPUs versus AI accelerators.”
Part 1 showed why this is its own job: the next model call often waits on a test, and fewer steps can beat faster hardware. Part 2 priced it, and the biggest lever was how often a task passes on the first try. The work waits, then bursts, and when it runs slow it can leave an expensive GPU sitting idle. We grade every contender on five things:
1. Speed under load. How much correct work one machine finishes as it fills up with agents.
2. Room for context. Where an agent’s saved context can sit between steps, and how fast it comes back.
3. Price per finished task. Total cost, retries included, divided by the tasks finally accepted, at the same quality and deadline (Part 2’s Full Bill).
4. Supply. Whether a buyer can get the chip in volume this year.
5. Moving cost. What it takes to move a fleet’s software onto it.
On Speed, Vera’s Lead Depends on the Rival
Daytona, which runs sandboxes for AI agents, packed one machine with up to 2,000 sandboxes on the same coding job. Vera peaked at 13.3 jobs a second against 11.1 on AMD’s current 5th Gen EPYC 9755, a 128-core chip, and still did 12.7 against 10.9 fully loaded. That’s about 20% at peak and 17% packed, the 17% we printed in Part 1, and the lead doesn’t grow as the machine fills. In July, DeepInfra, an AI cloud, found the same 1.2x on a single agent step.
Against Intel the gap is bigger, and it shrinks under load. DeepInfra’s single step ran 1.76x faster than on Intel’s Granite Rapids Xeon. Packed with agents, with 99 of every 100 responses due back within 6 seconds, Vera held only 1.33x as many. In Modal’s data, Vera beat a dedicated Granite Rapids server by 1.68x with 50 sandboxes running and 1.42x with 150, and rented Graviton machines on Amazon’s cloud by 1.2x to 2.3x.
Intel’s own sandbox study has its Xeon 6990E+ doing about 313 sandbox starts per chip against 246 on AMD’s EPYC 9965. Vera wasn’t in it. AMD’s September materials put Venice, its 6th Gen EPYC, at 1.2x Vera per core, using a 96-core version of a chip that goes up to 256 cores and NVIDIA’s own Vera score. On its November 3 call, AMD said Venice beats every other server CPU per core, per watt and per dollar, whether x86 (the Intel and AMD chip family) or Arm. There’s no methodology for it, but obviously they’re going to lead with it.
NVIDIA made its own case in a November 5 blog post. In what it calls a 100-megawatt data center comparison, it says Vera supports 1.6x more agents per watt than the fastest x86 CPU and runs agents 2x faster than the most energy-efficient one. It doesn’t name either chip or the workload, and the two multiples are against two different chips. The post also cites DeepInfra at up to 2.2x the speed of “the best x86 processor.” In DeepInfra’s own data, that 2.2x is against Intel’s older Sapphire Rapids. Against AMD’s current EPYC it’s 1.2x.
Arm’s numbers came out at the Open Compute Project summit (OCP) in October, in a Signal65 study released alongside Arm’s own proposed way to measure agent work. It credits Arm’s AGI CPU with about 2x the work per rack of AMD’s EPYC 9755 across seven workloads, two of them full agent tasks. Per chip, Futurum’s read of the appendix puts it at 1.2x on average, about what Vera showed over the same AMD chip in other tests, and AMD wins two of the seven. The rest of the 2x comes from fitting more Arm servers into a 24-kilowatt rack, using Arm’s own power ratings.
DeepInfra and Modal also compare chips at equal core counts. That flatters a chip with more memory bandwidth per core (how fast it moves data in and out of memory), but buyers rent the whole machine. Venice puts up to 256 cores on one chip, which a crowd of small agents should like, and it hasnt run any of these tests.
Neither has AMD’s all-CPU Helion rack. AMD says it holds up to 144 Venice processors and nearly 74,000 threads, with the first system shipping this quarter. Untill someone runs agent work on one, those threads are a number on paper.
So here’s what we’ll dig into:
Which of our earlier CPU calls held in October, and which didn’t?
Can any CPU hold an agent’s saved context between steps, and has anyone measured it?
What’s the most a faster agent host can cost before it stops paying?
If the biggest buyers approve chips from every vendor during a shortage, who keeps the margin?
How do Vera, EPYC, Xeon, Graviton and Arm’s chip score?
What would prove our definition wrong, and by when?
So should a buyer just use the cheapest CPU to run the agents?
Let’s get into it...
