Part 1 of 3: CPU economics for agentic AI, leading into GTC Berlin
Start with 2,000 concurrent agent execution environments. Each sandbox gives the work a separate software environment. Now give them a coding workflow: work through a broken Python package, apply a patch and run the tests.
That is the kind of work Daytona benchmarked. At 2,000 concurrent sandboxes, it reported 12.7 jobs per second on NVIDIA Vera, against 10.9 on AMD EPYC 9755 and 8.1 on EPYC 9575F. Vera’s lead was about 17% over the stronger AMD result.
These results get closer to what an agent operator buys than a core count: work completed by execution environments. They are Daytona’s vendor-reported numbers for one coding workflow, excluding the economics of a complete application making model calls, recovering from mistakes and delivering work a customer accepts.
A model can propose a fix. Someone still has to run it.
NVIDIA’s OCP materials give that work a platform of its own: the Vera CPU rack, built for tool calling, orchestration and sandboxed execution. Alongside the GPUs that run models, NVIDIA wants to supply the computers where agents act on their answers.
We favor NVIDIA’s system approach for demanding agent workloads. But the size of this opportunity, and the winner within it, will turn on a question that neither core counts nor token prices answer: what does it cost to complete the customer’s task correctly?
That is the starting point for this three-part CPU series. We will follow the work into the hardware, then put the hardware next to the bill as we head toward GTC Berlin.
The next model call has to wait for the test
A coding agent reads a repository, asks a model what to change, edits the files and runs the tests. The test result determines the next move. A failed test can send it around the loop again.
Part of that loop runs on accelerators. Another part runs in a CPU environment. Some of the elapsed time is spent waiting for storage, a network response or another service. Faster hardware helps only where it shortens a step the job actually depends on.
In July’s Vera analysis, we focused on this dependent work. A hundred spare cores cannot tell an agent whether its patch passed a test that is still running. Faster execution can get it to the next decision sooner.
At fleet scale, however, there are thousands of other tests to run. Core count matters because independent jobs can run together. Memory matters because each environment needs room to work. Memory bandwidth matters because those cores need data. The operator needs both quick individual jobs and a fleet that keeps delivering under load.
There is an important accounting limit here. An inference server can answer other agents while one agent runs a test. A second saved on a tool call does not automatically free an entire second of otherwise idle GPU capacity. It may still improve turnaround, reduce the resources held by the job or let the execution fleet handle more work. Those are benefits worth measuring on their own.
The CPU execution fleet needs its own measured economics.
How agent tool calls execute in isolated environments across a shared CPU fleet. Illustration; not a measured capacity benchmark.
A million users can bring very different CPU bills
Our September Muse analysis began with a practical problem: getting an agent through a task on the actual internet. Human checks, merchant restrictions and slow handoffs can dominate the experience. None tells us how busy the processor is.
A browser waiting for a company to respond and a coding environment running tests may both appear as one active agent. Their demands on the fleet can be very different. Even the same agent can move between long waits and bursts of execution.
The waiting environment still needs its files and state preserved. An operator can keep it in memory, share CPU time with other work, or pause it and pay the cost of restoring it later. Five minutes between request and result need not mean five minutes of CPU execution. Cloudflare’s documentation, for example, explicitly separates execution time from waiting on network requests; billing for a continuously allocated virtual machine can work differently.
The difficult capacity question arrives when many of those agents wake together. Memory can fill. Tests can queue. A fleet that looks cheap on average can miss the response time customers were promised.
That is why I would be careful with any forecast that goes straight from daily active users to physical CPUs. We need the task mix, how often users delegate, how long environments stay resident and how much work arrives at the busiest times. Consumer distribution creates an opportunity. It does not specify the hardware order.
Faster hardware can still lose to fewer steps
The software determines how much execution capacity turns into finished work. Run an unnecessary test faster and the job still carries the cost of that test. Remove the unnecessary test and the operator can save both execution time and the resources held while waiting. The valuable CPU improvement is the one that survives a well-designed workflow.
That changes how we would evaluate a hardware premium. First understand the job’s required steps, its retries and the time spent waiting between them. Then ask which of those steps a different processor can improve. A cheaper model call can be a poor bargain if it creates more work downstream; a faster CPU can be a poor bargain if the bottleneck sits in an external service.
One concrete example comes from FUNDA’s October 6 computer-use tests: an experiment using a faster decision model slowed execution because Gmail page-state changes, including autosave, triggered repeated handoffs. That is a workflow observation from the authors’ GUI-only Mac testing, not a CPU benchmark.
The investment tension follows directly. As more jobs become worth delegating, execution becomes a recurring expense. As software removes unnecessary work, execution demand per job can fall. More agents and fewer resources per agent can happen together. The hardware opportunity depends on which effect is larger, and on whether customers spend the savings on more work.
NVIDIA is putting a rack around the execution layer
This is what makes NVIDIA’s plans for the OCP Global Summit relevant to the CPU argument.
NVIDIA distinguishes two systems in its OCP materials. Vera Rubin NVL72 handles large-scale training and inference. The Vera CPU rack targets tool calling, orchestration and sandboxed execution. An agent can use the first to decide what to do and the second to carry out the work.
That expands the competitive question beyond the host CPU installed beside a GPU. NVIDIA is pursuing the execution fleet as a market in its own right. The CPU pool and the inference pool can contribute to the same finished task while having different capacity requirements, utilization patterns and costs.
The company’s MGX contributions address how builders deploy those systems. NVIDIA is contributing rack and compute-tray designs through OCP, with the Vera CPU rack sharing the third-generation MGX platform’s approach to compute, networking, power and cooling.
MGX 1.2 includes blind-mate liquid cooling, fanless liquid-cooled compute trays and a liquid-cooled 5,000-amp busbar. NVIDIA says the architecture supports rack deployments up to 250 kilowatts. That is a specification for the architecture, not a claim that a Vera CPU rack draws 250 kilowatts.
Why NVIDIA wants others to build it
Our read is that opening this layer can expand the industry’s capacity to deploy NVIDIA systems. Equipment makers can build against common mechanical, power and cooling interfaces instead of repeatedly engineering bespoke infrastructure. More suppliers can prepare compatible equipment in parallel, while customers face less integration work. That could turn available manufacturing capacity into usable AI capacity faster.
The software contributions address the same problem after installation. NVIDIA’s OCP package adds open-source DSX components for network configuration, cluster readiness checks, rack management and bare-metal provisioning. These give cloud providers and software partners building blocks for bringing hardware into service. They could reduce duplicated operating work; the announcement does not quantify that saving.
There is a standard-setting advantage too. If more suppliers design around MGX interfaces and more operators adopt the associated tools, NVIDIA’s architecture can become an easier starting point for the next deployment. In our interpretation, NVIDIA can benefit by selling differentiated CPUs, GPUs, networking and interconnect products, alongside commercial software and services, while partners build more of the infrastructure around them. Sharing selected designs can enlarge the market for what NVIDIA sells.
The scope matters. Open rack and reference specifications do not mean open Vera or Rubin chip IP, and releasing named DSX components does not make the CUDA platform open source. Nor do open interfaces guarantee NVIDIA every sale: they can also give buyers more choice and competitors useful common infrastructure. The test is whether this ecosystem delivers available, economical systems at greater scale.
This is why we like the integrated approach. NVIDIA can work across the CPU, memory, accelerator connections and the infrastructure around them. Its Vera design pairs custom cores with LPDDR5X memory and supports coherent CPU–GPU connectivity. The attraction is the ability to improve several parts of a workload together.
The configuration still has to fit the job. A standalone sandbox does not get a benefit from a CPU–GPU connection it never uses. Open rack interfaces also do not make competing processors interchangeable. Part 2 will examine where these architectural choices can change the outcome and where a different CPU or an existing cloud service could be the better purchase.
Explore the hardware: Our BEP-hosted Vera Rubin explorer lets you trace CPU, memory and GPU connections. Adapted with permission from Ben Dicken.
Put the finished work next to the bill
Return to the 2,000 sandboxes. Vera’s roughly 17% throughput lead over EPYC 9755 is a reason to investigate a purchase. To make the purchase, we need the price of that capacity, its power and memory costs, and the performance the customer actually requires.
In a separate test starting 1,000 sandboxes, Daytona reported roughly eightfold faster startup on Vera than on EPYC 9755. Startup is one phase of a job. It may matter greatly for short-lived environments or bursts of new work; a long-running environment may amortize it over many operations. An eightfold startup improvement cannot be carried over to the entire agent task.
Our DevDay analysis used the measure we will carry through this series:
Cost per accepted task = total attributable cost of the workload divided by original tasks completed to the required quality and deadline.
Count an original task once. Keep failed attempts and retries in the bill. Include human review or correction when delivery requires it, and decide in advance whether a corrected result qualifies. A system does not get credit for finishing cheaply by skipping the hard jobs or lowering the acceptance standard.
The bill includes model calls, execution environments and the resources they use or retain. For an owned fleet, it also includes depreciation, power, cooling and operations. A customer’s cloud charges and a modeled operator cost are different views of the economics; bundled components should not be counted twice.
This gives us a way to compare systems without declaring a universal CPU winner. Run the same work, at the same quality and service target, and price what it took to deliver. More completed jobs per second is valuable. How much more the buyer should pay for them is the next question.
So What
My base case is that demanding agents create a meaningful CPU opportunity, and NVIDIA is well positioned where execution speed, memory behavior and coordination determine how much useful work an expensive system can deliver. The Vera CPU rack gives that thesis a clearer commercial target beyond the host processors inside accelerated systems.
The easy version of the story is that every new agent requires another slice of CPU capacity. The harder, investable version asks how that slice changes as the software improves, how much of it customers will pay for and which supplier captures the spending.
AMD’s EPYC results belong in that discussion now. Arm, Intel and cloud-designed CPUs belong in the architectural comparison that follows. Buyers can also improve scheduling, reuse environments or stay with infrastructure they already operate. NVIDIA has to beat those alternatives on the work that matters to the buyer, including the cost of moving it.
Our preference for NVIDIA remains a thesis to test. Daytona shows that CPU choice can change the output of a busy execution fleet. OCP shows NVIDIA organizing a platform and an ecosystem around that work. Our test connects the two: does the system deliver more accepted work for the money, after the software and the full bill are accounted for? That remains unproven.
We have already put a falsification test on the record. In September we set a March 31, 2027 checkpoint: independently verified agentic throughput per megawatt from AMD’s Venice or Helios at or above Vera Rubin’s would make our published Vera view wrong. That checkpoint stands. Adding a cost-and-quality test in this series does not move it. A competing system delivering the same accepted work within the same deadlines at a lower fully loaded cost would earn our preference for those jobs.
There is also a separate investment question after the engineering result: does cheaper work create enough additional demand to outweigh the reduction in hardware needed per task? A more efficient system can be good for the customer without producing the revenue outcome a supplier’s valuation requires.
Next, we will look inside the competing CPU designs and ask where their advantages survive a busy agent fleet. Part 3 will put the workload, acceptance test and full bill together. I will be at GTC Berlin, where Jensen Huang’s October 21 keynote gives us a near-term opportunity to test what the industry is selling against what buyers need.
Two thousand environments are a useful starting point. The result we are following is the work that comes out of them, and what it costs to get it right.
This opening installment is free. For paid subscribers, Part 2 will examine the competing CPU architectures and Part 3 will put completed work next to the full bill. Subscribe to follow the series into GTC Berlin.
Disclosure: I am long NVDA, META, AAPL, LITE, CRDO, TSEM, ALAB, WOLF, NOW, SMCI, BE, NBIS, ORCL (2027 LEAPS), and the AI-Science Basket as published. I have been publicly long NVDA since 2016. I advise private companies, none of which are named here. This is investment research, not investment advice. Do your own work.