Modal’s NVIDIA Vera tests put the CPU back in the AI-agent story.

What Modal measured

On October 19, Modal published early tests of NVIDIA’s Arm-compatible Vera CPU. Its largest highlighted gain is concurrent throughput; quiet-host task latency showed a less uniform advantage. Modal says supporting Arm requires infrastructure and image-build changes and describes a future rollout. The post does not establish general customer availability or a universal price advantage. [1]

Why the benchmark is called agentless

Modal links Agentless Agent Bench, a public benchmark for the host-side work associated with agent workloads. Its repository describes removing LLM time using local network caching. That scope matters: it lets engineers investigate a computing bottleneck without treating model response time as a CPU result. The implementation is inspectable, but publication of code is not independent replication of a hardware claim. [2]

Cortist’s reading is that the announcement deserves attention because it changes what teams should measure. A fast model can sit inside a slow workflow. Equally, a faster execution environment may have little effect when the workload spends most of its time waiting elsewhere. The value of the result depends on the mix of operations in a real application.

Follow the entire task, not only token generation

Imagine a coding assistant that receives a request, asks a model for a plan, opens a project, installs dependencies, runs checks and returns a patch. For performance analysis, those phases should have separate clocks. Combining them into one headline obscures which component caused a delay. Treat this as an example of an evaluation design, not a measurement of any particular product.

The first practical step is therefore a task timeline. Record waiting time, execution time, retries and the point at which a result becomes acceptable. Include unsuccessful runs; removing them can make a system look fast by ignoring the work that frustrates users. A provider benchmark becomes useful when a team can map its workload to the tested part of that timeline.

Throughput is not the same as responsiveness

Throughput answers how many tasks finish in a period. Latency answers how long one task takes. A service can improve one while harming the other. For an interactive assistant, slow outliers may dominate the user’s impression even when aggregate output looks excellent. For a background batch, the completion deadline and total capacity may matter more than the fastest individual run.

Our suggested comparison fixes the required service quality before optimizing density. Specify the maximum acceptable delay for the relevant task mix, then increase concurrency and observe where that target breaks. Report the distribution of completion times, not just the average. This makes it harder to buy a headline improvement that disappears at the operating point the business actually needs.

A simple thought experiment sets expectations

Suppose a hypothetical job takes ten seconds, including two seconds in the CPU-bound stage under consideration. Halving that stage reduces the full job to nine seconds: a ten percent reduction in elapsed time. This arithmetic is illustrative and is not a prediction for Vera. It explains why a twofold improvement in one component should never be copied directly into an end-to-end product claim.

Now consider a queue with many jobs waiting for the same execution capacity. Additional throughput could reduce waiting as well as execution pressure. That is a different mechanism, and its importance depends on arrival rates and scheduling. A credible pilot should therefore include both a quiet baseline and a realistic burst of concurrent work rather than choosing whichever scenario gives the largest ratio.

Treat architecture migration as a separate workstream

Before changing CPU architecture, inventory the actual software environment. Identify native extensions, third-party binaries, build tooling, container images and assumptions in the deployment process. Run representative tasks through the entire pipeline and compare outputs, not just whether a process starts. Compatibility is a property of the application and its dependencies, so one successful demonstration cannot settle every workload.

A staged evaluation would keep a known baseline available, record the exact image and dependency versions and send a bounded task set through both environments. Any output differences should be explained before optimizing performance. Teams should also account for the effort of maintaining more than one build path. That work can be justified, but it should appear in the business case rather than vanish behind a processor benchmark.

Ask for cost per accepted task

The useful denominator is completed work that meets the acceptance criteria. Include compute charges, failed runs, repeated execution and any additional operating effort. State the comparison period and workload mix. Hardware acquisition cost, power, hosting terms and software changes can all influence the outcome; this article does not supply enough evidence to calculate a Vera return on investment.

For teams buying a managed sandbox, there is an additional question: how does a host-level gain reach the customer? It might support more capacity, a different service target or a future price change. Those are possibilities, not announced customer benefits we can assume. Evaluate the service contract and observed behavior of the product you can actually use.