Earlier than this week, the dominant story about Nvidia went one thing like this: For the primary few years of the AI increase, Nvidia was the one supply for state-of-the-art GPUs, which grew to become immensely worthwhile because the trade scaled out. In the previous few years, hyperscalers like Amazon and Google have began constructing their very own chips, and Nvidia is now not the one recreation on the town, main many traders to marvel how sturdy its benefit actually is.
It’s a compelling story, and largely true. After rising its market cap 10x between the beginning of 2023 and mid-2025, Nvidia shares have been on a extra modest trajectory for the previous 12 months, pushed by considerations about GPU competitors.
A brand new narrative has taken form because the firm’s earnings on Wednesday and traders are beginning to understand that Nvidia’s benefit goes far past GPUs. As AI’s compute grows into the gigawatt scale, orchestration has develop into an more and more complicated job. Not surprisingly, Nvidia has constructed a lot of the state-of-the-art {hardware} wanted to deal with it, giving the corporate an enormous benefit within the methods that encompass the GPU even because it sees elevated competitors on the GPUs themselves.
For all of the discuss of compute as a commodity, it’s nonetheless extremely troublesome to function a megascale information heart at peak effectivity — and as deployments get greater and sooner, that problem is just rising.
Rack by Rack
You’ll be able to see a few of this simply by wanting on the particulars of what Nvidia is definitely promoting. The corporate is at present rolling out its Vera Rubin structure, which pairs the Rubin GPU with a set of different models, together with the Vera CPU, the Groq 3 LPX inference accelerator and comparable racks for storage and networking.
Over the previous week, I’ve been speaking to people at Nvidia about what these methods truly do, and the outcomes have been stunning. Just like the Rubin GPU itself, they’re extraordinarily specialised methods, however as a substitute of churning by means of tokens, they’re ensuring all the things exterior the GPU works as effectively as potential. If the GPU is the engine, these are the remainder of the automobile.
The Vera CPU particularly is concentrated on the issue of orchestrating information. “Vera is vital as a result of there’s solely a lot reminiscence you could put in a single server or any form of compute platform,” Jason Hardy, Nvidia’s VP of storage expertise, advised me.
As information facilities have scaled up computing energy, reminiscence capability has scaled up too, which is why firms like Micron have gotten wealthy within the second wave of the infrastructure increase. However getting that information to the GPU on the proper time isn’t easy — and as firms look to drive tokens-per-watt decrease and decrease, they’re realizing how vital that type of visitors course is.
“We noticed upwards of 3x enchancment in these operations, the place the Vera CPU is permitting for acceleration,” Hardy mentioned. “So now we are able to use our flash to its fullest potential, as a result of we are able to get all that efficiency out of it with out bottlenecking.”
You’ll be able to see variations of the identical downside exterior of Nvidia. When OpenAI developed its Jalapeño chip, a significant focus was avoiding these challenges completely by minimizing the quantity of information that must be moved round.
“We designed Jalapeño to reduce information motion and communication delays,” the corporate mentioned in a weblog put up earlier this month. “Its massive area permits the complete workload to stay inside one linked system, minimizing information motion and serving to the whole request keep quick and environment friendly from starting to finish.”
It’s a special method, avoiding information motion completely by conducting a workload inside one built-in chip. However the total logic is similar, growing effectivity with smarter visitors management as a substitute of simply extra processor cycles. That in flip opens up an entire new layer of infrastructure for firms to compete over.
This new concentrate on information orchestration isn’t mechanically a win for Nvidia. The corporate must compete with rival chipmakers and hyperscalers simply because it has with GPUs. However the competitors has moved to a brand new layer, the place constructing a rival GPU issues lower than with the ability to make the complete system work effectively.
And no less than within the early levels, Nvidia appears to have a commanding lead.
If you buy by means of hyperlinks in our articles, we could earn a small fee. This doesn’t have an effect on our editorial independence.
