Shortly after Amazon CEO Andy Jassy introduced AWS’s groundbreaking $50 billion funding cope with OpenAI, Amazon invited me on a non-public tour of the chip improvement lab on the coronary heart of the deal, at (largely*) its personal expense.
Industry experts are watching Amazon’s Trainium chip, created at that facility, for its implications for lower-cost AI inference and, doubtlessly, a dent in Nvidia’s close to monopoly.
Curious, I agreed to go.
My tour guides for the day had been the lab’s director, Kristopher King (pictured under proper) and director of engineering Mark Carroll (under left), in addition to the group’s PR one who organized the go to, Doron Aronson (pictured with yours really later within the story).
AWS has been Anthropic’s main cloud platform for the reason that AI lab’s early days — a relationship vital sufficient to outlive Anthropic later including Microsoft as a cloud companion as properly, and Amazon’s rising partnership with OpenAI.
The OpenAI deal makes AWS the unique supplier of the mannequin maker’s new AI agent builder, Frontier, which may develop into an necessary a part of OpenAI’s enterprise if brokers develop into as huge as Silicon Valley thinks they may. We’ll see if that exclusivity stands precisely as introduced. The Monetary Occasions reported this week that Microsoft could consider OpenAI’s cope with Amazon violates its personal cope with OpenAI, particularly with Redmond getting entry to all of OpenAI’s models and tech.
What makes AWS so interesting to OpenAI? As a part of this deal, the cloud big has agreed to produce OpenAI with 2 gigawatts of Trainium computing capability. It is a big dedication, on condition that Anthropic and Amazon’s personal Bedrock service are already consuming Trainium chips quicker than Amazon can produce them.
Techcrunch occasion
San Francisco, CA
|
October 13-15, 2026
There are 1.4 million Trainium chips deployed throughout all three generations, and Anthropic’s Claude runs on over 1 million of the Trainium2 chips deployed, the corporate mentioned.
It’s price noting that whereas Trainium was initially geared towards quicker, cheaper mannequin coaching (a much bigger precedence a few years in the past), it’s now tuned and used for inference as properly. Inference — the method of truly operating an AI mannequin to generate responses — is at the moment the most important efficiency bottleneck within the trade.
Working example: Trainium2 handles nearly all of the inference site visitors on Amazon’s Bedrock service, which helps the constructing of AI purposes by Amazon’s many enterprise prospects and permits the apps to make use of a number of fashions.
“Our buyer base is simply increasing as quick as we are able to get capability on the market,” King mentioned. “Bedrock may very well be as huge as EC2 at some point,” he added, referring to AWS’s behemoth compute cloud service.
Trainium vs. Nvidia
Past providing a substitute for Nvidia’s backlogged, hard-to-acquire GPUs, Amazon says its new chips operating on its new specialty Trn3 UltraServers value as much as 50% much less to run for comparable efficiency than utilizing traditional cloud servers.
Together with Trainium3, launched in December, this AWS group additionally constructed new Neuron switches, and Carroll says that combo is transformative.
“What that provides us is one thing enormous,” Carroll mentioned. The switches permit each Trainium3 chip to speak to each different chip in a mesh configuration, decreasing latency. “That’s why Trainium3 is breaking all types of data,” notably in “worth per energy,” he mentioned.
When trillions of tokens a day are concerned, such enhancements add up.
The truth is, Amazon’s chip group was lauded by Apple in 2024. In a uncommon second of openness for the secretive firm, Apple’s director of AI publicly described the way it used one other of the group’s chips — Graviton, a low-power, ARM-based server CPU and the primary breakout chip this group designed. Apple additionally lauded Inferentia — a chip particularly designed for inference — and gave a nod to Trainium, which was new on the time.
These chips symbolize the traditional Amazon playbook: See what folks wish to purchase, then construct an in-house various that competes on worth.
The catch for chips, traditionally, has been switching prices. Functions written for Nvidia’s chips have to be re-architected to work with others — a time-consuming course of that daunts builders from switching.
However the AWS chip group proudly informed me that Trainium now helps PyTorch, a well-liked open supply framework for constructing AI fashions. That features most of the ones hosted on Hugging Face, an enormous library the place builders share open supply fashions.
The transition, Carroll informed me, requires “principally a one-line change, after which recompile, after which run on Trainium.” In different phrases, Amazon is making an attempt to chip away at Nvidia’s market dominance wherever attainable.
AWS has additionally this month introduced a partnership with Cerebras Systems, integrating that firm’s inference chip on servers operating Trainium for what Amazon guarantees can be superpowered, low-latency AI efficiency.
However Amazon’s ambitions transcend the chips themselves. It additionally designs the server that hosts the chips. In addition to the networking elements, this group has designed “Nitro,” a hardware-software combo that gives virtualization tech (which permits many situations of software program to run individually on the identical server); new state-of-the-art liquid cooling expertise; and the server sleds (pictured under) that host this gear.
All of that’s to manage value and efficiency.
Working 24/7 on the “bring-up”
Amazon’s {custom} chip-designing unit was born when the cloud big bought Israeli chip designer Annapurna Labs in January 2015 for about $350 million. So this group has now had greater than 10 years designing chips for AWS. The unit has retained its Annapurna roots and title — its emblem is all over the place within the workplace.
This chip lab is positioned in a shiny, chrome-windowed constructing in Austin’s upscale “The Area” district, a walkable space crammed with retailers and eating places that’s generally referred to as Austin’s Silicon Valley.
The places of work have your traditional tech company vibe: desks in cubicles, gathering spots, and convention rooms. However tucked away in the back of a excessive flooring within the constructing is the precise lab, with sweeping views of town.
The shelving-filled lab, concerning the dimension of two massive convention rooms, is a loud industrial area because of the followers on the tools. It seems to be like a cross between a highschool store class and a Hollywood set for a high-end lab, besides the engineers are wearing denims, not white lab coats.
Word that this isn’t the place the chips are manufactured, so no white hazmat fits had been mandatory. The Trainium3 is a state-of-the-art 3-nanometer chip, produced by TSMC, arguably the chief in 3-nanometer manufacturing, with different chips produced by Marvell.
However that is the room the place the magic of the “bring-up” happens.
“A silicon bring-up is while you get the chip for the primary time, and it’s like an enormous in a single day celebration. You keep right here, like a lock-in,” King explains. After 18 months of labor, the chip is activated for the primary time to confirm it really works as designed. The group even filmed among the Trainium3 bring-up and posted it on YouTube.
Spoiler alert: It’s by no means problem-free.
For Trainium3, the prototype chip was initially air-cooled, like earlier variations. The present chip is now liquid-cooled, which affords vitality benefits and was fairly an engineering feat.
In the course of the bring-up, the size for a way the chip connected to the air-cooling warmth sink had been off, so the chip couldn’t be activated.
Unfazed, the group “instantly obtained a grinder and simply began grinding off the steel,” King mentioned. As a result of they didn’t need the noise disrupting the bring-up pizza celebration ambiance, they snuck off and did the grinding in a convention room.
Staying up all night time and fixing issues “is what silicon bring-up is all about,” King mentioned.
The lab even has a welding station, the place {hardware} lab engineer and grasp welder Isaac Guevara demonstrated welding tiny built-in circuit elements by a microscope. That is such insanely troublesome work that senior chief Carroll brazenly admitted he couldn’t do it, to the guffaws of Guevara and the remainder of the engineers within the room.
The lab additionally accommodates each custom-made and industrial instruments for testing and analyzing points with chips. Right here’s sign engineer Arvind Srinivasan demonstrating how the lab assessments every tiny part on the chip:
Sleds are the star of the lab
However the star of the lab is a complete row showcasing every technology of the “sleds” the group designed.
Sleds are the trays that home the Trainium AI chips, Graviton CPU chips, and supporting boards and elements. Stack them collectively on a rack with the networking part, additionally custom-designed by this group, and also you get the programs which might be on the coronary heart of Anthropic Claude’s success.
Right here’s the sled that was proven off in the course of the AWS re:invent convention in December:
Confirmed by Anthropic and OpenAI
I anticipated my guides to crow concerning the OpenAI deal in the course of the tour. However they didn’t.
The reticence may have been associated to the aforementioned potential authorized haze which may cling over the deal. However the sense I obtained was that these boots-on-the-ground engineers (who’re at the moment designing the following model, Trainium4) haven’t had a lot likelihood to work with OpenAI but. Their day-to-day work has to this point been centered on Anthropic’s and Amazon’s wants.
At the moment, the most important chunk of Trainium2 chips is deployed in Mission Rainier — one of many world’s largest AI compute clusters — which went dwell in late 2025 with 500,000 chips. It’s utilized by Anthropic.
However there was a wall monitor in the primary workplace displaying a quote about how OpenAI can be utilizing Trainium. The satisfaction was there, if refined.
Along with this lab, the group additionally has its personal non-public information heart for high quality and testing functions. A brief drive away, it doesn’t run buyer workloads, so it’s housed at a co-location facility, not an AWS information heart.
Safety is tight: There are strict protocols to enter the constructing and to entry Amazon’s space inside.
The information heart’s cooling system is so loud that earplugs are obligatory, and the air is thick with the acrid odor of heated steel. It’s not a pleasing place for the common individual to hang around.
At this information heart, there are rows and rows of servers crammed with sleds that combine all of Amazon’s latest {custom} chips: Graviton CPU, liquid-cooled Trainium3, Amazon Nitro, all fortunately computing away. The liquid runs on a closed system, which means it’s reused, which also needs to assist cut back the environmental influence, the engineers mentioned.
Right here’s what a present Trn3 UltraServer seems to be like: A number of sleds are on high and backside, with the Neuron switches within the center. {Hardware} improvement engineer David Martinez-Darrow is seen right here performing upkeep on a sled:
Whereas consideration on the group has at all times been excessive, the scrutiny has actually ratcheted up as of late.
Amazon CEO Andy Jassy retains a detailed eye on this lab, publicly bragging about its merchandise like a proud dad. In December, he mentioned Trainium was already a multibillion-dollar enterprise for AWS and called it one piece of AWS tech he’s most enthusiastic about. He additionally gave the chip a shout-out when asserting the OpenAI settlement.
The group feels the stress, too. Engineers will work 24/7 for 3 to 4 weeks round every bring-up occasion to repair any points so the chips might be mass-produced and put into information facilities.
“It’s crucial that we get as quick as attainable to show that it’s really going to work,” Carroll mentioned. “Up to now, we’ve been doing very well.”
*Disclosure: Amazon offered airfare and coated the price of one night time at a neighborhood lodge. Honoring its Leadership Principle of Frugality, this was a back-of-the-plane center seat and a modest room. TechCrunch picked up the opposite related journey prices like Ubers and baggage charges. (Sure, I checked a bag for an in a single day journey. I’m excessive upkeep that method.)
