The race for artificial intelligence dominance has reached a fever pitch, and the latest data from Amazon Web Services (AWS) suggests that demand for AI compute capacity is now outpacing the industry’s ability to build it. In a detailed letter to shareholders accompanying the company’s 2025 annual report, Amazon CEO Andy Jassy described the AWS chip business as being “on fire,” highlighting a market where enterprises are no longer just shopping for compute—they are attempting to lock up entire years of capacity to prevent competitors from doing the same.
This surge in demand is creating a volatile landscape for cloud infrastructure. AWS reported that it added 3.9GW of modern power capacity in 2025 and aims to double its total power capacity by the end of 2027, yet the company continues to face capacity constraints that yield unserved demand. The intensity of this competition is best illustrated by the revelation that two large customers recently requested to buy out all available 2026 instance capacity for Graviton, AWS’s custom CPU chip. Whereas AWS declined these requests to ensure availability for other customers, the move signals a shift toward “strategic dependency,” where compute power is viewed as a critical competitive moat.
At the heart of this growth is a strategic pivot toward custom silicon. While AWS maintains a strong partnership with Nvidia, the company is aggressively scaling its own Trainium and Graviton lines to offer better price-performance ratios. This vertical integration—controlling everything from the power and data center to the custom silicon and the inference engine—is designed to reduce reliance on external chip providers and lower the cost of running massive AI workloads.
The implications extend beyond simple supply chain logistics. As enterprises migrate toward more cost-effective AI infrastructure, the battle for “token economics” becomes central. By optimizing the hardware specifically for the training and inference of large language models (LLMs) and multimodal models, AWS is attempting to replicate the success Graviton had against x86 architectures, now applying that same disruptive pressure to the GPU market.
The Rise of Custom Silicon: Trainium vs. Nvidia
The shift in the processor landscape is driven by a growing corporate appetite for better price-performance. According to CEO Andy Jassy, the second generation of custom AI silicon, Trainium2, which launched in late 2024, offers roughly 30% better price-performance than comparable GPUs and is already largely sold out. This efficiency is now the primary engine behind Amazon Bedrock, which runs most of its inference on these next-generation accelerators.
The momentum is accelerating with the rollout of Trainium3. This latest iteration, which has recently begun shipping, is reported to be 30% to 40% more price-performant than Trainium2 and is already nearly fully-subscribed. The demand is so high that a significant portion of Trainium4 capacity—which is still approximately 18 months away from broad availability—has already been reserved. This trajectory has led Jassy to suggest that AWS may eventually sell racks of these chips to third parties.
This strategy is not necessarily about eliminating Nvidia, but rather about reducing dependence on the chip leader in areas where AWS can win on economics. By integrating these chips with Bedrock and using AWS-designed interconnects, the company provides a holistic package that appeals to hyperscalers and cost-sensitive enterprises. This represents particularly evident in the training and inference of models in the hundreds of billions to trillion-plus parameter range.
However, the transition is not without friction. While marquee names like Anthropic and Uber are testing these efficiency claims, other customers, such as Cohere and Stability AI, have continued to prefer Nvidia’s mature tooling framework and superior chip designs, citing issues with AWS service and availability. Developers moving to Trainium must retool workloads away from Nvidia’s CUDA ecosystem and validate model accuracy parity on AWS frameworks according to industry reports.
Scaling Interconnects and the Trainium4 Roadmap
To support the massive scale required for the next generation of AI, AWS is evolving its networking capabilities. A key part of this evolution is the integration of NVIDIA NVLink Fusion, a collaboration announced at AWS re:Invent to enable faster deployment of custom AI infrastructure via the NVIDIA Developer blog. This high-bandwidth networking solution allows for the connection of up to 72 custom ASICs with NVIDIA’s sixth-generation NVLink Switch, which is critical for the upcoming deployment of Trainium4 AI chips and Graviton CPUs.
The scale of these systems is becoming astronomical. For context, the Trainium3 UltraServer—powered by a 3-nanometer chip—offers a system that is more than 4x faster with 4x more memory than the previous generation as reported by TechCrunch. These UltraServers can be linked together to provide a single application with up to 1 million Trainium3 chips, a ten-fold increase over the prior generation. These systems are 40% more energy efficient, addressing the critical challenge of data center power consumption.
Mantle: The New Backbone of AI Inference
Hardware is only half the battle. the software layer must evolve to keep pace. Amazon Bedrock, which scaled faster than expected, eventually required a complete overhaul of its inference engine. This led to the creation of Mantle, a new engine developed in just 76 days by a small team of six engineers utilizing AWS’s agentic coding service, Kiro.
Mantle has since become the backbone of Bedrock. The impact of this rebuild was immediate: Jassy claimed that Bedrock processed more tokens in Q1 2026 than it had in all prior years combined. Beyond raw volume, Mantle introduced critical features such as asynchronous inference, stateful conversation management, and higher default quotas.
The development of Mantle serves as a case study in “productivity compression.” By using agentic tools, a tiny team achieved what would have traditionally required a much larger engineering force. This shift in development speed suggests that AI-assisted coding is moving from a theoretical benefit to a production reality, fundamentally changing how cloud providers approach project timelines and “build-vs-buy” decisions.
Strategic Partnerships and Point-and-Click Simplicity
To further enhance performance, AWS has partnered with Cerebras. This partnership leverages a complementary hardware strategy: Trainium is optimized for “prefill,” while the Cerebras CS-3 is optimized for “decode.” Together, they aim to deliver peak inference performance without requiring user intervention, providing the “point-and-click” simplicity that enterprise users demand.
This approach targets the fastest-growing and most cost-sensitive workload in enterprise AI: inference. By combining custom silicon, high-speed interconnects like NVLink Fusion, and a streamlined inference engine like Mantle, AWS is positioning itself to capture the market of enterprises that are moving from AI experimentation to full-scale production deployment.
Key Takeaways: The State of AWS AI Infrastructure
- Extreme Demand: Customers are attempting to buy out entire years of Graviton capacity to secure a strategic advantage over competitors.
- Custom Silicon Momentum: Trainium3 is nearly fully-subscribed, with significant reservations already placed for the forthcoming Trainium4.
- Efficiency Gains: Trainium2 offers roughly 30% better price-performance than comparable GPUs, while Trainium3 improves upon that by another 30% to 40%.
- Infrastructure Scaling: AWS is doubling its total power capacity by the end of 2027 to combat ongoing capacity constraints.
- Software Innovation: The Mantle engine has drastically increased token throughput, processing more in Q1 2026 than in all previous years combined.
As the industry moves toward 2027, the primary challenge for AWS will be balancing the needs of massive “anchor” customers with the demand to maintain a broad, accessible cloud ecosystem. The risk is that constrained customers may hedge their bets by moving workloads toward Microsoft Azure or Google Cloud Platform (GCP).
The next major milestone for the AWS AI roadmap will be the broad availability of Trainium4, expected in approximately 18 months. Until then, the company will continue to experiment with disproportionate investments in the areas of the AI stack where it can most effectively challenge the current GPU hegemony.
Do you think custom silicon will eventually replace general-purpose GPUs for most enterprise AI tasks? Share your thoughts in the comments below.