Amazon to Push Custom AI Chips to Cut Nvidia Reliance
Amazon is set to unveil its latest AI chip, as the tech giant seeks to capitalize on its multi-billion-dollar semiconductor investments and reduce reliance on market leader Nvidia. Amazon's cloud computing division is heavily investing in custom chips to enhance efficiency across its data centers, ultimately aiming to lower costs for both Amazon and its AWS customers.
The initiative is led by Annapurna Labs, a chip startup based in Austin, Texas, which Amazon acquired for $350 million in early 2015. Annapurna's latest achievement, the "Trainium 2" AI chip, is designed to handle the most complex AI models and is set to be showcased in December. Amazon plans to make the Trainium 2 widely available, building on its previous chip offerings.
Trainium 2 is already being tested by leading companies such as Anthropic (a competitor to OpenAI, backed by a $4 billion investment from Amazon), Databricks, Deutsche Telekom, Ricoh Japan, and Stockmark. Amazon AWS and Annapurna are targeting Nvidia's dominance in the AI chip market, with Nvidia's chips currently powering some of the most advanced AI models globally.
Dave Brown, VP of AWS Compute and Networking Services, shared, “We want to be the best place to run Nvidia chips. At the same time, we believe it's beneficial to offer alternatives.” Amazon claims that its Inferentia chip, designed for generative AI models, has already reduced operational costs by 40%.
Looking ahead, Amazon anticipates capital expenditures of $75 billion in 2024, with the majority directed toward technological infrastructure. CEO Andy Jassy recently stated that spending will likely increase in 2025. This marks a significant rise from 2023, which saw $48.4 billion in total expenditures. Major cloud providers like Microsoft and Google are also heavily investing in AI, with little sign of slowing down.
In addition to being key customers of Nvidia, companies like Amazon, Microsoft, and Meta are also designing their own data center chips to support the anticipated AI growth. Daniel Newman of Futurum Group explains, “Every major cloud provider is rapidly shifting towards more vertical integration and, where possible, standardizing their ‘chip technology’ stack.”
Annapurna's development of the Graviton series, low-power Arm-based processors, has already proven a successful alternative to traditional Intel or AMD servers. G. Dan Hutcheson, an analyst at TechInsights, notes, “AWS’s advantage is that their chips use less power, making their data centers potentially more efficient,” which helps reduce overall costs. He adds that while Nvidia's GPUs are versatile tools, Amazon's chips are optimized for specific tasks, akin to the difference between a compact car and a sports sedan.
Despite Amazon's efforts, Nvidia continues to dominate the AI infrastructure market. Nvidia's AI data center chip sales reached $26.3 billion in Q2 of FY 2024, matching AWS's entire reported revenue for the same period. However, only a small portion of this figure is attributed to AI workloads running on Annapurna’s infrastructure.
As for performance comparisons, Amazon refrains from directly matching its chips against Nvidia's and does not submit its chips for independent benchmarking. Patrick Moorhead, a chip analyst at Moor Insights & Strategy, believes Amazon's claims of a fourfold performance improvement between Trainium 1 and Trainium 2 are credible, although he suggests that offering customers more options may be more important than raw performance metrics.
