Intel Unveils Gaudi3 AI Chip to Rival Nvidia's H100

On April 9th, Intel unveiled its latest advancements in AI chip technology during the Vision event, showcasing a new iteration targeted at challenging Nvidia's dominance in the AI semiconductor landscape.

The technology sector is actively seeking alternative sources of scarce AI chips. Intel's announcement highlighted its new Gaudi 3 chip, boasting a 50% faster training speed for specific large language models compared to Nvidia's previous-generation H100 processor. Additionally, it showcased superior performance in computing generative AI responses, outpacing certain Intel H100 chip models in speed.

Intel and Advanced Micro Devices (AMD) have been diligently working to develop a range of innovative chips and accompanying software to meet the growing demand for AI applications, potentially providing viable alternatives to Nvidia. By 2023, Nvidia held approximately 83% of the data center chip market, with the remaining 17% primarily held by Google's custom Tensor Processing Units (TPUs), though not directly available for sale.

Fabricated using TSMC's 5nm process, the Gaudi 3 chip introduces a novel Chiplet design with two compute clusters, departing from the single-cluster approach of its predecessor, Gaudi 2. It features 8 matrix math engines, 64 tensor cores, 96MB SRAM, and 128 GB HBM2e memory, along with 16 PCIe 5.0 channels and 24 200GbE links. With a bandwidth of 3.7 TBps, the Gaudi 3 chip demonstrates significant computational capabilities.

While representing Intel's latest innovation in AI acceleration, the Gaudi 3 chip faces competition from Nvidia's latest B200 and AMD's MI300X series. However, it offers compelling advantages, including a 40% faster training speed for popular Large Language Models (LLMs) and a 50% improvement in inference efficiency for large models like H100, according to Intel's official data.

In specific comparisons, Intel's Gaudi 3 showcases superior performance metrics, achieving up to 1.7x faster training times in comparison to H100 for LLAMA2 with 70 billion parameters, and a 1.5x advantage for LLAMA2 with 130 billion parameters. It also holds a 1.4x advantage over H100 for GPT-3 with 175 billion parameters.

Currently, Nvidia's B200 and AMD's MI300X are equipped with 192GB HBM3e/HBM3, while Nvidia's previous-generation H200 also comes with 141GB HBM3. Clearly, Gaudi 3 lags in this aspect, offering slightly more than H100 but still featuring older HBM2e, resulting in a memory bandwidth of only 3.7 TBps, significantly lower than Nvidia's H200's 4.8 GBps and AMD's MI300X's 5.3 TBps.

The Gaudi 3 chip is set to be available to server manufacturers like Supermicro (SMCI.O) and Hewlett-Packard Enterprise in the second quarter of this year.