Intel Gaudi2: Accelerate AI Training & Inference

Intel recently unveiled its latest AI product strategy and introduced the Habana® Gaudi®2, the second generation of its Gaudi deep learning accelerator, at a launch event in Beijing.

Sandra Rivera, Executive Vice President and General Manager of Intel's Data Center and AI Business Group, highlighted Intel's commitment to advancing AI technology by offering customers a diverse range of hardware options and supporting an open software environment. Through products like the Intel Xeon Scalable processor and Gaudi2 deep learning accelerator, Intel aims to democratize AI and empower customers to deploy critical AI solutions at both the cloud and intelligent edge, thereby shaping the future of AI in China.

The Gaudi2 deep learning accelerator, along with its accompanying Gaudi2 Mezzanine Card HL-225B, builds upon the success of the first-generation Gaudi architecture. It introduces significant performance and energy efficiency improvements to effectively run high-performance large language models. Key features of the accelerator include 24 programmable Tensor Processor Cores (TPCs), 21 100Gbps Ethernet interfaces, 96GB of HBM2E memory, 2.4TB/s total memory bandwidth, 48MB on-chip SRAM, and an integrated multimedia processing engine.

The Gaudi2 accelerator has been rigorously tested and certified through the MLCommons® MLPerf® benchmark tests. It has demonstrated outstanding training results on various models, including GPT-3, ResNet-50 for computer vision (using 8 accelerators), Unet3D (using 8 accelerators), and BERT for natural language processing (using 8 and 64 accelerators). When compared to competing products, Gaudi2 offers superior performance and cost-effectiveness, empowering users to optimize operational efficiency while reducing overall operating costs.

Furthermore, Gaudi2 excels in delivering exceptional inference performance for large-scale multimodal and language models. It has consistently outperformed industry standards in recent evaluations, particularly in tasks like running Stable Diffusion, a state-of-the-art generative AI model for text-to-image generation, and handling models with billions or even trillions of parameters.

Intel's launch of the Gaudi2 deep learning accelerator marks a significant milestone in the development of AI technology. By providing the electronic components industry with powerful and cost-efficient solutions, Intel is poised to play a crucial role in shaping the future of AI and enabling widespread adoption of AI across various industries.