AMD Plans to Ship 400K MI300 AI GPUs in 2024

Industry insiders have disclosed that AMD is on the verge of releasing its highly anticipated next-generation Artificial Intelligence (AI) GPU series, Instinct MI300, with an estimated shipment of 300,000 to 400,000 units projected for 2024.

AMD is set to unveil the Instinct MI300 data center GPU accelerator during a live streaming event scheduled for December 6, 2023. The launch aims to underscore the company's escalating collaboration with AI hardware and software partners, positioning itself as a key player in this evolving landscape.

Lisa Su, AMD's Chairman and CEO, will engage in discussions with company executives, AI ecosystem partners, and customers during the event. The focus will be on how AMD's products and software are reshaping the realms of AI, adaptive computing, and high-performance computing (HPC).

Widely perceived as a groundbreaking product challenging NVIDIA's AI GPU dominance, the Instinct MI300 is expected to garner significant attention. Microsoft, a major client of AMD, is anticipated to showcase support for the Instinct MI300 during the event.

Despite tightened export restrictions on semiconductor chips and devices to China, industry experts highlight the sustained growth in demand for AI GPUs. The MI300 series is poised to ship between 300,000 to 400,000 units in 2024, with Microsoft and Google positioned as major clients for AMD's latest AI GPU accelerator.

Insiders suggest that capacity constraints at TSMC CoWoS and NVIDIA's reservation of 40% of TSMC's CoWoS capacity have impacted potential shipment targets for AMD.

In a recent event hosted by Microsoft in mid-November, AMD and Microsoft demonstrated the capabilities of AMD products, including the upcoming Instinct MI300X accelerator, AMD EPYC CPUs, and AI-enabled AMD Ryzen CPUs. These products are expected to introduce new services and computing capabilities in areas such as cloud and generative AI, confidential computing, cloud computing, and enhanced personal computers (PCs).

AMD has revealed that the MI300X, based on the next-gen AMD CDNA 3 accelerator architecture, supports up to 192GB of HBM3 memory, catering to the computational and memory needs of large language model training and inference in generative AI workloads. With its expansive memory, the AMD Instinct MI300X allows customers to install large language models, such as Falcon-40 (a 400 billion parameter model), on a single MI300X accelerator.

Additionally, AMD is actively developing the ROCm software ecosystem for data center accelerators, signaling its readiness to collaborate with industry leaders and integrate an open AI software ecosystem.

Microsoft has disclosed the creation of virtual machines (VMs) with 1.5TB HBM (High Bandwidth Memory), leveraging the powerful capabilities of the AMD flagship MI300X GPU. These Azure VMs equipped with the MI300X GPU offer customers a broader range of AI-optimized VM choices.

AMD has noted that its fourth-generation EPYC processors are now deployed to run Microsoft's new generation of general-purpose, memory-intensive, and computation-optimized VMs.

Insiders emphasize that the AMD MI300 series leverages TSMC's advanced packaging services, primarily supporting CoWoS SoIC stacking technology. AMD holds the position of being TSMC's largest customer for proprietary SoIC services, with Apple and NVIDIA expected to adopt this technology post-2025.

TSMC is actively expanding CoWoS capacity at its Longtan and Taichung factories, with the Zhunan AP6 factory supporting CoWoS packaging. TSMC is also outsourcing the oS process for CoWoS to OSAT, albeit with lower profit margins. With the rising demand for AI chips, TSMC aims to achieve a monthly capacity of over 30,000 wafers by the end of 2024.

Regarding the more expensive SoIC process, TSMC's monthly capacity currently falls below 2,000 wafers. The introduction of AMD's new chips is expected to propel SoIC development, with TSMC's capacity projected to exceed 2,000 wafers per month in 2024, rising to over 7,000 wafers per month by 2027.

TSMC President Wei Zhejia has highlighted that the company's advanced packaging capacity will triple in 2022 compared to 2018. By 2026, SoIC capacity is expected to expand to over 20 times the level at the start of SoIC commercialization in 2022.

AMD has already secured orders for the MI300 series from numerous clients, prompting an upward adjustment in its fourth-quarter AI GPU sales expectations from $300 million to $400 million. AMD anticipates that its AI GPU sales in 2024 will surpass $2 billion.

Industry insiders suggest that improvements in the AI chip supply chain sales are anticipated to commence from the first quarter of 2024 onward.