Google Introduces PaLM 2 and A3 Virtual Machines for Advanced AI

May 11, during today's Google I/O conference in the early morning, Google spent over an hour discussing its numerous advancements in artificial intelligence, including the new PaLM 2 large language model (LLM) for its generative AI, Bard. This serves as a foundational pillar for adding AI capabilities to Google's product portfolio, such as Google Maps, Google Photos, and Gmail, among others.

In addition to this, Google has also unveiled a high-performance supercomputer called A3 Virtual Machines. It features the 4th generation Intel Xeon Scalable processors, 2TB DDR5-4800 memory, and 8 NVIDIA H100 "Hopper" GPUs, delivering a bandwidth of 3.6 TBps through NVLink 4.0 and NVSwitch.

Google stated that the new A3 supercomputer is "dedicated to training and servicing the most demanding AI models that power today's generative AI and large language model innovations." This supercomputer boasts an impressive AI performance of 26 exaFlops.

According to Google, A3 represents the first production-level deployment of their GPU-to-GPU data interface, enabling data sharing at a speed of 200Gbps without involving the host CPU. This interface, referred to as the Infrastructure Processing Unit (IPU), enhances the available network bandwidth for A3 virtual machines (VMs) by more than 10 times compared to A2 virtual machines.

Ian Buck, Vice President of NVIDIA for High Performance and Large Scale Computing, commented, "Google Cloud's A3 VMs, powered by next-generation NVIDIA H100 GPUs, will accelerate the training and servicing of generative AI applications. We are proud to continue our collaboration with Google Cloud, following the recent launch of G2 instances, and assist global businesses in their transformation through purpose-built AI infrastructure."