Google Launches TPU v5p: 2.8x Cloud AI Performance

On December 7th, Google unveiled its cutting-edge multimodal large model, Gemini, and introduced the groundbreaking TPU v5p designed specifically for cloud-based AI acceleration. Representing Google's most powerful and cost-effective Cloud Tensor Processing Unit (TPU) to date, this announcement carries significant implications for the field of artificial intelligence.

The TPU v5p Pod is composed of up to 8,960 chips, featuring chip-to-chip connections with the highest bandwidth (4,800 Gbps per chip). This sophisticated architecture ensures rapid data transfer speeds, facilitating optimal performance for AI applications in the cloud.

In terms of AI capabilities, the TPU v5p achieves remarkable benchmarks, delivering 459 teraFLOPS in bfloat16 (16-bit floating-point format) performance or 918 teraOPS in Int8 (8-bit integer) performance. With support for 95GB of high-bandwidth memory and a data transfer speed of 2.76 TB/s, the TPU v5p sets a new standard in AI hardware.

Compared to its predecessor, the TPU v4, the TPU v5p demonstrates a doubling of floating-point operations (FLOPS) and a threefold increase in high-memory bandwidth, underscoring its significant advancements in AI technology.

In the realm of model training, the TPU v5p exhibits a 2.8-fold generational improvement in Large Language Model (LLM) training speed, surpassing even the TPU v5e with an approximate 50% enhancement. Google has further enhanced computational capabilities, with the TPU v5p boasting four times higher scalability in total available FLOPs per Pod compared to the TPU v4.

To summarize, the TPU v5p, in comparison to the TPU v4, achieves:

· A doubling of floating-point operations (459 TFLOPs Bf16 / 918 TOPs INT8)

· A tripling of memory capacity compared to TPU v4 (95 GB HBM)

· A 2.8-fold improvement in LLM training speed

· A 1.9-fold increase in the speed of training for embedded dense models

· A 2.25-fold boost in bandwidth (2765 GB/s vs. 1228 GB/s)

· A doubling of chip-to-chip interconnect bandwidth (4800 Gbps vs. 2400 Gbps)