TLDR:
1. Cerebras Systems has launched a breakthrough AI wafer-scale chip, WSE-3, with double the performance of its predecessor, the WSE-2.
2. The WSE-3 powers the CS-3 supercomputer, allowing training of AI models with up to 24 trillion parameters.
Key Elements:
Cerebras Systems has introduced its Wafer Scale Engine 3 (WSE-3), a revolutionary AI wafer-scale chip with 4 trillion transistors, 900,000 AI cores, and 44GB of on-chip SRAM. The WSE-3 has a peak performance of 125 FP16 PetaFLOPS and is designed to train some of the industry’s largest AI models. Additionally, the WSE-3 powers the CS-3 supercomputer, which can support 1.5TB, 12TB, or 1.2PB of external memory, streamlining the training process and enhancing developer efficiency.
The CS-3 supercomputer can be configured in clusters of up to 2048 systems, allowing it to fine-tune large AI models and achieve significant performance gains. The latest Cerebras Software Framework offers support for PyTorch 2.0 and dynamic sparsity, leading to up to eight times faster training compared to traditional methods.
Despite doubling its performance, the CS-3 maintains the same power consumption as its predecessor and simplifies the training of large language models requiring less code. Cerebras has already received significant interest in the CS-3, with a substantial backlog of orders across various sectors, including enterprise, government, and international clouds.
A strategic partnership between Cerebras and G42 aims to deliver tens of exaFLOPs of AI compute globally, with the construction of the Condor Galaxy 3, featuring 64 CS-3 systems. This partnership highlights the potential of the CS-3 in healthcare and propels innovation in the AI industry.