Microsoft unveils the Maia 300 inference chip and says Maia will be the primary inference system for its products, at multi-gigawatt scale from 2027

Microsoft announced Maia 300, the third generation of its own AI chip, on 12 October. Scott Guthrie, executive vice president of Cloud + AI, wrote that Microsoft will scale its Azure Maia family “to multi-gigawatt deployments starting in 2027”, making Maia the primary inference system for its products and services. Inference is the work of running AI models once they are trained. “Maia 300 will begin production deployments in 2027,” the post says.

The chip is designed for inference first. Its compute chiplets, smaller dies combined into one package, are made on TSMC's N2P process. It uses eight stacks of HBM4 memory with 18 TB/s of peak memory bandwidth per chip, within 950 watts. A full rack holds 96 accelerators, and Microsoft's architecture post puts the rack's power at 167 kW.

Microsoft claims Maia 300 will deliver 40% more tokens per megawatt than “leading GPU-based systems”. It does not say which systems, so the claim cannot be checked. It also claims a full 96-accelerator rack gives up to seven times the rack-scale token throughput of Maia 200. The earlier chip already serves Copilot, with GitHub Copilot and Microsoft Foundry to follow by the end of the year, the post says.

• Confirmed “Maia 300 will begin production deployments in 2027.” Microsoft Official Blog
• Claimed Microsoft claims 40% more tokens per megawatt than “leading GPU-based systems”, without naming them, and up to 7x Maia 200's rack-scale throughput. Microsoft Official Blog
• Confirmed A full Maia 300 rack holds 96 accelerators and draws 167 kW, Microsoft's architecture post says. Azure Blog