Microsoft will scale its Azure Maia accelerators to multi-gigawatt deployments starting in 2027, calling Maia the primary inference system for its products and services. Maia 200 is already in production serving Microsoft Copilot, while Maia 300 begins production deployments in 2027 with a promised 40% more tokens per megawatt than leading GPU-based systems. The pledge matters because inference cost now shapes pricing and margins for Copilot, Teams, GitHub Copilot and Foundry workloads.
Maia 200 in production, Maia 300 specs and fabric
Maia 200 serves Microsoft Copilot and the MAI team, with GitHub Copilot and Microsoft Foundry to follow by the end of the year. Deployment spans the United States, Europe and Australia, and Microsoft reports up to 1.7 times more tokens per watt for built-in AI in Teams and Copilot than leading GPU infrastructure. Neither announcement names the GPU systems used for comparison, which leaves the baseline open. The rollout sequence shows internal consumption first, with no mention of direct customer provisioning of Maia capacity.
Maia 300 is inference-first for long-context, memory-intensive and decode-heavy work, with no mention of training in either announcement. Each accelerator carries eight HBM4 stacks with 288 GB and 18 TB/s inside a 950-watt package. A rack holds 96 accelerators in 167 kW across 24 four-accelerator trays, reaching 27.6 TB of HBM and 1.7 PB/s aggregate bandwidth. Compute chiplets use TSMC N2P, while memory-controller and I/O functions sit on separate dies, lifting compute FLOPs by 36% and supporting 2.5 times the memory bandwidth and roughly twice the HBM bandwidth per watt of Maia 200.
Networking centers on a custom AI transport built on Ethernet, described as one fabric rather than separate scale-up and scale-out domains. Accelerators initiate communication to any peer in a superpod without involving the host or a discrete scale-out NIC. Two accelerators link directly within a tray, a switched fabric joins all 96 in a rack, and optical second-tier switches connect up to 128 racks into a superpod of 12,288 accelerators. Microsoft names three latency domains, from a direct pair to one hop within the pod and three hops across pods, and expects tensor-parallel traffic to stay in the lowest. The split between in-rack and cross-rack bandwidth is set by job packing, allowing up to 750 GB/s to any peer when only a subset cross racks.
What the efficiency claim means for AI buyers
For companies running Copilot, Teams AI features, GitHub Copilot or Foundry-based applications, the design targets lower cost per token and higher throughput at scale. Microsoft cites up to seven times the rack-scale token throughput and more than 2.5 times the performance per watt of Maia 200 for the new chip. Prefill and decode workers are sized and placed independently, with KV state moving directly to decode workers over the same fabric. A spare tray in one rack can offset a failed tray in another transparently to software kernels, which points to steadier capacity for long-running agent workloads.
Several limits need checking before treating the 40% figure as a buying signal. Commenters questioned the unit itself, with Rick Bullotta calling tokens per watt inconsistent and rewarding verbosity, and Michel Floyd asking for tokens per second per watt or tokens per KWh. Vinay Bansal of UBIQS asked at what context length the gain was measured, noting that sustained multi-step agents carry long contexts where memory binds performance. Microsoft frames the goal as useful tokens per watt and useful tokens per dollar without defining useful, and confidence differs between posts that expect the gain and a post that states it delivers it.
The marker to watch is Microsoft Ignite 2026 in San Francisco next month, where the company says it will share more beyond the engineering post by Rani Borkar and Saurabh Dighe. Confirmation would be a named comparison system, context length, and whether external customers can reserve Maia throughput. Phillip Pham already asked whether gains reach cheaper provisioned throughput reservations or stay inside Microsoft services. Those details will show if the benefit lands in enterprise AI budgets.
