For three decades, datacenter cooling meant moving air. Computer-room air handlers pushed cold air under raised floors and up through perforated tiles; hot exhaust was contained and returned to be chilled again. The approach scaled gracefully through generations of server hardware because the heat produced per rack stayed within a range that moving air could handle. That era is ending. The arrival of AI-optimized rack designs with power densities several times higher than conventional infrastructure has pushed air cooling past its physical limits, and liquid cooling — long a niche technique reserved for supercomputers and enthusiasts — has become a mainstream requirement.

The Density Problem

The constraint is straightforward physics. Air has a low heat capacity. To remove a given amount of heat with air, you must move a large volume of it, and beyond a certain heat density the required airflow becomes impractical — the fans grow too large, too loud, and too power-hungry, and the temperature differential needed to carry the heat away exceeds what the equipment can tolerate.

A conventional enterprise rack dissipating 5 to 15 kilowatts is comfortably within air’s capability. The dense accelerator racks now being deployed for AI training operate at dramatically higher densities — current top-end rack designs built around tightly integrated accelerator systems can reach the order of a hundred kilowatts or more per rack. At those densities, air cooling is not merely inefficient; it is physically unable to keep the components within their thermal limits. Liquid is the only practical medium.

The reason for the density jump is architectural. AI training performance depends on packing many accelerators close together with high-bandwidth, low-latency interconnects between them. The tighter the integration, the better the performance — and the higher the concentration of heat in a small volume. The same design pressure that makes these systems fast makes them impossible to cool with air.

Two Approaches: Direct-to-Chip and Immersion

Two liquid cooling techniques have moved into mainstream deployment, each with distinct trade-offs.

Direct-to-chip (cold plate) cooling routes a liquid coolant through cold plates mounted directly on the hottest components — the accelerators and CPUs. The coolant absorbs heat at the source and carries it to a coolant distribution unit, which transfers it to a facility water loop. This is the more incremental approach: the rest of the server, and lower-power components, may still rely on some air cooling, so direct-to-chip is often described as hybrid. It integrates relatively well with existing datacenter designs because it reuses the concept of a facility water loop that many large datacenters already have for chillers.

Direct-to-chip has become the default for the densest AI deployments precisely because it targets the hottest components without requiring a wholesale reinvention of the server or the datacenter. The plumbing — manifolds, quick-disconnect couplings, leak detection — adds operational complexity, but the approach is well understood and supported across the major hardware vendors.

Immersion cooling submerges entire servers in a bath of dielectric (electrically non-conductive) fluid. In single-phase immersion, the fluid is pumped through a heat exchanger; in two-phase immersion, the fluid boils on contact with hot components and condenses on a coil above, using the latent heat of the phase change to carry away enormous heat density.

Immersion offers the highest cooling capacity and can eliminate server fans entirely, but it demands a fundamentally different facility design — tanks instead of racks, fluid handling, and servers engineered or modified for submersion. It also raises practical questions about serviceability and, for two-phase systems, about the fluids themselves. Immersion has strong advocates and growing deployment, but direct-to-chip has captured the larger share of mainstream AI buildouts because it disrupts existing operations less.

Why “Mainstream” Is the Right Word in 2025

What distinguishes the current moment from earlier liquid-cooling cycles is that the decision has been taken out of the operator’s hands. In the past, liquid cooling was a choice an operator could make for efficiency or density reasons. With the latest accelerator-rack designs, it is a vendor requirement: the hardware ships expecting liquid cooling, and there is no air-cooled configuration at the top of the range. An operator that wants to deploy the densest AI systems must provide liquid cooling to the rack.

This has rippled through the industry. Colocation providers are retrofitting facilities and designing new ones around liquid-ready halls. The supply chain for cold plates, coolant distribution units, manifolds, and dielectric fluids has expanded rapidly. Standards bodies and industry consortia are working to bring consistency to interfaces — coupling types, fluid specifications, and facility water-loop parameters — that were previously bespoke. The transition has the character of an industry-wide shift rather than a set of isolated experiments.

Efficiency and Sustainability Dimensions

Liquid cooling is not only a density enabler; it is also more energy-efficient, which matters as datacenter power becomes a scarce and scrutinized resource. Moving heat with liquid requires far less energy than moving the equivalent heat with air, which improves the facility’s power usage effectiveness — the ratio of total facility energy to energy delivered to the computing equipment. Energy that would have gone to fans and chillers can instead go to computation.

There is a further benefit: liquid cooling captures heat at a higher and more useful temperature than air systems, which makes heat reuse more practical. Warm water leaving a direct-to-chip loop can, in suitable settings, be fed into district heating or other thermal uses rather than simply rejected to the atmosphere. Several operators in colder climates have begun designing for heat recovery as a deliberate part of the cooling architecture.

These efficiency gains connect liquid cooling to the broader resource-constraint story facing the industry. When the binding limit on capacity is electrical power, every watt diverted from cooling to computation is capacity recovered, and the thermal efficiency of the facility becomes a competitive and capacity variable rather than an afterthought.

Operational Realities

The shift is not free of friction. Introducing liquid into the rack changes the operational model in concrete ways:

  • New failure modes. Leaks, coupling failures, pump failures, and fluid degradation are risks that air-cooled operations never had to manage. Leak detection, redundant pumps, and well-rehearsed response procedures are now part of datacenter operations.
  • New skills. Operations teams accustomed to airflow management must learn fluid dynamics, coolant chemistry, and the maintenance of distribution units. The skills gap is real, particularly for operators retrofitting existing facilities.
  • Serviceability. Swapping a component in a liquid-cooled or immersed system is a different procedure than hot-swapping an air-cooled server. Maintenance workflows have to be redesigned.
  • Standardization gaps. Because the field is maturing quickly, interfaces are not yet fully standardized, which can create vendor coupling at the facility level — a familiar lock-in concern transposed into the physical plant.

The Outlook

Liquid cooling’s move into the mainstream is a durable structural change, not a passing trend tied to one hardware generation. The trajectory of accelerator design points toward ever-higher density and power, and there is no indication that the curve will bend back toward air-coolable levels. Facilities being designed now are being designed liquid-first, and the operational practices, supply chains, and skills are organizing around that assumption.

For infrastructure planners, the practical implication is that liquid cooling readiness is becoming a baseline requirement for any facility intended to host high-density AI workloads. The question is no longer whether to adopt liquid cooling but which approach fits a given facility’s constraints, and how to build the operational competence to run it reliably. The thirty-year reign of moving air is giving way, and the change is happening across the industry at once.

Further Reading

Back to Blog