A rack containing eight accelerators rated at 700 W each already has 5.6 kW of accelerator power to account for. The host processors, memory, storage, network adapters and power-conversion losses push the electrical requirement higher. Multiply the design across a row of racks and the building's electrical and cooling capacity starts to matter as much as the hardware order.
From a chip rating to a rack budget
A GPU's rated power is not a meter reading for its server. NVIDIA's workload power documentation distinguishes several GPU power concepts and describes different operating profiles. A configured power limit, maximum specification and observed workload consumption are separate values.
A rack budget must account for the compute trays, switches, fans, pumps where applicable and any supporting equipment powered from that rack. Some vendor system specifications already include particular components, so adding them again would double-count the load. Facility distribution equipment and cooling plant introduce further overhead.
Power describes the rate of energy use. A steady 100 kW IT load consumes 2,400 kWh in 24 hours. If measured consumption varies with utilization, the energy estimate should use that profile rather than assuming every rack draws its nameplate maximum around the clock. The same distinction matters when sizing feeds for peak loads versus estimating a monthly electricity bill.
Where the heat goes
Direct-to-chip liquid cooling puts cold plates in contact with heat-producing components. Coolant carries the heat through a loop rather than relying entirely on airflow inside the server. The Open Compute Project's cooling work covers the complete system, including manifolds, tubing, quick disconnects and coolant distribution units.
A coolant distribution unit (CDU) commonly provides the interface between equipment and facility cooling circuits. Its requirements depend on the design. OCP's CDU project addresses integration in existing and new buildings. The hardware supplier's coolant temperature, pressure and flow requirements must match what the facility can deliver.
Moving heat into liquid does not make it disappear. Schneider Electric's cooling architecture paper discusses the piping, distribution and heat-rejection equipment needed beyond the server. Depending on climate and design, a facility may use chillers, dry coolers or other systems. Some components can still need air cooling even when the main processors use cold plates.
For scale, removing 100 kW with water experiencing a 10°C temperature rise would require roughly 2.4 kg/s of water flow, or about 144 litres per minute, using a simplified heat balance and water's approximate specific heat of 4.18 kJ/kg/K. That is not a CDU sizing specification. Real systems must account for coolant composition, temperature limits, pressure losses, redundancy and the equipment manufacturer's requirements.
High-density racks change the facility design
Putting the same IT load in fewer racks can free floor space, but it concentrates power and cooling demand. A room designed for conventional servers may not have distribution circuits or liquid infrastructure suitable for a dense AI installation. Even a rack with sufficient nominal feed capacity needs a redundancy design that can carry the intended load during maintenance or failure.
This also explains why announced accelerator capacity and usable hosted capacity are different. Hardware has to be delivered, installed, connected and commissioned before a customer can use it. Delays can occur at any stage, including the supporting electrical and mechanical work.
PUE does not measure useful model output
The Green Grid's definition of power usage effectiveness uses total data-center energy divided by IT equipment energy. At a PUE of 1.2, an IT load consuming 100 kWh corresponds to 120 kWh of total facility energy over the same measurement period and boundary.
A better PUE reduces the facility overhead associated with a given IT energy consumption. It does not tell us how many useful model responses the IT equipment produces. Two servers in the same facility may have identical cooling overhead but different inference throughput, accuracy or idle consumption.
To compare AI energy efficiency, use a defined workload and report the measurement boundary. A server-level measurement should include the host and accelerators. A facility estimate can add the relevant measured overhead. Inference measurements also need a latency and output-quality requirement, otherwise higher throughput might simply reflect a different task.
What a hosting customer should ask
For a dense GPU allocation, ask which power profile the supplier guarantees, whether that profile can be sustained and what cooling or power maintenance means for the service. A cooling system's peak heat-removal capacity says little about fault detection or serviceability. The rack may have excellent theoretical performance while the facility cannot reliably support it at that level.
This is why accelerator selection is a whole-system question. The useful evidence is measured application performance at the operating conditions the supplier can actually deliver.