As reported by Nvidia’s Q2 financial statement, its second-quarter data center’s revenue was a record US$10.32 billion, up 141% from the previous quarter and up 171% from a year ago. With the increasing demand for high-performance servers in data centers, IDTechEx has seen challenges in data center thermal management and tremendous opportunities in this industry.
In 2022, Nvidia announced its new plan for reducing the energy use of data centers, crunching massive amounts of data, or training AI models: liquid-cooled graphics cards. The company announced its new liquid-cooled version of its A100 compute card at Computex and reported that it consumes 30 percent less power than the air-cooled version. Earlier this year (2023), Nvidia received US$5 million from the US Department of Energy for investigating innovative cooling approaches.
With the increasing thermal design power of chips, traditional air-cooling struggles to meet cooling requirements. Although direct-to-chip cooling (also known as direct liquid cooling (DLC)) has been widely adopted, D2C cooling can only cool a limited number of major components, such as chipsets and GPUs, rather than the entire board. Immersion cooling, on the contrary, offers a high and evenly distributed cooling, but the process of submerging the servers into tanks can be cumbersome and require significant retrofitting of existing rack-based data centers.
Regardless of D2C cooling or immersion cooling, coolant liquid plays a critical role. Depending on if phase change happens, liquid cooling can be split into single-phase and two-phase. While two-phase coolant presents higher cooling performance, they are susceptible to regulatory risks such as PFAS. On the contrary, single-phase coolant utilizes convection to carry the heat away, thereby mitigating the regulatory risks.
Data center liquid coolants can be categorized into oil-based fluids and engineered fluids. Leading companies include Shell, Castrol, ExxonMobil, Fuchs, M&I Materials, Engineered Fluids, 3M, Chemours, and Solvay.
The prices of coolant fluids can vary significantly, and the cost can vary from around US$10/kg to over US$100/kg. When it comes to choosing the right coolant, there are a number of considerations. One of them is the maintenance complexity. Oil or hydrocarbon-based coolants often have impurities, leading to potential corrosion of server boards and the piping. On the contrary, engineered fluids are usually free of this issue, offering reduced maintenance complexity over time. When it comes to upfront costs, an immersion cooling tank can cost around US$4000, and the cost of coolant liquid per tank can be around US$7,000 per tank:
– Size of an immersion tank: Vtank = 1.5m x 0.655m x 1.2m = 1.18m3 = 1180 litres
– Coolant volume = Vtank x 50% = 1180 litres x 50% = 590 liters
– Coolant Cost = 590 litres x US$12/litre ~= US$7000
Despite the relatively high upfront costs, over time, the total cost of ownership of immersion cooling can be beneficial thanks to the amount of energy it can save. IDTechEx believes that by 2033, the yearly revenue of data center liquid cooling hardware will exceed US$900 million, representing significant opportunities. More details can be found in IDTechEx’s report, “Thermal Management for Data Centers 2023-2033“.
