DATA CENTER
LIQUID COOLING

What is Liquid Cooling? Is it Just for the Data Center? Why is it So Popular?

According to a 2020 survey conducted by Information Technology Intelligence Consulting, a single hour of IT system downtime could cost an organization $300,000 or more – much more. The fear of system failure is motivating many IT Managers and data center Facility Managers to search for better heat-removal solutions than the traditional air-based method.

Today, as processor capacities and densities increase at the board and chassis level, many end users are turning to liquid cooling to support the higher thermal densities associated with these new systems. The U.S. Department of Energy (DoE) touts liquid cooling as having the benefits of higher compute densities and higher efficiency, saying, “As heat densities rise, liquid solutions become more attractive.”

In a 2019 survey by 451 Research, 45% of companies surveyed said they expect to see an average density of 11kW per rack or higher at some point in the near future. This represents a dramatic increase since 2014, when only a small percentage of data centers reported densities beyond 10kW. According to the DoE, traditional air-cooled systems only allow for rack power densities of 1kW–5kW, with a well designed system maybe 10kW–12kW per footprint. The various liquid cooling solutions can support rack power densities in the 5kW–80kW range, with some even pushing the 100kW threshold.

Finding better heat removal solutions is not just about adequately protecting the equipment; with as much as 40% of all energy used in some data centers going directly to cooling, greater efficiency also leads to dramatically reduced operational costs.

Unlike air-based methods, where massive amounts of cold air are mixed with the heat rejected from equipment to achieve the appropriate room temperature, liquid cooling is targeted to the sources of heat, reducing wasted energy. In fact, liquid systems require 80% less air flow, significantly reducing fan energy. Targeting makes liquid cooling ideal for preventing data center hot spots, too; the cooling system can be incorporated into a single enclosure (cabinet) and set to cool as needed.

You can learn more about the features and benefits of liquid cooling by reading these helpful resources:

• Immersion Cooling: A Great Idea, Just Probably Not For You
• The Growth In Edge Data Center Computing
• Is Your Server Rack Cooling Infrastructure Capable Of Handling More Equipment?
• Your Top 4 Data Center Challenges, Solved
• Data Center Cooling: How To Deploy High-Density Solutions Without Disrupting Infrastructure

Benefits of Liquid Cooling

There are a number of benefits that make liquid cooling the best choice for higher density installations – and the only solution for high density deployments at the Edge. Organizations of all sizes – from those with a single rack to enterprise data centers – are turning to these solutions to protect their IT investments because this method:

• Enables higher rack densities, with some options offering variable capacity that follows heat load variations from 15kW–75kW (or greater) per footprint
• Reduces overall floor space required to support IT installations, especially in space-constrained facilities
• Upgrade legacy spaces to support higher density installations in spaces with limited heat removal capacity or available floor space. These spaces can be repurposed to support new deployments with minimal infrastructure buildout
• Reduce or eliminate the need for room based climate control (CRAC/CRAH), with additional floor space savings and improved energy efficiencies realized
• Eliminate risk of IT failure due to leakages with direct-on-chip, waterless, evaporative liquid cooling leveraging a non-conductive refrigerant
• Allows targeting of hot spots and higher density enclosures more efficiently
• Delivers significant cost savings. Improved energy efficiencies result from:
  
  • Reduction in fan power consumption
  • Higher server inlet temperatures
  • Elevated chill water supply temperatures
  • Ability to use other facility water sources – e.g., return warm water
  • Reduction or elimination of air-cooled components, e.g., CRAC, CRAH
• Allows for precise climate control. Set point temperature is maintained even as heat loads vary; proper humidity and air flow are maintained
• Improves flexibility. Liquid cooling can support a single enclosure or multiple; many enclosures allow for different cooling methods within the same space
• Allows easy scalability. Adding additional cooling capacity is as simple as adding another enclosure to the row
• Can be monitored in real time. Enclosures offer ongoing monitoring and remote notification of climate and security/fire/smoke issues

Learn more about the features and benefits of each design by reviewing these related resources:

• 2020 Trends In Data Center Enclosures
• Enclosure Cooling Solutions: Thinking About All That Hot Air?
• Data Center Cooling: How To Deploy High-Density Solutions Without Disrupting Infrastructure
• The TCO Of Data Center Liquid Cooling

Liquid Cooling or Air Cooling – Which is Right for My Situation?

Air-based systems have long been the go-to for cooling IT equipment in both small deployments and large data centers. Recently though, liquid cooling’s superior efficiency has led IT professionals to switch to liquid, especially supporting next generation high performance compute applications.

Liquid Cooling vs. Air Cooling in Small Data Closets/Rooms

Pressed to find some unused space to house its IT racks, some organizations believe they can achieve proper cooling simply using the building’s AC system. There are several downsides of this approach, three of the most obvious are these:  

• Cold air paths to the IT appliances could be long, making it difficult to maintain the right temperature (and the temperature needed will likely be lower than that at which people are comfortable)
• Most buildings reduce operating costs by moderating the temperature overnight and on weekends, a very real risk to optimum equipment performance 
• Adding equipment only increases the demand for adequate cooling, likely making it impossible to scale

As discussed below, air-based cooling for larger environments is not much different in terms of drawbacks.

Liquid Cooling vs. Data Center Air-Based Systems

This type of cooling uses chilled water to cool the air in the IT space by computer room air handlers (CRAHs). Chilled water flows through a cooling coil inside the unit, which then uses fans to draw air from outside the facility. Computer room air conditioners (CRACs) work on the same principle but are powered by a compressor that draws air across a refrigerant-filled cooling unit. 

Neither of these methods is energy-efficient but have been popular because the equipment itself is relatively inexpensive. And are best suited for low density (<10kW per footprint) applications.

Either system supports traditional/legacy raised floor cooling: cold air from a CRAH or CRAC is forced into the plenum of the raised floor. Perforated tiles direct the cold air to enter the space in front of server intakes. The air passes through the server and, now heated, is returned to the CRAC/CRAH to be cooled again. And with a change in airflow paths, can be deployed in non-raised/slab floor spaces as well.

Some of the fundamental problems with traditional air-based systems are:

• High energy costs
• Increased maintenance and potential mechanical failure
• Significant space requirements
• These systems introduce moisture into the data center – a real issue for hardware performance

Until recently, data centers had no other options for heat removal. But with developments in liquid cooling, many data centers are beginning to try new methods for solving their ongoing – and growing – heat problems.

Chilled water systems use liquids to absorb and carry heat from the hardware, but they do it using air, while using liquid to take that heat away – a much more efficient, cost-effective solution. According to the Center of Expertise for Energy Efficiency in Data Centers, liquid cooling helps reduce energy usage because liquids have a much larger capacity for heat than does air. Air cooling also requires a great deal of power and introduces both pollutants and condensation into the data center; liquid cooling is cleaner, more targeted and scalable.

Read more about today’s most efficient cooling systems, and to understand the long-term value of liquid.

• 4 Things To Consider As You Evaluate Liquid Cooling
• Data Closet Basics: The Hidden Costs Of Traditional Climate Control Units
• Enclosure Cooling: Why Building AC Isn’t The Best Way To Cool IT Equipment

Closed Loop & Open Loop Liquid Cooling

Open loop and closed loop are two possible configurations for liquid cooling, and which is right for the data center depends on thermal loads and other factors:

Open loop configuration

In an open loop system, thermal loads are removed from the rack via an air-to-water heat exchanger. Cooling units are placed between server enclosures in the row to deliver cool air to the equipment. Aligned to the cold aisle/hot aisle orientation, cold and hot airflow paths in an open loop system are not contained inside the enclosures, going instead out into the IT space: heated exhaust air from the IT appliances is drawn in from the room or from the hot aisle at the rear of the unit, cooled and then directed to the room or to the cold aisle and the intakes of the components.

An in-line open loop system can support multiple racks with a wide range of IT equipment loads. When aligned with a cold aisle/hot aisle orientation, these in-row solutions reduce the mixing of hot and cold air, improving the overall efficiency of the system.

Closed loop configuration

In the closed loop installation, hot air expelled by the equipment in the rear of the footprint is drawn across the heat exchangers to be cooled; the cooled air is then sent out the front of the unit and back to the servers. No air – cold or hot – is rejected into the surrounding space. A closed loop system becomes its own microclimate, as it cools only the environment within the rack and is not impacted by any ambient air (nor does it affect that air). Because hot and cold air never mix, heat removal efficiencies are significantly improved. A further benefit – airflow paths from hot to cold and back are greatly reduced, requiring even less fan energy.

Closed loop cooling systems are often used in space-constrained locations, in high density areas within large data centers to eliminate hot spots or to support traditional cooling systems in high density environments. Additionally, there is no need for an aisle containment, cold or hot aisles or, in a well-designed space, even room-based climate control systems. ALL thermal management is maintained by the closed loop systems.

To learn more about these and other liquid cooling configurations (and about terms like in-row, in-rack, close-coupled and others you might want to better understand), read these resources:

• IT Enclosure Climate Control: Closed Loop Cooling For Rack And Row Solutions
• Data Center Cooling Trends: Room, Row And Rack Cooling
  
  

Edge Systems

Much has been written lately about The Edge (and Edge computing), and for good reason: streaming services, self-driving cars, artificial intelligence, smart factories and dozens of other technologies are requiring more – and faster – data, and one of the best ways to achieve that is to reduce data latency by bringing computing closer to where it’s generated and used. Yet, bringing IT equipment right to the heart of things comes with several challenges:

• Harsh and uncontrolled environments. The air in many of these locations, such as factory floors, is laden with a combination of dust, debris and moisture, all of which can affect equipment performance and require not just reliable cooling but physical protection from these and other contaminants
• Ability to easily scale. As densities increase, so too must cooling. IT Managers relying only on the facility’s AC to control the climate in the server room know they can’t prevent equipment overheating using this method
• Lack of staff. Often, Edge locations don’t have the trained staff on hand to monitor and manage the equipment to prevent failure

Pre-engineered modular enclosure systems solve these and other challenges related to on-premise Edge computing.

• With enclosures featuring liquid cooling, heat removal is aligned with the specific densities within the rack. This helps to make the entire deployment future proof, particularly when those enclosures feature closed loop cooling, which often requires no changes to the existing infrastructure
• The flexibility of modular enclosures also makes it simple to swap out equipment when needed and enables users to place equipment of different sizes (and often densities) within the same rack
• Enclosure cooling also dramatically reduces the need for professional staff, with simple network interfaces that connect to devices to provide active, real-time monitoring of critical parameters to help prevent system failure
• No water - no problem. Refrigerant-based systems provide a complete heat removal circuit
  
  

Read more about Edge – what it is, where it’s headed and how it’s best deployed – by exploring some of these resources:

• Your Top 4 Edge Challenges, Solved
• 3 Biggest Mistakes Companies Make When Moving To The Edge
• 4 Key Decisions To Make When Designing An Edge System
• Edge Computing 101
• Edge Data Centers: Best Practices For IT Equipment In Uncontrolled Environments
• How To Choose The Right Enclosure For Edge System Applications