Rack level transient CFD modeling of data center

Yogesh Fulpagare (Department of Mechanical Engineering, Indian Institute of Technology Gandhinagar, Palaj, Gujarat, India)
Yogendra Joshi (Woodruff School of Mechanical Engineering, Georgia Institute of Technology, Atlanta, Georgia, USA)
Atul Bhargav (Department of Mechanical Engineering, Indian Institute of Technology Gandhinagar, Palaj, Gujarat, India)

International Journal of Numerical Methods for Heat & Fluid Flow

ISSN: 0961-5539

Publication date: 5 February 2018

Abstract

Purpose

The paper aims to capture the rack-level thermal dynamics in data center. It proposes the rack-level response experiments as well as transient Computational Fluid Dynamics (CFD) analysis to characterize the local thermal environment of the system.

Design/methodology/approach

A single sever simulator rack and its two neighboring racks with its cold and hot aisle containment have been modeled with known cold air supply temperature and flow rate for transient CFD analysis. The heat load was kept constant initially and varied case-to-case basis, which includes capturing the rack-level response with respect to changes in input. However, the response experiments on simulator rack were performed for 14 h by variation of server heat loads as step and ramp input.

Findings

The paper provides the detailed transient CFD analysis of data center racks. The local cold air flow rates and temperature at the vicinity of the racks showed significant effect due to changes in input. It was concluded that the rack-level dynamics impacts the thermal environment of data center and hence cannot be ignored.

Research limitations/implications

The high computing devices and faster internet demands have led to major thermal management concerns for data center operators. To tackle this issue, capturing the system thermal dynamics is imperative. However, the system-level CFD analysis is computationally expensive. Therefore, this paper deals with the rack-level transient CFD study using commercial tool STAR CCM+.

Practical implications

This paper includes the modeling of the servers as a porous media as well as the multigrid method to enhance the computational speed. The successful implementation of this approach validated through experiments. This would help to establish a base for research in any type of data center.

Originality/value

This paper provides the porous media approach to model servers and multigrid method to enhance the computational speed. At the same time, the thought of characterizing the local dynamics at the vicinity of data center racks is unique.

Keywords

Citation

Fulpagare, Y., Joshi, Y. and Bhargav, A. (2018), "Rack level transient CFD modeling of data center", International Journal of Numerical Methods for Heat & Fluid Flow, Vol. 28 No. 2, pp. 381-394. https://doi.org/10.1108/HFF-10-2016-0426

Download as .RIS

Publisher

:

Emerald Publishing Limited

Copyright © 2018, Emerald Publishing Limited


1. Introduction

About 2 per cent of the total electricity generated is consumed by data centers in the USA (Brown et al., 2008). The power consumption in data centers is dominated by the servers and cooling units (Koomey, 2011). Server power consumption is vital, and hence cannot be minimized but cooling unit consumption can be reduced. Cooling units of raised floor plenum (RFP) data centers provide airflow to the servers. Data center environment involves the circulation of hot and cold air flows that make the system highly dynamic. An efficient way to understand these dynamics is through the response experiments and Computational Fluid Dynamics (CFD) modeling.

1.1 Literature background

RFP data centers have a distinctive pattern of airflow to supply the cold air to each server in the rack. Computer Room Air Conditioning (CRAC) units provide cold air to the server racks via perforated tiles on the plenum. Heated air collected toward the ceiling from the rear side of the racks and directed at the top of CRAC. This airflow pattern creates alternate cold and hot aisle zones inside RFP data center. Because of such pattern, there are chances of recirculation of hot air into cold aisle, and hence a proper ceiling is necessary (Makwana et al., 2014). Therefore, various parameters such as rack inlet–outlet temperatures, tile airflow rates and CRAC air supply temperature have been studied for different data center configurations using steady-state CFD analysis (Schmidt et al., 2005; Shrivastava et al., 2005; Fulpagare and Bhargav, 2015; Fulpagare et al., 2015; Zhang et al., 2008; Rambo and Joshi, 2003; Bhopte et al., 2006).

Thermal conditions within a data center are dynamic due to many parameters and hence cannot be captured through steady-state analysis alone (Jonas et al., 2012). The first transient numerical analysis was performed by Beitelmal and Patel (2007) to study the impact of CRAC failure on the temperature variations within the data center. The thermal mass characteristics with detailed transient boundary conditions for data center system-level simulations were highlighted by Ibrahim et al. (2012, 2010). However, the airflow rate entering rack cabinet can vary based on the pressure difference across the servers. This can be quantified based on the characterization of server fans (Nemati et al., 2015). Tracking of transient performance inside the data center is essential especially during cooling failure. Based on such an experimental study, Hussam et al. have recommended the modifications in ASHRAE (The American Society of Heating, Refrigerating and Air-Conditioning Engineers) guidelines (Alissa et al., 2016). Another room-level transient CFD study was presented to capture the effect of rack shutdown based on the relation of the time constant of server and its thermal mass by Erden et al. (2013) recommending to investigate the unexplored data center dynamics.

The CRAC set points play a very important role to achieve the dynamic thermal management of data center. This was achieved by HP Laboratories by proposing the control scheme that controls the CRAC set points based on the sensor data in the system (Bash et al., 2006). A CFD base tool (ThermoStat for rack mounted servers) was developed by Choi et al. (2008) to design the cooling system for optimized thermal management of the system. However, the dynamic response study and thermal control for different operating conditions were suggested as a future research scope by making ThermoStat as an open-source tool.

A transient numerical model of data center clusters for power variation and CRAC failure were presented to demonstrate that steady-state CFD could not predict the dynamic behavior of the system (Gondipalli et al., 2010). With similar data center layout, the cold aisle containment configurations were studied by Alkharabsheh et al. (2012). The server heat capacity, rack frames and CRAH (Computer Room Air Handler) were updated continuing with the same work by Alkharabsheh et al. (2014) and found that the modeling details of heat capacity and CRAH affect the system significantly compared to rack frame modeling.

Erden (2013) had studied the transient thermal response of air-cooled data center using experiments and CFD simulations. The server characteristics were determined experimentally and used for developing models. The transient response of server inlet air temperatures was recorded with various cases, namely, rack shutdown, chilled water interruption, CRAH airflow change and repeated CRAH fan failure. A room-level transient CFD results were validated with the experiments along with lumped capacitance model (Erden et al., 2013, 2014). Data center room-level transient study was conducted by Zhang et al. and found the significant impact on energy consumption by transient variation of environmental conditions and system load (Alkharabsheh et al., 2014).

Therefore, if the server and rack-level dynamics are well known, the system-level transient study can be understood easily. In this scenario, the accuracy of the numerical model of server rack becomes imperative. Zhang et al. (2008) studied the effect of rack modeling details in a data center test cell and found that the numerical details of the rack have a significant impact on CFD predictions. The conclusion was based on steady-state CFD investigations with leakage and turbulence flow modeling.

A black box compact server model with its physical properties was demonstrated first time by Vangilder et al. (2013) based on the experiments and numerical modeling (Pardey et al., 2015). The correlation of server capacitance with server thermal effectiveness (ratio of actual heat transfer to maximum heat transfer between air stream and thermal mass) was demonstrated for specific cases of servers to provide the server thermal mass properties. The porous media approach for the server was successfully implemented and validated with experiments by Almoli (2013).

As a summary, the dynamic study was performed on data center for the server characteristics, CRAC failure, rack shut down to observe the effect on the rack inlet air temperature. We have not found a study that has highlighted the response of rack outlet air temperature. The majority of the transient studies were focused on the server and room level and very few on the rack level. The focus on developing the details of the boundary conditions especially server characteristics in the past three to four years have added significant contribution in the literature. However, the rack-level local dynamics is yet to be understood and hence in the recent literature, it is highlighted to investigate the dynamics of the data center and reframe the ASHRAE guidelines. In this context, the rack-level transient study is the focus of the paper to track the local dynamics in the data center.

1.2 Research objective and approach

Based on the literature survey, especially on the transient study of data center, the dynamic study is imperative to achieve the self-sensing of the system. The extensive transient study on rack level would add some more insight in the existing literature. Therefore, the main objective of the paper is to study the rack-level dynamics in the RFP data center.

To fulfill the objective mentioned above, the research approach is as follows.

1.2.1 Dynamic response experiments.

This section highlights the lab facility, design of experiments and measurement details in a RFP data center. In this study, the transient variation of rack air temperatures on server simulator rack is captured for heat load variation.

1.2.2 CFD modeling.

Two computational domains were considered for transient CFD modeling, namely, single rack and three racks with their inflow and outflow aisles. Based on the rack outlet air temperature response and airflow uniformity, the single rack domain extended to three rack.

1.2.3 The insight of the study.

The thermal behavior and the model validation have been discussed based on the experiments and numerical simulation.

2. Experimental details

This section highlights the experimental lab layout, measurement details and design of the experiment. Server rack from the RFP data center room was considered for the analysis (Figure 1, Nelsonpartial, 2007). The 42U (1U = 4.45 cm) rack houses four server simulators, each having a height of 10U with its control settings of heating load and fan flow rates (Nelsonpartial, 2007; Arghode et al., 2013). One down-flow CRAC 2 was operational during the experiments, and other two were off. The tests were performed on the simulator and other two neighboring racks with its cold and hot aisle zone as highlighted in Figure 1.

Sensor placement plays a key role in the experiments. We placed eight (total 48) T-type calibrated thermocouple sensors (uncertainty ± 0.4 per cent) on the front and rear side of each rack in a way that temperature in the vicinity of all the servers would be covered (Figure 2). The sensors were placed in connection with wireless modules to an OSIsoft PI system network. The rack inlet, as well as outlet air temperature, was recorded until the system reaches steady-state condition. The heat loads of the simulator rack were incremented as a step input once the steady state was achieved.

2.1 Measurement details

To capture the effect of server heat load on the rack inlet and outlet air temperature, several parameters were held constant such as total heat load of all the racks, CRAC supply air temperature, server fan speed and perforated tile airflow rates. All the racks were kept at 10 kW constant heat load before starting the experiments. All four simulators were kept at 2.5 kW heat load with 0.15 m3/s fan speed. CRAC supply air temperature was kept at 17°C with CRAC blower speed of ∼3 m3/s. The perforated tile airflow rates were almost constant with an average velocity of 1.54 m/s.

The anemometric tool with perforated tile size was used to measure the tile air flow rate measurements as shown in Figure 3. The tool has four flexi glass walls with open top and bottom sides. The top side was covered by a stainless-steel wire grid (4 × 4) with 16 non-directional thermal anemometers (UAS 1200LP) of accuracy ±5 per cent of the measured value (operating range: 0.5-5 m/s, temperature range: 0-70°C) (Arghode et al., 2017).

Though we have not measured the airflow rates across the server simulator in this study, the measurement of average fan speed as per the dial setting is highlighted in Figure 4. The optical tachometer used to measure the average server simulator fan speed for different dial settings [Figure 4(b)]. The uncertainty for simulator fan speed were ±2.5 per cent (Arghode and Yogendra, 2016). The CRAC unit (model: FH740CMAJEIS055) was maintained at constant 38 per cent of blower speed. The corresponding reading in terms of CRAC flow rate was computed based on the manufacture data (Liebert, 2007, 2005).

2.2 Design of experiments

To record the response of rack inlet and outlet air temperatures, the initial conditions mentioned in the above section kept constant until steady state. The variation of input heat source for perturbation is one of the crucial steps in designing the experiment. The input heat source was stepped and ramped as shown in Figure 5. Each input was held constant for an hour after perturbation. The experimental results are highlighted in the results and discussion section.

3. Computational model

This section highlights the mathematical modeling, meshing and boundary condition details of the computational domain considered for transient CFD analysis. To perform the rack-level transient CFD analysis, we have selected two computational domains, namely, single simulator rack and three racks with their inflow and outflow aisles. The governing equations and boundary conditions are same for both the computational domain.

3.1 Governing equation and boundary conditions

The mathematical model for fluid flow in the computational domain was formulated from an incompressible form of Navier–Stokes equation with k-epsilon turbulence model (Karki et al., 2003). Three-dimensional, transient CFD analysis of current model involves momentum forces coupled with complex thermo-fluid interaction. The governing equations were discretized using finite volume method. The commercial tool STAR CCM+ from Siemens was used for the analysis (Siemens PLM Software). The hexahedral mesh with cell count around 0.9 and 2 million was used based on the mesh sensitivity analysis for single and three-rack domain, respectively (Celik et al., 2008) (Figure 6).

The transient CFD runs were started for time step of 1 ms with the initial constant boundary conditions. The solution was stabilized at ∼24 h on 2.5 GHz i5 processor with parallel runs on 4 cores. After getting the steady-state solution of around 2 h of total solution time, the input perturbation initialized as shown in Figure 5 using Algebraic Multigrid Method (AMG).

AMG method focuses on the relationship between grid width and frequency that helps to increase the computational speed, especially for nonlinear equations by increasing the convergence rate (Briggs et al., 2000). The AMG solver loses its efficiency taking longer to converge, but AMG with acceleration takes fewer cycles to converge. Sakaino (2016) has shown that the use of the multigrid method as an accelerated solver helps in saving computational time by comparing the multigrid method with MICCG (Modified Incomplete Cholesky Conjugate Gradient) in conjunction with GPU (Graphics Processing Unit). Therefore, we have implemented the AMG solver with acceleration in our case to enhance the computational speed. Though the experiments were performed with one simulator rack, the computational domain was considered with single simulator rack and then extended to three racks that include two neighboring server racks with its cold and hot aisle as shown in Figure 6.

The cold air inflow passes through perforated tiles from plenum chamber with tile flow rate specified by average inlet velocity of 1.54 m/s and 17°C temperature. To reduce the computational complexity, the plenum chamber was replaced by applying the vertical velocity flow inlet at perforated tiles. The top surface of the rear domain is modeled as exhaust with zero gauge pressure. All the server racks and simulators were modeled assuming porous media with 75 per cent porosity. The method for determining the inertial and viscous resistance for the porous media was adopted from Almoli (2013), which exactly replicates the flow conditions as that of the actual flow inside the servers (Table I). Each server simulator was assigned total heat source of 2,500 W, which contributes 10 kW heat on the rack. The interface between the rear side of the rack and the outflow aisle was modeled as server fan provided by the manufacturer (Nelsonpartial, 2007). The buoyancy effects were included using a Boussinesq approximation.

4. Results and discussion

This section highlights the results of the experiment and transient CFD analysis for single rack and three racks with its cold and hot aisle computational domains.

4.1 Experimental results

The responses were recorded for about 12 h of continuous runs for stepped and ramped heat load after the steady state. The rack inlet air temperature variation was negligible because of no mixing of hot air into cold aisle.

The most affected rack outlet air temperature response (highlighted in Figure 7) has shown the increase in temperature from 23 to 54°C. This significant increase of more than 30°C temperature shows the need of rack-level dynamic study. In general, with an increase in heating, the server fan speed increases to enhance the convective heat transfer. Here, the server simulator fan speed was kept constant, and there was no way to enhance the convective heat transfer with an increase in heat load. Therefore, such a large temperature difference has been observed after repetition of the same experiment. However, the similar responses on neighboring two racks have shown an increment of ∼2°C for the same input.

4.2 Transient CFD results

The flow and thermal dynamics of the first system (only simulator rack) were captured for horizontal and vertical sections of the domain as shown in Figure 8. It is observed that temperature increases with height and ∼10°C maximum temperature difference between the top and bottom section. Also, the rear side of the rack was found to be at a higher temperature than front section. The flow is passing through servers from tiles and then to the ceiling as shown in velocity streamlines in the second part of Figure 8.

From the velocity flow direction, three distinct zones have been highlighted. The inlet aisle temperatures are in a narrow band of ∼3°C with low-velocity recirculation. It was observed that the high-temperature zone at the bottom of the outlet aisle is due to low-velocity zone in the same area. The similar analysis performed on three-rack regions have shown more uniformity of flow and temperature in the domain than the previous model (Figure 9).

The experimental and computational responses were compared to simulator rack on input heat source as shown in Figure 10. The initial simulation results were underperformed due to the small domain with its inlet–outlet aisle. The experimental temperature response is a sudden increase in stepped input, but the simulation resulted in curvature response. It is because of the inertial resistance of the server mass contributing to the slower response than experiments. Due to the restricted airflow within one rack domain, ∼5-6°C higher temperatures have been observed in steady as well as transient variations. However, in the three-rack domain, the simulation results were validated with experiments. This validation within the acceptable range has been achieved by expansion of the computational domain as well as improving the server thermal characteristics (Table I).

The best design for data center environments recommends the temperature of 5-45°C (ASHRAE, 2013). However, in the current study, the maximum of 55°C temperature was observed, which is beyond the specified range. In such situations, the system’s hot aisle thermal profile will indicate the high alert to decrease the supply air temperature and ultimately increase in power consumption. Thus, the major contribution of this study is understanding the effect of rack outlet air temperature due to heat load variations. Therefore, the air temperature should be monitored at the outlet of the server racks apart from rack inlet air and cooling unit temperatures to understand the dynamic effects.

The rack-level dynamic can be summarized further by monitoring the rack inlet and outlet air temperature inside a data center with stepped or ramped heat loads after the steady state of the system. The same domain can be easily simulated in CFD with the extended domain (including neighboring racks) to understand the other parameter effects such as cold air flow rates, cold air temperature and different server configurations, which are not possible via experiments. This study can be extended to aisle level after understanding these effects on a single rack. Though we have the standard safe variable range for RFP data center provided by ASHRAE, understanding the effective critical limit of the system variable is imperative. This study not only provides the response experimental details but also provides simple steps to understand the rack-level dynamics.

5. Conclusions

The dynamics of the data center depends on the server workload (heat released due to workload) and cold air flow rate through CRAC units. The rack-level transient effects were studied through CFD as well as response experiments to characterize rack outlet air temperatures with a variation of the server rack heat load. During the experimental run, it was found that the system stabilizes in an hour. It has been observed that the rack-level dynamics have a significant effect on the thermal and fluid characteristics in hot and cold aisles. The velocity of air increases as it flows through the server, which enhances the convective heat transfer at the rear end of the rack. The additional workload or heating does not affect the neighboring thermal profiles significantly but majorly affects on the same rack. The rack outlet air temperature rise was significant and may cross the critical limit. Also, the airflow at the exhaust of the servers is influenced by the server fans, as well as the temperature differential.

The computational study on three racks with its cold and hot aisle have shown better performance compared with a single domain. The porous media approach with thermal mass characteristics has improved the computational results and validated with experiments. The multigrid method helped to reduce the computational time by increasing the convergence rate. This validated transient CFD results can be further explored to observe the effect of perforated tile airflow rate on the rack inlet and outlet air temperatures. The response data with all possible input variations will form the basis for data-driven modeling to be used in automation of the system.

Figures

The overall system layout with highlighted area of interest for experimentation and modeling. The details of server simulator rack with its control settings

Figure 1.

The overall system layout with highlighted area of interest for experimentation and modeling. The details of server simulator rack with its control settings

3D experimental domain for three racks with detailed sensor placement

Figure 2.

3D experimental domain for three racks with detailed sensor placement

Tile airflow rate measurement tool with photograph and schematic

Figure 3.

Tile airflow rate measurement tool with photograph and schematic

(a) Server simulator rack fan details, (b) measured fan speed

Figure 4.

(a) Server simulator rack fan details, (b) measured fan speed

The input heat source variation for the experiments

Figure 5.

The input heat source variation for the experiments

Computational domain with meshing details for simulator rack and three-rack models

Figure 6.

Computational domain with meshing details for simulator rack and three-rack models

Rack outlet air temperature response for the heat input to 2nd simulator for the highlighted temperature sensors

Figure 7.

Rack outlet air temperature response for the heat input to 2nd simulator for the highlighted temperature sensors

Temperature and velocity contours on horizontal and vertical planes for one rack domain

Figure 8.

Temperature and velocity contours on horizontal and vertical planes for one rack domain

Temperature and velocity contours on horizontal and vertical planes for three-rack domains

Figure 9.

Temperature and velocity contours on horizontal and vertical planes for three-rack domains

The first and second graphs are experimental and CFD response comparison on server outlet temperature (sensor 8 temperature as shown in Figure 7) for one rack and three-rack computational domain with the same heat source input on the 2nd simulator

Figure 10.

The first and second graphs are experimental and CFD response comparison on server outlet temperature (sensor 8 temperature as shown in Figure 7) for one rack and three-rack computational domain with the same heat source input on the 2nd simulator

Thermal property values assigned in computational model

Parameter Value Unit
Supply air temperature from tile 17 °C
Tile air flow rates (velocity) 1.54 m s−1
Server simulator porous inertial resistance 0.91 kg m−4
Server simulator porous viscous resistance 4.1 kg m−3 s−1
Server-specific heat 800 J kg−1 K−1
Server density 2,100 kg m−3
Server thermal conductivity 150 W m−1 K−1
Server simulator heat load 2,500 W
Neighboring rack heat load 10,000 W

References

Alissa, H.A., Nemati, K., Sammakia, B.G., Seymour, M.J., Tipton, R., Mendo, D., Demetriou, D.W. and Schneebeli, K. (2016), “Chip to chiller experimental cooling failure analysis of data centers: the interaction between IT and facility”, IEEE Transactions on Components, Packaging and Manufacturing Technology, Vol. 6 No. 9, pp. 1-18.

Alkharabsheh, S., Sammakia, B., Shrivastava, S. and Schmidt, R. (2014), “Dynamic models for server rack and CRAH in a room level CFD model of a data center”, IEEE Intersociety Conference on Thermal and Thermomechanical Phenomena in Electronic Systems (ITherm), Orlando, FL, pp. 1338-1345.

Alkharabsheh, S., Ibrahim, M., Shrivastava, S., Schmidt, R. and Sammakia, B. (2012), “Transient analysis for contained-cold-aisle data center”, Proceedings of the ASME 2012 International Mechanical Engineering Congress & Exposition IMECE2012, Houston, TX, pp. 1-8.

Almoli, A. (2013), “Air flow management inside data centers”, PhD thesis, University of Leeds.

Arghode, V. and Yogendra, J. (2016), Air Flow Management in Raised Floor Data Centers, Springer, New York, NY.

Arghode, V.K., Kumar, P., Joshi, Y., Weiss, T. and Meyer, G. (2013), “Rack level modeling of air flow through perforated tile in a data center”, Journal of Electronic Packaging, Vol. 135 No. 3, p. 30902.

Arghode, V.K., Kang, T., Joshi, Y., Phelps, W. and Michaels, M. (2017), “Measurement of air flow rate through perforated floor tiles in a raised floor data center”, Journal of Electronic Packaging, Vol. 139 No. 1, p. 11007.

ASHRAE (2013), “Data center networking equipment – issues and best practices”, ASHRAE TC9.9.

Bash, C.E., Patel, C.D. and Sharma, R.K. (2006), “Dynamic thermal management of air cooled data centers”, The Tenth Intersociety Conference on Thermal and Thermomechanical Phenomena in Electronics Systems, 2006. ITHERM ‘06, San Diego, CA, pp. 445-452.

Beitelmal, A.H. and Patel, C.D. (2007), “Thermo-fluids provisioning of a high performance high density data center”, Distributed and Parallel Databases, Vol. 21 Nos 2/3, pp. 227-238.

Bhopte, S., Iyengar, M.K., Sammakia, B., Schmidt, R. and Agonafer, D. (2006), “Numerical modeling of data center clusters – impact of model complexity”, Proceedings of IMECE2006, 5-10 November, Chicago, IL, pp. 1-10.

Briggs, W.L., Henson, V.E. and McCormick, S.F. (2000), “A multigrid tutorial”, Applied Optics, Vol. 37, pp. 7587-7595.

Brown, R., Masanet, E., Nordman, B., Tschudi, B., Shehabi, A., Stanley, J., Koomey, J., Sartor, D. and Chan, P. (2008), Report to Congress on Server and Data Center Energy Efficiency: Public Law 109-431, Environmental Energy Technologies Division Alliance to Save Energy ICF Incorporated.

Celik, I.B., Ghia, U., Roache, P.J., Freitas, C.J., Coleman, H. and Raad, P.E. (2008), “Procedure for estimation and reporting of uncertainty due to discretization in CFD applications”, Journal of Fluids Engineering, Vol. 130 No. 7, p. 78001.

Choi, J., Kim, Y., Sivasubramaniam, A., Srebric, J., Wang, Q. and Lee, J. (2008), “A CDF-based tool for studying temperature in rack-mounted servers”, IEEE Transactions on Computers, Vol. 57 No. 8, pp. 1129-1142.

Erden, H.S. (2013), “Experimental and analytical investigation of the transient thermal response of air cooled data centers”, PhD thesis, Syracuse University.

Erden, H.S., Khalifa, H.E. and Schmidt, R.R. (2013), “Room-level transient CFD modeling of rack shutdown”, ASME 2013 International Technical Conference and Exhibition on Packaging and Integration of Electronic and Photonic Microsystems InterPACK, Burlingame, CA, pp. 1-8.

Erden, H.S., Khalifa, H.E. and Schmidt, R.R. (2014), “A hybrid lumped capacitance-CFD model for the simulation of data center transients”, HVAC&R Res, Vol. 20 No. 6, pp. 688-702.

Fulpagare, Y. and Bhargav, A. (2015), “Advances in data center thermal management”, Renewable & Sustainable Energy Reviews, Vol. 43, pp. 981-996.

Fulpagare, Y., Mahamuni, G. and Bhargav, A. (2015), “Effect of plenum chamber obstructions on data center performance”, Applied Thermal Engineering, Vol. 80, pp. 187-195.

Gondipalli, S., Ibrahim, M., Bhopte, S., Sammakia, B., Murray, B., Ghose, K., Iyengar, M.K. and Schmidt, R. (2010), “Numerical modeling of data center with transient boundary conditions”, 12th IEEE Intersociety Conference on Thermal and Thermomechanical Phenomena in Electronic Systems (ITherm), Las Vegas, NV, pp. 1-7.

Ibrahim, M., Bhopte, S., Sammakia, B., Murray, B., Iyengar, M. and Schmidt, R. (2010), “Effect of thermal characteristics of electronic enclosures on dynamic data center performance”, IMECE Conference, 12-18 November, Vancouver, British Columbia, pp. 1-10.

Ibrahim, M., Bhopte, S., Sammakia, B., Murray, B., Iyengar, M. and Schmidt, R. (2012), “Effect of transient boundary conditions and detailed thermal modeling of data center rooms”, IEEE Transactions on Components, Packaging and Manufacturing Technology, Vol. 2 No. 2, pp. 300-310.

Jonas, M., Gilbert, R.R., Ferguson, J., Varsamopoulos, G. and Gupta, S.K.S. (2012), “A transient model for data center thermal prediction”, International Green Computing Conference (IGCC), San Jose, CA, pp. 1-10.

Karki, K., Radmehr, A. and Patankar, S. (2003), “Use of computational fluid dynamics for calculating flow rates through perforated tiles in raised-floor data centers”, HVAC&R Research, Vol. 9 No. 2, pp. 153-166.

Koomey, J.G. (2011), Growth in Data Center Electricity Use 2005 to 2010, Oakland, CA, Analytics Press, July, available at: www.analyticspress.com/datacenters.html

Liebert Emerson Network Power (2007), Liebert Deluxe System/3™ – Chilled Water – System Design Manual.

Liebert Emerson Network Power (2005), “Liebert Deluxe System™/3-6 to 30 Tons Precision Air Conditioning for Sensitive Electronic Equipment”.

Makwana, Y.U., Calder, A.R. and Shrivastava, S.K. (2014), “Benefits of properly sealing a cold aisle containment system”, IEEE Intersociety Conference on Thermal and Thermomechanical Phenomena in Electronic Systems (ITherm), Orlando, FL, pp. 793-797.

Nelsonpartial, G. (2007), “Development of an experimentally-validated compact model of a server rack”, Master thesis, Georgia Institute of Technology.

Nemati, K., Alissa, H.A., Murray, B.T., Sammakia, B.G. and Seymour, M. (2015), “Experimentally validated numerical model of a fully-enclosed hybrid cooled server cabinet”, ASME 2015 International Technical Conference and Exhibition on Packaging and Integration of Electronic and Photonic Microsystems InterPACK, San Francisco, CA.

Pardey, Z.M., Demetriou, D.W., Vangilder, J.W., Khalifa, H.E. and Schmidt, R.R. (2015), “Proposal for standard compact server model for transient data center simulations”, ASHRAE Transactions, Vol. 121 No. 1, pp. 413-422.

Rambo, J. and Joshi, Y. (2003), “Multi-scale modeling of high power density data centers”, IPACK’03, Kauai, HI, pp. 1-7.

Sakaino, H. (2016), “A dynamic stochastic optimization control model for data centers based on numerical modeling”, 15th IEEE Intersociety Conference on Thermal and Thermomechanical Phenomena in Electronic Systems (ITherm), Las Vegas, NV.

Schmidt, R.R., Cruz, E.E. and Iyengar, M. (2005), “Challenges of data center thermal management”, IBM Journal of Research and Development, Vol. 49 Nos 4/5, pp. 709-723.

Shrivastava, S., Sammakia, B., Schmidt, R. and Iyengar, M. (2005), “Comparative analysis of different data center airflow management configurations”, Adv. Electron. Packag. Parts A, B, C, pp. 329-336.

Siemens PLM Software, STAR CCM+: Commercial CFD software, available at: https://mdx.plm.automation.siemens.com/star-ccm-plus

Vangilder, J.W., Healey, C.M., Pardey, Z.M. and Zhang, X. (2013), “A compact server model for transient data center simulations”, ASHRAE Transaction, Vol. 119 No. 2, pp. 358-370.

Zhang, X., Vangilder, J.W., Iyengar, M. and Schmidt, R.R. (2008), “Effect of rack modeling detail on the numerical results of a data center test cell”, 11th Intersociety Conference on Thermal and Thermomechanical Phenomena in Electronic Systems, Orlando, FL, pp. 1183-1190.

Corresponding author

Atul Bhargav is the corresponding author and can be contacted at: atul.bhargav@iitgn.ac.in

About the authors

Yogesh Fulpagare has completed BTech Mechanical, MTech Thermal and Fluids Engineering (Gold Medal) from Dr Babasaheb Ambedkar Technological University Lonere. He is pursuing his PhD in Mechanical Engineering from IIT Gandhinagar in Energy Systems Research Laboratory as INSPIRE Research Fellow from Government of India. His research projects focus on Thermal Management of Data Center. He had worked as a doctoral research visitor at CEETHERM Laboratory in Georgia Institute of Technology Atlanta as an overseas research fellow. His research interests include heat transfer and fluid flow applications including system thermal management.

Dr Yogendra Joshi is a Professor at Georgia Institute of Technology, Atlanta, USA. He has BTech IIT Kanpur, MS State University of New York Buffalo and PhD University of Pennsylvania. He worked at University of Maryland, Naval Postgraduate School and in the semiconductor assembly industry on process thermal model development. He was named to the McKenney/Shiver Chair in 2004. His research deals with transport processes associated with emerging technologies in general such as electronic devices, packages and systems. He is a recipient of ASME Electronic and Photonic Packaging Division Outstanding Contribution Award in Thermal Management. He is a founding Co-Director of the Consortium for Energy Efficient Thermal Management (CEETHERM) and Emerging Technologies Thermal Laboratory (METTL).

Dr Atul Bhargav completed his BTech from IIT, Madras in 2002 and received his MS and PhD from University of Maryland, College Park, USA in 2008 and 2010, respectively. He has worked on diesel-, NG- and LPG-based fuel cell systems at Ballard Power Systems and University of Maryland. He has been a faculty member at IIT Gandhinagar since 2011 and is currently leading research and development efforts (through sponsored projects) in electronic cooling and hydrocarbon-based fuel cell systems.