Integrated System Testing – Paul Smethurst, CEO Hillstone, highlights the issues of load bank testing 100MW data centres.
The insatiable demand for data coupled with the growth of cloudbased services has changed the European data centre landscape with the arrival of the mega data centre. The mega data centre, which allows global software giants like Microsoft, Google and Apple to provide our day-to-day IT services, is also the foundation for colocation providers such as Digital Reality Trust, Equinix, Telecity and Interxion to facilitate connectivity to the cloud for multinational conglomerates in banking, telecoms, oil and gas. With such a rapid expansion of cloud services, how do you commission mega data centres of 20MW, 40MW, 80MW and 100MW?
Historically, the largest ever load bank solutions have been in the oil and gas sector and used 50-70MW of containerised load banks. The load bank would be situated outdoors as part of very large temporary generator power projects. Fortunately, the evolution of the mega data centre has taken a practical, modular build approach, with roll out phases of dual halls at 2500KW or as a single 5000KW empty white space. However, such a reduction in rating does not reduce the challenges of sourcing the quantity of required load banks needed to complete integrated system testing (IST).
Integrated System Testing
The primary objective for data hall IST commissioning is to verify the mechanical and electrical systems under full load operating conditions, plus maintenance and failure scenarios, to ensure the data hall is ready for the deployment of active equipment. Todays’ IST requires a package of equipment that will closely replicate the data hall when in live operation. Server simulators, load banks, flexible cable distribution, automatic transfer switches, data logging for electrical power, environmental conditions (temperature and humidity) and the ability to incorporate the load banks within temporary hot aisle separation partitions give the foundations for a successful IST. These tools allow the commissioning report to present a computational fluid dynamics (CFD) model of the actual data hall operation.
The selection and use of server simulators, typically rated between 3KW to 6KW as per the expected IT rack loads, gives a granular distribution of low delta temperature across the data hall. Such detailed consideration of air distribution during testing is required due to the scale of the IST and the increased volumes of air that’s affected in the mega data centre environment. This replicated heat allows mechanical cooling systems to run at optimum design temperature, which ensures deployment of future active IT equipment will not overheat and fail. If the commissioning occurs prior to deployment of IT cabinets, server simulators can be used and housed in portable mini-towers for distribution across the empty space.
The requirement for using flexible cable distribution facilitates the ease of cabling high quantities of 5KW to 20KW-rated load banks to A and B feeds on a PDU or busbar infrastructure. If the cable distribution also includes the ability to automatically transfer the load then the commissioning team can replicate maintenance procedures and failure scenarios during the IST. In order to report the successful operation and performance of the room during the IST, the commissioning team will need to monitor and record electrical and environmental data. Having electrical data available within the load package avoids the use of a power analyser with exposed connections to live terminals in the data hall. When server simulators include temperature sensors extensive temperature analysis allows CFD modeling to be performed in the testing period. While the data centre will ultimately have a building management system (BMS) and a data hall fitted out with the latest DCIM system, they are unlikely to be fully operational at the time of testing. The project team should source a provider offering only server simulators at the earliest opportunity in the project and avoid alternative load bank solutions that will cause delays to the IST program.
The restricted choice in the market dilutes the availability of load bank choices and equipment. Decisions to select on cost the wrong type of load bank solution can compromise the validity of the IST, but the unknown hidden problems will not manifest until the data hall goes live with active IT equipment. The temptation to choose 20KW 3 phase industrial space heaters rather than load bank server simulators effects the commissioning of mechanical cooling systems. The design of such heaters prevents the elevation of the ambient room temperature reaching the design criteria needed to commission the CRAC or AUH units. Some suppliers have removed the thermostatic controls only to find the space heater over heats and in some circumstance catches fire. The choice of large 110KW load banks can be justified when testing site equipment such as PDU panels, bus bars or switchboards to Level 3 ASHRAE requirements. These load banks provide a cost effective solution to proving the electrical infrastructure of the mega data centres, however they will create localised hotspots or areas of concentrated heat should they be used for the commissioning of the cooling systems. In extreme circumstances during tier certification the electrical load has been provided from 2KW infrared heaters or 1KW hair dryers. Infrared heaters create an ambient temperature of >40 degrees Celsius and wall skin temperatures of 70 degrees Celsius. Hair dryers are not designed for continuous operation as required in an IST. This type of low cost solution should not be considered to replicate the operation of IT equipment and risks costly delays while compromising the integrity of the testing program.
Achieving Cost Savings
Completing an IST to budget takes on a greater importance when commissioning a mega data centre, especially with the number of rental load banks that will be required. The increase in size of the facility will increase the time needed to complete the commissioning, so by combining the latest technologies traditional delays often associated with load banks can now be avoided. By selecting solutions that give enhanced data logging, the commissioning report will also give the client detailed levels of operating information and fully auditable reports for future tenants considering use of the space. Data logging can also be used in CFD models to ensure that the mega data centre is ready to use.
Originally Published Here