Nov 2021

Property Loss Control in Data Centers

The amount of data exchanged occurring every minute is mind-boggling. Worldwide data centers are the engine which drives this data exchange. Cloud computing which we hear a lot about today is typically performed in large data centers.

The data center industry continues to rapidly expand especially the so-called hyperscale data center which is one that exceeds 5,000 servers and 10,000 sq. ft. building footprint. There are about 600 hyperscale data centers worldwide which is twice as many as there were 5 years ago.

A very popular option instead of having your own data center is the use of a colocation center or carrier hotel.  This type of facility is where equipment, space, and bandwidth are available for rental to customers who typically house privately-owned servers and networking equipment.

THE RISKS
The physical risks associated with data center operations include fire/explosion, utility (e.g., power) interruption, natural hazard exposure and liquid release impact. These risks can result in loss of data center operations and could be extensive depending on the situation.

The following briefly summarizes the key physical hazards found in data centers and the standards/codes which recommend ways minimize the hazards. See the referenced documents for complete details.

Type Hazard  Hazard Details Protection 
Fire Energized Electrical Equipment   Fire involving energized electrical equipment (e.g., servers) will grow slowly, release large amounts of corrosive black smoke, and cannot be completely extinguished until the power is shut off. Provide fire detection and suppression as well as a means to power down per FMDS 5-32 & NFPA 75
  Group Cables/Wires Power, network, communication, data, signaling, etc. type cables (multiple conductors) and wires (single conductor) are typically used extensively and can be found in large groups(i.e., ~15 or more). The conductor insulation as well as the outside jacket may be combustible plastic such as PVC, polypropylene or polyethene, etc. Vertical configuration of grouped cables/wires and multi-tier cable/wire trays present the most severe fire hazard arrangement. Use Group 1 (non-fire propagating) cables/wires per FMDS 5-32 and FM Approvals Standard 3972. Or use cables/wires with maximum flame spread distance of 5 ft (1.5 m) or less per NFPA 262
  Cable Trays, Raceways & Routing Assemblies Raceways and routing assemblies are used to hold and support cables and wires. The raceway is an enclosed channel while the routing assembly is not enclosed. Noncombustible (i.e., metal) and combustible (i.e., plastics) materials are used to construct the cable/wire trays, raceways and routing assemblies. Materials should be non-combustible, approved to FM Approvals Standard 4910, plenum-rated raceways listed to UL Standard 2024 and NFPA 262. Fire protection for multi-tier cable/wire trays should follow FMDS 5-31 
  Insulation Materials Combustible foam insulation can be used in the following applications: under the raised floor; chilled water piping; make-up air ductwork; inside computer room air conditioning (CRAC) units. Groups of piping and ductwork with combustible insulation in the vertical configuration present the most severe fire hazard arrangement. Insulation should be noncombustible or approved to FM Approvals Standard 4924 and NFPA 274
  Hot/Cold Aisle Containment The physical barriers (i.e., walls & ceiling) which separate the hot air generated from equipment (i.e., servers) from mixing with the cold equipment cooling air from the HVAC system. Containment systems can be constructed of combustible materials including plastics Use noncombustible materials or plastics specification tested to FM Approvals Standards 4882 & 4910. Provide fire detection & suppression per FMDS 2-0, 4-9 & 5-48 and NFPA 13, 72 & 2001
NaturalHazards Flood  Do not locate data center building and support utilities within the 500-year flood zone (0.2% annual exceedance).  Avoid flood prone areas per FMDS 1-40 & 5-32. If located in flood prone area, provide protection per FMDS 1-40.
  Earthquake  Anchoring and bracing of equipment (e.g., server racks, mainframes, etc.) and raised floor. Bracing of sprinkler systems and other water piping. Seismic shutoff valves on natural gas piping Avoid high frequency and intensity earthquake regions. Anchoring and bracing of buildings and equipment per FMDS   1-2 &  5-32. Follow FMDS 2-8 & NFPA 13 for bracing of sprinkler piping and water supplies. Install seismic gas shutoff valve per FMDS 1-11.
  Windstorm  Design buildings for expected wind forces in order to prevent any breach of the data center envelope. Limit exterior windows and doors to the data center.  Do not locate in areas subject to severe windstorm exposure. Use FMDS 1-28 for best building protection against windstorm.
  Wildland Fire Wildfires can occur when there are long periods of dry weather which cause vegetation to become very dry. When this is combined with strong winds, they increase the intensity of a fire spreading over a wide area, exposing buildings and equipment in the vicinity.   Avoid known regions where there is a high frequency of seasonal wildfires. Use proper construction materials, limit vegetation around buildings and consider outdoor sprinkler protection per FMDS    9-19 & NFPA 1143
 Power Power Supply Reliability Power interruption at a data center is a high frequency event and strategies to limit impact by a reliable design should be followed. Reliable source of power is critical for data centers. Utility company power, generators, batteries should be arranged to provide an uninterruptible source of power per FMDS         5-23, 5-32 & 5-33 and NFPA 37, 110 & 111
  Diesel Generator  Fuel Storage & Distribution Diesel generators/storage tanks can be located in basement areas and generators on roof tops. A diesel fuel leak can result in an ensuing spray and/or pool type fire. Arrange fuel storage tanks & piping per FMDS 5-23 & 7-32 and NFPA 30 & 110
  Static (e.g., battery) Uninterruptible Power System (UPS)  Batteries can be lead-acid, VLRA (valve-regulated) and lithium-ion. Batteries are usually placed on metal shelves and located in a cut-off room.  Thermal runaway leading to fires and/or explosions is critical hazard. Locate and protect battery UPS installations in accordance with FMDS 5-23 & 5-33 and NFPA 110, 111 & 855
  Rotary UPS Motor-generator/battery UPS – Uponpower interruption the battery bank provides sufficient ride-through to sustain the load until the standby generator comes up to full operating speed.Engine-coupled rotary UPS – Upon power interruption, power is supplied by the flywheel to the motor-generator unit, while in turn supports the critical load for a few seconds before the diesel engine runs up to full speed. The cutch is then engaged to provide mechanical power to the motor-generator to supply continuity of power to the load For protection of the various electrical, mechanical and fire protection  hazards associated with rotary UPS systems, the following codes/standards provide such guidance: *  FMDS 5-12, 5-17,      5-23, 5-33, 13-7 and    13-18.* NFPA 30, 37, 110, 111 & 855
  Energy Storage Systems (ESS) Large numbers of lithium-ion batteries assembled together which are capable of storing energy in order to supply electrical energy at a future time. Thermal runaway leading to fires and/or explosions is critical hazard. Presently, there is a high frequency of fire events associated with ESS systems so extra care should be taken. Location siting, construction features, fire detection & suppression and safety controls/interlocks such as detection of thermal runway in its incipient stage should follow FMDS 5-33 & NFPA 855. 
Liquid Liquid Release Sources of liquid which expose the data center include roof drain piping, domestic water piping, chilled water piping, HVAC air handlers, condensate drains, heating water, coolants, steam condensate, etc. Avoid sources of liquid release in the data center space, especially at or above the ceiling. Provide liquid leak detection when appropriate per FMDS 1-24, 5-32 & NFPA 75

 

 

Contact Risk Logic with any questions you may have regarding Loss Control in Data Centers

References include but are not limited to:

FM Global Property Loss Prevention Data Sheets (FMDS)

FMDS 1-2 – Earthquakes

FMDS 1-11 – Fire Following Earthquake

FMDS 1-24 – Protection Against Liquid Damage

FMDS 1-28 – Wind Design

FMDS 1-40 – Flood

FMDS 2-0 – Installation Guidelines for Automatic Sprinklers

FMDS 2-8 – Earthquake Protection for Water-Based Fire Protection Systems

FMDS 4-2 – Water Mist Systems

FMDS 4-9 – Halocarbon and Inert Gas (Clean Agent) Fire Extinguishing Systems

FMDS 5-12 – Electric AC Generators

FMDS 5-14 – Telecommunications Systems

FMDS 5-17 – Motors & Drives

FMDS 5-31 – Cables and Bus Bars

FMDS 5-32 – Data Centers and Related Facilities 

FMDS 5-23 – Emergency and Standby Power Systems

FMDS 5-33 – Electrical Energy Storage Systems

FMDS 5-48 – Automatic Fire Detection

FMDS 7-32 – Ignitable Liquid Operations 

FMDS 9-13 – Evaluation of Flood Exposure

FMDS 9-19 – Wildland Fire

FMDS 13-7 – Gears

FMDS 13-18 – Industrial Clutches & Clutch Couplings

FM Approvals Standards

Class Number 3972 – Test Standard for Cable Fire Propagation

Class Number 4882 – Approval Standard for Class 1 Interior Wall & Ceiling Materials or Systems for Smoke Sensitive Occupancies 

Class Number 4910 – Examination Standard for Cleanroom Materials

Class Number 4924 – Approval Standard for Pipe and Duct Insulation

National Fire Protection Association (NFPA) Codes/Standards

NFPA 13 – Standard for the Installation of Sprinkler Systems 

NFPA 30 – Flammable & Combustible Liquids Code

NFPA 37 – Standard for the Installation and Use of Stationary Combustion Engines and Gas Turbines 

NFPA 70 – National Electrical Code, Article 645

NFPA 72 – National Fire Alarm & Signaling Code

NFPA 75 – Standard for the Fire Protection of Information Technology Equipment

NFPA 76 – Standard for the Fire Protection of Telecommunications Facilities

NFPA 110 – Standard for Emergency and Standby Power Systems

NFPA 111 – Standard on Stored Electrical Energy Emergency and Standby Power Systems

NFPA 262 – Standard Method of Test for Flame Travel and Smoke of Wires and Cables for Use in Air-Handling Spaces

NFPA 274 – Standard Test Method to Evaluate Fire Performance Characteristics of Pipe Insulation 

NFPA 287 – Standard Test Methods for Measurement of Flammability of Materials in Cleanrooms Using a Fire Propagation Apparatus (FPA)

NFPA 855 – Standard for the Installation of Stationary Energy Storage Systems

NFPA 1143 – Standard for Wildland Fire Management

NFPA 2001 – Standard on Clean Agent Fire Extinguishing Systems 

Underwriters Laboratories Standards

UL 2024 – UL Standard for Safety Cable Routing Assemblies and Communications Raceways