Data hall protection challenges

At FM Global, we are in the business of property loss prevention and we realise the importance of data centres remaining operational. We believe in the simple proposition: it is better to prevent a loss rather than recover from one. When any company is unable to function due to a loss, its customers go elsewhere, its brand evaporates, competitors move in and share value can tumble. For data centres the risks are even higher. When they experience downtime, the fundamental reputation that they have worked so hard to create – and upon which its whole future relies – is at stake. To keep this value secure, it's important to do the right things to make sure you don't get knocked down in the first place.

Fire is a key threat
From our loss history, the key threats to data centres in order of frequency are liquid damage e.g. from cooling systems (not from fire sprinkler systems); electrical breakdown; fire and theft. Although fire is only the third most common threat, it causes, on average, four times the amount of damage as liquid or electrical breakdown. In addition, in the worst case a fire might not be contained to the hall of origin and can therefore threaten the whole facility in a way the other perils can’t.
Equipment housings, cabling, trunking and hot/cold aisle containment systems are largely made of combustible plastic, which can allow a fire to spread rapidly. In fact, testing carried out at our Research Campus has shown that these materials cause serious fires, which spread easily and are capable of disrupting data centres for long periods of time. The most likely ignition source is an electrical failure.

Fire-fighting from the emergency services alone may not contain a data centre fire as they will not risk lives by entering a compartment in which there is an established, uncontrolled fire just to protect property. Instead, they often try to contain the fire to the room or hall of origin. Any equipment in the room of origin may be destroyed, while equipment not directly damaged by the fire is likely to be severely affected by smoke or hose-streams.

Protection Strategies
Thankfully, there are a number of measures that data centres can take to reduce the impact of fire on their business.
Prompt detection and manual intervention are an excellent first line of defence but experience shows that they should not be relied upon and automatic fire protection should be installed:

The misconceptions of automatic sprinklers
The most reliable and established form of fire protection is automatic sprinklers. There is sometimes reluctance to install water based systems over data centres, but it’s important to remember that only the heads over the fire area discharge water. In almost all circumstances, whatever gets wet is already in direct contact with fire or smoke.

Many of the world’s major IT companies specify sprinkler protection for their data centres. Accidental discharges are extremely rare and the best approach is to use ‘wet’ sprinkler systems where pipes are permanently full of water and the systems can respond instantly.

Where concerns about accidental discharge cannot be assuaged, ‘pre-action’ systems can be used. For these to discharge water, a sprinkler head has to open and the fire or smoke detection system has to operate. A damaged head or pipework will not cause a release of water. Sprinkler systems have the advantage that their supply of extinguishing agent is relatively unlimited and should a fire restart, they can be turned on again.

Water mist protection
Water mist protection is similar to sprinkler protection but typically uses much higher pressures from smaller orifice heads or nozzles to produce a fine mist rather than the relatively steady downpour of sprinklers. This can swirl around and envelop fires and even enter equipment enclosures. They can use significantly less water than sprinkler systems but typically at the expense of flexibility; there are limits on the ceiling height they can be used under and are not currently approved for use as pre-action systems. However, ongoing research and testing continues to expand their capabilities.

Gas extinguishing systems
Gas extinguishing systems are a common choice. The protected room or compartment must be sufficiently sealed to allow the necessary concentration to be maintained long enough to extinguish the fire. A ten minute hold time is a typical minimum, though designs should also allow for the time taken for trained personnel to respond. The gas tightness of the room can be established by ‘door fan’ testing without discharging the system. If necessary, the room can be made more gas tight and/or the quantity of gas increased to allow for leakage.

For gas to be effective, equipment needs to be arranged so that when they are activated, ventilation systems cease to draw fresh air into the compartment to avoid dilution of the extinguishing agent. Cooling systems should continue to run and re-circulate air in the compartment until affected equipment is shut down.
A key point with gas protection is that generally there is only enough gas for a single discharge (although some high value locations have a reserve to give them a second shot). Once the gas concentration is lost, the data hall is unprotected until the agent can be recharged. This is a problem as there is a significant chance that unless power to the affected area has been shut off, an electrical fault that started the fire will still be present. Faults can be serious ignition sources without drawing enough current to trip circuit breakers. This could leave the hall with an active ignition source, a fuel package that is heated, and no protection. It’s critical that following a fire and gas discharge, equipment is safely shutdown and power isolated.

Removing the ignition source is a challenge
Power shutoff is a challenging issue. Even thinking about it is heretical in most data centres and staff may be reluctant to act for fear of censure if a false alarm has occurred. Prompt investigation may be difficult if it is not judged safe to enter a compartment where a fire may still be smouldering and into which extinguishing gas has been released (the gases in use are safe for areas that are normally manned but it is expected that the area will be evacuated when the systems activate). This makes it difficult to determine exactly where the fire started and hence what equipment to shutdown, meaning the whole hall or a large part of it has to be de-energised. Bear in mind that the fire may not be immediately obvious as modern detection systems can trigger protection at a very early stage (generally a good thing).

Even when the decision is made to shutdown, multiple connections to grid; UPS units; diesel generator back-ups and the needs of the data processing equipment itself can make turning the power off a complicated process. In the stress of an emergency, perhaps in the middle of the night with few trained staff on hand, things can go badly wrong.

If a controlled automatic power down activated by the fire protection systems is not judged practical, then good manual procedures are a partial solution. Companies need to make sure that:
· There are clear instructions on how to achieve a controlled shutdown
· It is clear when power-down should be initiated
· It is clear who should make the decision and that person will be available on site
· Adequately trained employees cover all shifts
· Employees are empowered to initiate shutdown in an emergency without fear of censure
· The extinguishing concentration can be maintained for as long as it takes to make the decision to power down and act on it or to have trained personnel in place within the hall ready to respond manually to any re-ignition

Failure is not an option
In today’s business world, the importance of data protection means automatic fire protection is essential for data halls. The reality is that the short term costs of implementing effective protection is nothing compared to the long term reputational and financial consequences of extended downtime, or even losing a facility altogether. Despite this, different organisations will have varying requirements, and clients leasing server time or just space for their own equipment may also have their own ideas about protection. For instance, certain organisations prefer sprinklers or water mist, and others may be against the idea of water based protection and insist on gas. Whatever the form of fire protection used though, it’s imperative that data centres ensure that they do not suffer the kind of operational disruption that can damage their reputation, market share or even share price.