Not all Flash arrays are designed for today’s data centres

THE CONCEPT OF VIRTUALISATION has existed for a long time. Mainframes, for example, were the first to use this concept in the 1970s. However, this model of virtualisation was monolithic – i.e. based on a single compute structure (the Mainframe) that communicates with a single shared data repository (storage array) through a data path on an isolated network.

The modern data centre uses virtualisation much differently. It is implemented in a distributed fashion, whereby many workloads across multiple servers are consuming I/O. However, data is still predominantly stored in a shared monolithic device.

Does it still make sense for this data to be stored in a single shared array - even if that array is using high speed media like flash?
Applying a system built for an old model introduces significant challenges, leading to inadequate performance. Let’s explore this further…

Storage Area Network (SAN) latency
The most common virtual data centre architecture consists of a group of virtualised hosts connected via a network to a centralised storage array, a concept known as a Storage Array Network (SAN). SAN design typically follows the Core-Edge fabric topology, whereby high capacity core switches are placed at the centre of the fabric. The servers are sometimes connected directly to the core switch, but more often linked to switches at the edge. An inter-switch link connects the edge switch to the core switch. The same design applies to the storage array; it’s either connected directly to the core switch or to an edge switch. (See Figure 1)
Each layer has its inherent role in the design. Although core switches can offer a lot of capacity, there is a limitation on the number of ports. Hence, edge switches are used to extend the port count and connect a greater number of devices.

While the Core-Edge fabric topology delivers the necessary resilience and performance along with scalable port density, there are a number of design tradeoffs to consider. Most notably, each additional network hop increases latency. Connecting more systems per switch decreases the number of hops, but this impacts port count and limits future growth. It also restricts bandwidth availability, as network oversubscription is amplified.

These SAN design challenges are not addressed by adding faster media, like flash, into the storage array. Network latency plays a pivotal role in application performance; this means AFAs will never be an optimal solution for dynamic data centre environments.

Storage controller bottlenecks
The other big issue with shared storage environments, like AFAs, comes down to storage controller bottlenecks. Storage controllers are built from commodity server hardware running proprietary software to support storage protocols and data services. The most popular storage arrays are configured with storage controllers with two CPUs ranging from four to eight cores. This means that the typical enterprise storage array with a redundant pair of controllers is equipped with 16 to 32 cores in total, with CPU cycles dedicated to the following types of activities:
 Setting up and maintaining data paths.
 Redundancy and data availability.
 Data movement and data integrity.
 Maintaining RAID levels and calculating and writing parity data.
 Data services such as snapshots and replication.
 Internal data saving services such as deduplication and compression.

The de-facto method for adding storage performance is to add more disks to the array. However, storage controller cycles are a fixed resource. As you add more disks the controller becomes an increasingly bigger bottleneck. To address this, many storage arrays contain read/write caches. For example, writing to a write cache is much faster than writing to disk, allowing for faster write acknowledgments. However, in order to avoid making the storage controller a single point of failure, redundancy is necessary to avoid data loss. This means writes must be mirrored between caches on both controllers, which requires careful co-ordination between the controllers to ensure data coherency. This co-ordination consumes substantial CPU cycles on both controllers, which can actually degrade performance and obviate some of the benefits of the cache.

Increasing cache sizes at the controller layer just delays the point at which the write performance problems begin. No matter the size or speed of the cache, it’s still the back-end that is being overwhelmed.
Decoupled storage – The path ahead
How do you solve the above architectural issues with monolithic storage, like AFAs, to deliver the best performance in today’s virtualised data centre?

You decouple storage performance from capacity.
Decoupling puts active data close to the VM in server-side flash or RAM, while capacity is left in shared storage. This greatly reduces the amount of data traffic traversing the storage network, and thus avoids the classic shared storage bottlenecks described above. This also creates an environment where you can add storage performance (i.e. IOPS) as needed by adding inexpensive server-side resources in a non-disruptive manner. Because IOPS grow linearly with compute resources, performance can be added precisely and cost-effectively, eliminating the need to overprovision storage.

The era of traditional shared storage is at an end. The fundamental architectural limitations of a monolithic array are an imperfect fit for the requirements of the virtualised world. The emergence of the decoupled architecture provides a better, more scalable solution.
The future data centre is one step closer!

Caption
Figure 1: The typical SAN design follows a core-edge topology, which provides resiliency but introduces potential performance and scale issues