Friday, 14th August 2020

Elastic NVMe storage powers GPU servers

Storage architecture using Excelero NVMesh®, BeeGFS file system on Flash‐IO Talyn® systems from Boston Ltd. reduces machine learning training time from three‐to‐four days to under an hour, enables new analyses.

Excelero says that the Science and Technology Facilities Council (STFC) has deployed a new high performance computing (HPC) architecture to support computationally intensive analysis including machine learning and AI‐based workloads using the NVMesh™ elastic NVMe block storage solution. The deployment, done in partnership with Boston Limited, a provider of high performance, mission‐critical server and storage solutions, is enabling researchers from STFC and the Alan Turing Institute to complete machine learning training tasks that formerly took three to four days, in just one hour – and other foundational scientific computations that researchers formerly could not perform.

The Science and Technology Facilities Council is a part of U.K. Research and Innovation (UKRI) and supports pioneering scientific and engineering research by over 1,700 academic researchers worldwide on space materials and life sciences, nuclear physics and much more.

Research involves a wide variety of data‐rich analyses on data generated by large‐scale experimental facilities and observatories. These include cryo electron microscopy, synchrotron light, and other techniques. Workloads are massive – often hundreds of terabytes (TB) – and require both fast compute and fast storage. For example, a typical workload involves a significant amount of computing multimodal data, such as those obtained from X‐ray and neutron sources.

STFC’s Scientific Machine Learning (SciML) Group was established with the aim of enabling scientists to analyse large amounts of data, with the group bringing machine learning and AI expertise. The group routinely utilises deep neural networks running on state of the art NVIDIA® DGX‐2™ GPU computing systems located at the Scientific Data Centre at its Rutherford Appleton Laboratory site near Oxford. As the need for image processing expanded, the use of GPU‐based workstations needed to be extended to support the high throughput and low latency required for end‐user response times. Adding NVIDIA DGX‐2 servers offered higher computational support, yet lacked the enterprise‐level storage functionality required to scale out the resource across the hundreds of researchers.

High performance computing solutions provider, Boston Ltd., worked with STFC to evaluate all‐flash arrays and open systems‐based storage options, and commissioned a benchmark of Excelero’s NVMesh for share NVMe Flash storage at local performance.

Boston Ltd.’s benchmark results showed the proposed STFC architecture delivered an average latency of 70 microseconds – nearly one‐quarter of the typical 250 microsecond latency of traditional controller‐based enterprise storage when running NVIDIA validation tests on each NVIDIA DGX‐2 system. The combined NVMesh and BeeGFS deployment therefore showed potential for meeting STFC’s high throughput, low latency demands.

STFC’s storage architecture now includes two Boston Flash‐IO Talyn® systems built on SuperMicro building blocks, networked via a Mellanox 100G InfiniBand network to two NVIDIA DGX‐2 computing systems, each with 16 NVIDIA 32GB V100 SXM modules.

Operational since July 2019, STFC’s storage architecture enabled running training sets that formerly took three to four days, in under an hour. With the BeeGFS file system providing a single name space to simplify management and virtualisation, and the low latency and high throughput of its NVMesh system, STFC now has a GPU computing architecture where storage no longer presents a bottleneck, even with its complex research needs.

Backed by its new deployment, user communities surrounding this new system, including users from the Alan Turing Institute, are now able to carry out machine learning research projects covering a number of disciplines, including environment, life sciences, materials, space sciences and astronomy.

“In benchmark testing we quickly saw that our Flash‐IO Talyn systems with the Excelero NVMesh software delivered a significant performance enhancement over traditional controller‐ based architectures found in all‐flash arrays – and the ease of installation of a packaged solution,” said Matthew Parfitt, HPC commercial manager at Boston Ltd., who oversaw the deployment.

“Fundamental scientific research needs a clear computational path to completion, without the storage bottleneck that is endemic when NVMe resources are not virtualised,” said Lior Gal, CEO and co‐founder of Excelero. “We’re proud that our NVMesh software helped our partner Boston Ltd. put an essential building block in place in the STFC architecture to support STFC’s vital initiatives.”

The Fibre Channel Industry Association (FCIA) says that the International Committee on Information T...
The Flashtec NVMe 3108 controller enables power and form-factor optimized enterprise NVMe SSDs with...
Nebulon has released the results of an independent survey completed by IT decision makers at 500 com...
Fifth annual data storage report aids industry in navigating the budgetary and infrastructure challe...
E3.S form factor enables higher density and performance driving data center scalability.
Containerised applications can enjoy reliable disaggregated storage that performs like local flash.
Kingston Digital Europe has begun shipping the 7.68TB model of the Data Center 1000M (DC1000M) NVMe...
Intel has introduced its 3rd Gen Intel Xeon Scalable processors and additions to its hardware and so...