Data Efficiency…the Transformational Technology of this storage era

Ambuj Goyal, GM of IBM storage, says IBM will compete by “delivering more value (efficiency)” to customers. Ambuj and IBM have it right. I believe DATA EFFICIENCY technologies will define the competitive landscape in storage over the next 10 years. Why? We are in the midst of the most profound technological change in storage since the spinning disk and today’s mega-change is being driven by four major technology advancements: cloud, virtualization, flash and open systems/software defined storage. Data efficiency technologies are critical to the widespread adoption of each of these technologies. Here’s how:

•Cloud – Cloud changes storage from a capital investment to an operating expense item and makes it very easy to compare service levels and cost. The metric becomes: $s/work unit and to compete service providers are aggressively deploying data efficiency technologies. Performance, management, and reliability are governed by the SLA, so all are assumed and comparable from provider to provider. Cloud storage providers will lead the charge with data efficiency at the source and target, so they can overcome network bandwidth cost and performance issues and offer the most cost effective storage solution to compete and grab market share and build sustainable businesses.

•Data transmission is still painful. In order to play in the hybrid world, data must be compressed and deduped and thereby only unique data is transmitted to the cloud. That means the enterprise will no longer rely upon cumbersome post-process, data efficiency schemes – it is too integral to the hybrid workflow. Data efficiency technologies will be automatic like RAID protection is today – inline, high performance, scalable and not requiring a super computer to operate.

•Virtualization – Virtualization (server and desktop) puts extraordinary I/O demands on conventional storage systems, creating huge performance bottlenecks and storage bloat. Data efficiency technologies eliminate these challenges and provide savings of 25X or more in most environments. In order to address this expanding market opportunity storage companies must have thin provisioning, dedupe and compression to be competitive.

•Flash – Performance (million + IOPS!) will not be an issue going forward as advancements in SSD technology and management take hold. Affordable and resilient performance is the challenge. Deduplication reduces write amplification and extends the life of flash to drop the effective cost of $s/work unit below spinning disk and promotes rapid and widespread adoption of SSDs in the market.

•Open Systems/Software Defined Storage – Enable a new breed of competitor to enter the market – intensively distributed, fast to market, easy to use and capable to address entry level spinning disk to high performance flash arrays and cloud. Efficiency and cost are the key business drivers for the open system companies who are aggressively implementing data efficiency technologies.

Last week, IDC analyst Natalya Yezhkova released a study forecasting 30% storage capacity growth 2013-2017 (down from 50% CAGR in recent years),” Data deduplication, data compression, thin provisioning and storage virtualization all will help enterprises limit their purchases of new storage capacity” and result in a 40% reduction of storage needs! In addition, IDC points to deduplication, compression and thin provisioning as three of the top five required storage attributes (Adoption and Benefits of Storage and Data Efficiency Technologies in the U.S. Storage Market, Jan 2013).

Today’s high impact technologies: Cloud, Virtualization, Flash and Open systems/Software defined storage are impacting IT. Data efficiency technology is the single enabler for their broad adoption making “Data Efficiency the Transformational Technology in this storage era”.