Data centre skills shortage threatens growth

By 2016, the availability of capacity and performance management skills for horizontally scaled architectures will be a major constraint or risk to growth for 80 per cent of major businesses.

"While major organisations continue to maintain and sustain their conventional capacity-planning skills and tools, they need to regularly re-evaluate the tools available and develop the capacity and performance management skills present in the web-scale IT community," said Ian Head, research director at Gartner. "Web-scale IT organisations do things differently — they learn from one another. Conventional IT organisations can use some of the web-scale techniques heading for mainstream adoption in the next three years."

To help I&O professionals focus their efforts, Gartner recommends the following action:

Embrace stateless application architectures and horizontally scaling infrastructure architectures to improve capacity and performance management
Web-scale organisations extend the virtualisation concept by architecting applications to be stateless wherever possible (i.e., not recording and relying on user session data to function) and to work within a horizontally scaling infrastructure that facilitates rapid and near-real-time reallocation of resources. Furthermore, workloads associated with these applications must be standardised and categorised (e.g., latency-sensitive and compliance-driven workloads) to enable the infrastructure team to assign infrastructure resources appropriately. These vital characteristics of web-scale IT form the foundation for capacity and performance management.

Services constructed in this way are better equipped to scale geographically and share multiple data centres with limited impact on user performance. This approach also blurs the lines between capacity planning, fault-tolerant designs, and disaster recovery. Because individual virtual machines, nodes and then larger parts of the infrastructure may be lost, careful adjustment of both application and infrastructure will allow graceful performance degradation.

Develop demand-shaping techniques to provide acceptable performance
Demand shaping uses various techniques to adjust the quantity of resources required by any one service so that the infrastructure does not become overloaded. Gartner predicts that through 2017, 25 per cent of large organisations will use demand shaping to plan and manage capacity, up from less than 1 per cent in 2014.
Techniques for demand shaping include limited launches — sometimes known as "canary launches" — where new functionality is released to a limited section of the user base. The take up and load can then be measured and extrapolated to the wider user base and decisions made to allocate more or fewer resources to the new services as they are rolled out to different segments of users. "Dark launching" is also used to estimate and shape demand. In this case, functionality is released, usually to a subset of the user base, but users are not notified that the functionality is available.

IT leaders must plan both the application and infrastructure architecture carefully. Infrastructure and product teams must work together to use application functionality, which allows an orderly degradation of service by reducing nonessential features and functions when necessary. Such techniques will allow limited IT resources to be shared among different applications, providing an acceptable user experience and keeping vital applications running in the event of any difficulties with IT infrastructure.

Become proficient in operational analytics tools and big data capabilities
The different architectures and the vastness of web-scale IT organisations make traditional capacity planning tools of limited utility. Demand shaping requires different functionality than current off-the-shelf tools provide, and different organisations will adapt different web-scale techniques to fit their requirements. However, the common theme is extensive use of large volumes of operational data to plan IT capacity.

In-memory computing and deep analytics tools are generally used to extract the required information directly from the infrastructure and from instrumentation built into applications. This information is used to inform real-time decisions to allocate resources and manage potential bottlenecks. Similar functionality is also used to model the impact of moving workloads and to simulate the effects of potential infrastructure and application changes.

"These operational skills and tools are currently unique to each web-scale organisation and are not yet available in most end-user organisations," said Mr Head. "However they will be in increasingly high demand as large organisations of all types begin to pursue the tangible business benefits of a web-scale approach to IT infrastructure."