Revolutionising storage management

Q In general terms, what are the drivers behind the trend

towards software-defined ‘everything’?
A There is a degree of hardware lock in that has always been resisted but for the most part software defined “everything” is more concerned with removing the barriers and simplifying management.
I don’t want to downplay the advances in hardware technology, in fact without some of the memory and CPU developments we’d not be in a position to run this advanced software on commodity hardware.
When you look at the majority of storage vendors they are without a doubt accomplished software development houses. They do software very well and typically outsource the hardware build or OEM existing hardware. I doubt they want to fix power supplies or repair disk drives…
Q For how long has software defined storage existed, and is it
a general storage networking industry trend, or only being
offered by certain vendors?
A It depends on your interpretation of software defined. We’ve had storage virtualisation appliances for some time but when you look under the covers of popular storage platforms you’ll notice a common thread; commodity hardware with almost all of the intellectual property existing in the software. What we’ve been missing up until now has been the automation and policy driven approach to delivering storage to the enterprise.
Q Is there a commonly accepted definition of software defined
storage, or are there very different, but equally valid, interpretations?
A There are many different interpretations of software defined storage or SDS, especially from vendors who might stretch the definition to be more applicable to their offering. In the true sense of a definition, software defined storage is where the software is separate from the hardware it is managing. Of course the hardware may have some built in features such as automation or an abstraction layer but the management software is a separate entity.
Q How does software defined storage compare to traditional
SAN/NAS storage?
A Traditional SAN and NAS products typically have their own, what I would call, premium features. These could be replication, snapshots, tiering and management tools but they are usually specific to the product itself. All that so called “intelligence” costs money and not all customers need all the options. The systems themselves have processor and memory resources to ensure they can deliver the storage services a customer might need regardless of whether they are actually used. These systems are managed in groups, from the same manufacturer and can develop into silos of storage.
Software defined storage takes most of the “intelligence” out of the traditional storage system and keeps it in the software that overlays the storage components. It removes the silo problem in traditional storage allowing storage to be grouped and classified in ways that are much more relevant to your business. Eventually this will build in to a catalog approach where the application owner requests more storage resources and the software delivers the required space from a predefined policy greatly simplifying the steps and tasks while ensuring the resources are appropriate to the application.
Q What are the advantages/disadvantages of being able to use
commodity hardware with software defined storage, as opposed
to the traditional customised storage/server hardware?
A Most people hear commodity and think low cost but there is much more to consider. Okay, low cost is a good start but if the acquisition cost is low and the operating cost is high there is little to be saved in the long term. A poorly conceived system can cause countless problems and understanding the requirements and limitations is just as important in commodity hardware based solutions as it is in traditional storage systems.
I believe commodity hardware is just the start but we are going to see a shift to appliance based solutions where the commodity hardware is replaced with a vendors own commodity hardware offering with the reassurance of quality and control.
Q Software defined storage offers much in the way of agility?
A Absolutely, agility and control are two major benefits to software defined storage. If the intelligence is in the software and it’s built from the ground up as a software defined system then it should be able to grow and shrink as the customer’s business dictates. As the whole point is the ability to use commodity hardware, the system should allow the customer to deploy new storage capacity from one vendor while retiring old storage from another.
Q Similarly, scalability is a major part of the software defined
storage’s attraction?
A Scalability and flexibility have combined into a new buzzword; elasticity. 6TB disk drives are here and this brings with it greater density than ever before. Performance with SSD devices has gone through the roof too. Being able to use the latest and greatest technology where you need at a fraction of the traditional monolithic storage system costs is a big attraction.
We can achieve compelling storage densities and efficiencies in commodity based hardware but it means nothing without a reliable software layer managing the feature set.
Q As is reliability?
A There is a perception that the convergence of products from different vendors will lead to complications and put reliability at risk. Software defined storage relies on tight interoperability testing, well in most cases. The lower cost hardware can enable a customer to build in more protection levels and create protection policies for specific data types without blowing the budget.
Q How does software defined storage help with the Cloud/multi-
tenancy environment?
A As a service provided it would seem to make perfect sense. If you want to offer self-service storage products to your customer base then software defined storage is the first step. Don’t forget that the cloud / multi-tenancy market is very aggressive with cost and functionality being the key differentiators.
This goes back to the agility and scalability points already made. As a service provider you would have the freedom to choose the most cost correct commodity hardware and build a policy approach to delivering the storage to the end user.
Q What about Big Data, does software defined storage have a
contribution to make here?
A To a degree but it depends on what the SDS vendor is offering. In a big data environment where we are managing petabytes, software defined storage is one step on the road to the software defined datacentre.
With a true orchestration layer between IT services and the applications we can utilise different storage types for the correct applications and even use a policy based catalog to determine the compute and networking requirements and build a fully automated datacentre.
Q Does software defined storage remove all of the intelligence
from the storage hardware?
A Not necessarily as there will always be specific technologies that exist in hardware or productised software alone. We are likely to see commodity hardware replace midrange and scale-out storage systems in the long term but the Tier 0 and Tier 3 storage vendors always offer some type of enhanced technology embedded in their solutions that will remain.
Q Can you tell us a little bit about how virtualisation fits in the
software defined storage world – indeed, did it help to create it?
A Virtualisation could be considered the forerunner to all software defined datacentre products. We’ve virtualised servers, we’ve virtualised the networks so let’s virtualise the storage and put it all together in a virtual datacentre and define it all in software so that repetitive tasks can be automated and human error can be reduced. Say hello to the future and the future is now.
Q Where do the virtual volumes, being offered by VMware for
example, fit in the software defined storage landscape?
A VMware virtual volumes rely on the VASA APIs to enable the storage system to become aware of the virtual volumes and their associations with the virtual machines. This visibility can only be a good thing from a storage perspective and within a software defined storage environment enables the correct management of the virtual volumes.
To use these API enhancements previously you would need a storage system that had been certified by VMware and that was on the hardware compatibility list (HCL). With software defined storage we only need to worry about the software layer. The theory is that any commodity storage hardware could benefit and we wouldn’t need to worry about the HCL provided the software stack was certified.
Q Clustered file systems are also a part of software defined storage?
A We should expect that all storage types should be part of the software defined storage ecosystem. Most of the well-known clustered file systems are primarily software products first and foremost. Almost all of the clustered file vendors uses commodity LINUX servers already, it’s the next logical step.
Q How does software defined storage work with the Fibre Channel
and iSCSI storage networking protocols?
A That’s a bit of a red herring and a common question. The protocols don’t change it’s where the features reside. If you need Fibre Channel and or iSCSI then that’s just fine. Fibre channel might impose some inflexibility in the physical layer but it’s not a show stopper
Q Software defined storage may well bring about significant
IT automation, but it will never remove the need for humans to
make those initial policy decisions?
A We need to build the policies and understand the environment before we can work on the automation. Like any new technology there is some upfront investment in time and resources but these should be paid back tenfold in future efficiencies.

Even in well-established disaster recovery solutions they need a person to make the final decision to invoke the disaster recovery plan.
I’m not sure mapping a LUN or resizing a volume is the most rewarding of tasks and with datacentre automation we can let those administrators move on to something more fulfilling.
Q Is it fair to suggest, therefore, that software defined storage can
be ‘dangerous’ in the wrong hands?
A Just because its software doesn’t make it fool proof. We are building a software defined storage system that has storage, connectivity and maintenance to consider – just as in the physical world. Architecture needs to be right and needs to cater for the scalability and security the customer demands. Get it wrong at the beginning and pay for it at the end.
Q How do you see the storage hardware vendors adapting to life
where their embedded/integral software offerings are being
replaced by external, software defined storage solutions?
A We are seeing some of the storage vendors on the software defined express train and making good progress. Hardware vendor lock in will be a thing of the past with customers having the freedom to choose what they want, when they want it. There will always be a market for traditional storage systems for those customers who are too critical to embrace SDS or too small to afford it
Q Indeed, does the use of software defined storage guarantee an
open storage architecture, or are the hardware vendors
reluctant to give up their proprietary ‘edge’?
A The cynic in me would say that you’ll be locked in anyway. Without doubt the storage hardware choices will broaden significantly. If you are not locked into the hardware then you’ll be locked in to the software but is it such a bad thing.
The vendors that get criticised for “lock in” invest considerable funds into research and development so they need to see a return. We want to see the next great feature so we need to be on board to help fund it. It doesn’t make it a bad thing.
Q And how is S3 helping its customers come to terms with the
possibilities of software defined storage?
A Well our view is to get close to the customer and understand what they are trying to achieve. You’d be amazed how many organisation do not have an accurate picture of what they actually have today. S3 helps them understand where they are and listens to where they want to be.
This is when the magic happens – our presales architects spring
into action and work out the best options to get to between the two.
If agility and elasticity are key business drivers then maybe the
answer is SDS. We show our customers how the advances in technology can benefit their business, save them valuable time
and reduce costs.
Q In conclusion, is software defined storage here to stay, or just
another step in the cyclical nature of the storage networking
industry, so the software will eventually go back in the box?!
A I’ve been the technical lead at S3 for over 20 years, we’ve seen a lot of new technologies come and go.
Storage was direct attached to the server and then we centralised it into disk arrays. Now we are going to break up the disk arrays and distribute the data into storage servers.
Storage is no different to other IT resources, we are constantly thinking up new ways to make them better but the best doesn’t always prevail. Without a doubt the benefits of software defined storage are clear to see. Some of our customers are managing 10s of Petabytes, something that was impossible 10 years ago.
If we position it correctly and deliver it properly SDS will revolutionise the way we manage storage.