If you could capture the IT transformation of the past decade in one big idea, it would have to be “infrastructure abstraction.” Or, put simply, separating the stuff you want to do from the IT infrastructure needed to do it. That is, after all, the idea behind cloud computing. Who wants to worry about buying and managing a bunch of hardware to run your business applications? When you can abstract an application from the specific servers in your data center, you can now focus on what it does, instead of how it runs.
The same story holds true for networks. With software-defined networking (SDN), your instructions to infrastructure become declarative (“Connect to this thing I need”) instead of procedural (“Follow this list of 25 steps to establish connectivity”).
But, if IT infrastructure is a three-legged stool of compute, network and storage, what about that third leg? Unfortunately, dealing with storage is just as arcane and procedural as ever. If anything, it’s even more complicated. As organizations collect more and more data, storage has become massively complex. High-performance storage for hot data. Object storage is for cold. Multisite storage, containerized storage and more --there's an ever-growing list of specialized protocols and infrastructures across public, private and hybrid clouds.
The problem should be obvious: Even as data becomes more central to every business, we’re still relying on an infrastructure-first approach to organize and work with it. When will real abstraction come to storage? When will we be able to think about data as just data and let the underlying infrastructure figure itself out, like with every other resource in the cloud? The answer is now. For the first time, companies are starting to unshackle their data from the infrastructure where it lives and implement real data agility.
Wrestling With Infrastructure
It’s not that no one has tried to virtualize storage. It’s just that most efforts have focused on making it easier to manipulate different kinds of storage infrastructures. That is, they still think about data in terms of the container it happens to be in, rather than the data itself.
This infrastructure-first approach is so ingrained that we don’t even realize it. But it has real consequences for businesses. For starters, it makes it far more difficult for different stakeholders to create value from an organization’s data. Data scientists, marketing research analysts, healthcare providers -- name the user -- all have different motives and agendas for the data they’re working with. But instead of freeing them to focus on the innovative things they’d like to do with data, we force them to constantly think in terms of the constructs and restrictions of the company’s storage hardware.
Enter Data As A Service
If you’re going to transform your organization into a modern digital business, your data needs to be far more agile. It should just be there for the people who need it, whenever and wherever they need it, without users having to speak the language of storage hardware. In short, it should be consumable as a service, just like everything else in the cloud.
This concept of cloudlike data as a service implies a layer of abstraction between your data and the storage it happens to live on at any given moment. Among other things, it requires two big capabilities:
- Descriptive Metadata: Details about what the data is and what it needs (such as security, compliance or other requirements) should be attached to the data itself.
- Machine Intelligence: This determines where and how to store data based on that metadata -- and automatically makes it accessible to any user, application or service that needs it
Users should no longer be forced to think about data in terms of hardware requirements. Instead, AI/ML should handle all the complexity, making sure data is stored in the right place, at the right price, without anyone having to explicitly dictate how. For data consumers, the data is just there -- whenever and wherever they need it.
Data As A Service In Action
Under today’s infrastructure-first approach to data, different stakeholders within an organization often have competing interests. Business users under pressure to turn data into value get bogged down by the hoops they have to jump through to work with the data they need. IT, which is trying to keep costs down, must constantly make new copies of data (and store it and manage it) to meet user demands. Meanwhile, compliance and security feel they’re in a losing race to enforce proper data governance and prevent mistakes that could prove hugely costly to the business. Everybody loses -- especially the business at large, which needs to use its data to transform and evolve.
Once you’ve virtualized your data and can consume it as a service, however, those clashing interests largely disappear. Business users can now self-service their data access, without having to ask IT to move or copy it. And, with properly managed metadata, requirements for access control, compliance, security and governance are now attached to the data itself and handled automatically, instead of by overburdened human beings.
Journey Before Destination
Moving to an “as a service” model for how data is consumed can feel overly aspirational for IT managers who have huge investments in existing infrastructure and adopt technologies iteratively. Some straightforward advice I can give is to think beyond the “cloud-first” mandate of moving to the cloud. The new lens through which we will evaluate our future capabilities of data orchestration will be agility, control and efficiency, but those concepts can be mapped to the traditional criteria of performance, cost and manageability. Keeping this vector in mind can help influence the decisions we make today.
We know that businesses need to transform for the modern digital world. And we know that data is the No. 1 resource they need to do it. But, as more businesses are discovering, just collecting data is not enough. Just building an analytics and data science capability is not enough. There’s just too much data, and more coming all the time -- 463 exabytes created every day, according to the World Economic Forum. It’s time for something different.
