The data-driven organization is within reach — if CIOs can tap the power of cloud

The amount of data businesses store in the cloud is growing 33% per year. Yet more than two-thirds of that information is never used.

This failure is undermining CIOs in critical ways. Just 24% say they’ve created a data-driven organization and only 29% say they’ve achieved transformational business outcomes, according to a recent survey of data executives by NewVantage Partners, a strategic advisory firm specializing in data strategy.

The message is clear: CIOs must make extracting more value from their companies’ data a strategic imperative. Organizations that take full advantage of cloud-based data are more likely to make faster, smarter decisions, improve customer and employee experiences, and drive growth, experts say.

“Companies need to be able to mine for and use much more intelligence than they do today,” says Sarbjeet Johal, a cloud strategy consultant who formerly led cloud architecture at Oracle, VMware, and other companies.

That path begins with tapping the full power of the cloud, Johal and other IT leaders say. To start, organizations must focus on a few areas that are key to  successful cloud infrastructure services — data integration, observability, search, and security. 

Modernize data integration, both in cloud and on-prem 

One of the first steps for organizations is data integration — consolidating data from all its disparate sources. Companies can’t even begin to put data to work when it’s spread across a hodgepodge of cloud and on-premises systems. It must be clearly organized and centralized.

“There’s no shortage of data pools today,” says Stephen Elliot, program vice president of management software and DevOps at IDC. “But there is a tremendous shortage when it comes to integrating those data sets and extracting business and technical advantage from it.”

Once data is consolidated, organizations can use advanced analytics tools to improve decision-making. Executives equipped with these tools ask better questions — about their operations, the experience of their customers, and their strategic outlook — and “choose the best course among available courses of action,” says Bart de Langhe, a behavioral scientist and associate professor of marketing at Spain’s Esade business and law school. 

Move from monitoring to observability

Even the best data-integration tools and practices can do only so much when data volumes are growing so quickly. 

Organizations need to know what kind of data they have and where it resides — in real time — in order to manage and derive meaningful insights from it.

That’s one reason why observability — the ability to continually measure performance of systems by tracking their outputs — has become a critical feature of enterprise cloud systems.

Observability tools go beyond traditional monitoring to show SRE, DevOps, and IT teams what’s going on and why problems occur within increasingly complex and dynamic infrastructure and application architectures, says Gagan Singh, vice president of product, observability, at Elastic.

Those data-driven insights, in turn, help teams improve business results. “Organizations that adopt the observability mindset gain new levels of agility and the ability to deliver bigger, better customer experiences,” Singh says.

Observability tools are also gaining AI capabilities and AI operations. New AIOps platforms use machine learning to collect, analyze, and utilize data, enabling enterprise apps to diagnose causes of system problems on the fly, reducing the time required for troubleshooting.

Gartner predicts that, by 2025, more than half of DevOps observability teams will augment their tools with AIOps, up from less than 15% in 2020.

Put cognitive search insight at the center

Even when all the data is integrated and observable, it can still be difficult to find important data no matter where it resides, so that workers can solve problems and identify opportunities.

That’s why search technology that can connect cloud systems and various data sources inside an enterprise, and can apply machine learning and natural language understanding to improve the discovery of data — an approach sometimes called cognitive search — is critical.

“Cognitive search enables companies to analyze and act upon far more data than we’ve been able to consider in the past,” says Mark Settle, a former CIO of Okta and now a business consultant and author.

Zero Trust your data

Cloud infrastructure services offer several security advantages over on-prem systems, but more importantly, they give CIOs and CISOs a way to rethink and modernize how they protect their organizations’ data.

With the cloud, outdated models of perimeter defense can be replaced with a Zero Trust approach, which treats every user, device, or data request as a potential threat that must be verified. Cloud-based security technology also enables a more advanced approach to cyber risks by assessing which ones pose the greatest threats to the company and its assets.

There’s little argument that migrating tech infrastructure to the cloud can deliver far-reaching benefits. But only with improved data integration, observability, search, and security can CIOs convert the insights into better business performance.

“Organizations must consider new strategies to better monitor, analyze, and protect data at scale,” Bragin says. “Once IT leaders deepen their view into their company’s infrastructure as a whole and break down data and tool silos, they will be on the true path to becoming data-driven organizations.”