The most vulnerable points in modern business operations are not where you think they are.
It’s not the scale and complexity of modern data architecture that makes business operations vulnerable. It’s the thousands of integrations in the data architecture that make the scale and complexity possible — and the fact that change is the new constant. That’s why the DataOps mindset is on the rise.
When One Small Change Causes A World Of Hurt
Here’s a perfect example. After my neighborhood bank merged with another bank, I had to memorize two additional digits to fill out a deposit slip. “But wait,” you say. “You went into a bank?”
Of course not. I don’t even know my bank account number or care that it changed. As a banking customer, I expect data to flow seamlessly and securely to the intelligent applications that I choose to manage my personal finances. Behind the scenes, there may be hundreds or even thousands of applications working automatically and without intervention to provide a seamless, secure customer experience, and somewhere there is a CTO managing an organization that keeps it running.
The Tyranny Of Intelligent Applications
A radical transformation has occurred over the past 10 years, and it touches every industry. Tiny, specialized applications have taken the place of monolithic software systems. As a result, the vast majority of business logic that drives the modern enterprise now resides in the integrations between those applications. Yet, most companies do not bring the same level of discipline to integrations as they do to application code. As a result, these integrations have become the most vulnerable points in modern business operations.
The transition from 10-digit to 12-digit account numbers at the bank I mentioned earlier affected 18,000 known applications. Would you like to be responsible for managing the data drift on that migration?
Data Drift And The Rise Of The DataOps Mindset
Businesses running on data are vulnerable to data drift: the unexpected, unannounced and unending changes to data infrastructure that disrupt data flow. From the rise of data centers until the move to the cloud, business analytics could rely on predictable datasets from expected sources that did not change. But as data sources have expanded and data silos break down, the data has become messy and complicated. A well-intentioned, even necessary change can have vast known and unknown consequences.
What is the answer? Lock down systems and fight data drift? That’s doing battle against innovation. Reducing the risk of vulnerability at these points of integration requires a DataOps mindset. We’ve defined DataOps this way: It’s a set of practices and technologies that operationalize data management and integration to ensure resiliency and agility in the face of constant change.
Think about it this way: When you drive a car down a highway, you focus on the driving, not what’s happening under the hood. That’s the benefit of a modern data infrastructure built for a DataOps discipline: You can keep your head up, scanning the horizon for changes in the environment. An occasional glance at the dashboard tells you everything you need to know about what’s happening under the hood.
Getting Started With DataOps
To take the analogy a step further, sensors with artificial intelligence are making cars better at noticing small changes in the environment and adjusting to them automatically. A reliable data fabric allows you to keep your focus on the particular mission of your business. Visibility at the dashboard level allows you to monitor the health of your pipelines in real time without the noise and complexity of tracking what works without intervention.
How can you get started with the implementation of DataOps? Begin here:
• Start with smart data pipelines that are drift-aware and drift-resilient and that decouple everything, so they are not vulnerable to change.
• Design, deploy and operate these smart data pipelines at scale, using a DataOps platform with automation and monitoring of live data flows.
• Give your applications space to grow and express themselves as they evolve.
• Build in instrumentation and drift handling to operate continuously in the face of change.
The DataOps mindset does more than address the complex challenges of today’s data infrastructure; it lays the foundation for future innovations — from machine learning and artificial intelligence to adoption of the latest cloud technologies — into your business practice.