For the US Navy, digital keel should start with search

While we understand the intent behind the goal of being able to securely move data wherever it needs to go, moving data can come with risk and complexity. When you duplicate data or move it to a central repository, like a data lake, so that you can then access it from anywhere, you run the risk of data getting lost, in the wrong hands, and paying high and often duplicative data storage and transfer costs. The time and bandwidth required to operate a centralized data store quickly becomes untenable when dealing with the large volumes of operational and mission data that the DON uses on a daily basis. Operating in a DDIL environment can also make it riskier and more difficult to centralize your data.

The Elasticsearch® platform allows data to remain in its original point of collection, yet enables you to search across all data to find the exact information you need for your mission. Elasticsearch’s capability to search across environments is called Cross-Cluster Search (CCS). Without the need to duplicate or move data, this distributed approach — also known as a global data mesh — can significantly save on the costs of data transfer and storage, as only the query and its matching results would need to traverse the network.  

>> Read more about distributed search

Leveraging distributed data and storage, using a global data mesh approach offers several benefits for the DON, including:

1. Ownership: Instead of relinquishing control to a centralized team or external partners, in a data mesh model, the people who are responsible for and understand the nuances of the specific data and its context are in control of its use and distribution. 

2. Data democratization: By implementing a data mesh, the DON can democratize access to data within teams, enabling faster, more-informed decision-making processes. Unlike with a centralized approach, gleaning insights from data is not limited to those with specialized data science expertise. 

3. Accuracy: Information remains at the source without needing to be correlated or translated, which promotes proactive data cleansing, validation, and monitoring practices, resulting in improved data accuracy and trustworthiness. 

4. Fast decision-making: Accessing data in-situ is also inherently faster, which makes data more relevant, accurate, and valuable. 

5. Flexibility: Individual teams and cross-functional areas have the flexibility to use, collaborate, and maneuver their data based on their mission-specific needs. 

6. Comprehensive view: A data mesh layer unifies disparate data and provides a single source of truth that reduces reliance on incomplete viewpoints from disconnected systems and teams. 

7. Simplified planning and training: A data mesh approach empowers various departments and units to take ownership of data. This decentralization promotes data accountability and encourages subject matter experts to manage and govern their data effectively. And having a unified platform to acquire, learn, and deploy simplifies planning, reduces training time, and bridges the data aptitude gap.

>> Read more about how a global data mesh can accelerate missions

The capstone design project lists two primary outcomes for the DON: operational resilience and customer experience. The Elasticsearch platform can help the DON accelerate these outcomes through our distributed approach as well as our search-powered technical capabilities. By unifying your tech solutions around your data capabilities, you ensure that accurate, comprehensive, and accessible data is the foundation of any mission you embark on. 

Elasticsearch capabilities such as ESRE, SIEM, and APM are all built on Elasticsearch and therefore benefit from the platform’s ability to ingest, search, and store all data. As a result, you can trust that you’re getting the most accurate, up-to-date, and complete information, no matter what goal you’re striving to achieve.

Operational resilience is crucial for Navy missions, especially in DDIL and air-gapped environments. When operating in high latency or disconnected environments, a global data mesh approach is a benefit. Distributed architectures split the load across multiple servers or systems, which reduces reliance on a single, centralized data store or data lake — while at the same time facilitating data access at speed and scale. Even if a server or data center experiences an issue, the others can continue to operate.

A data store built with geo-distributed disaster recovery logic can further boost operational resilience. Disaster recovery strategies ensure that operations can continue even in the event of significant system failures or disasters, usually by maintaining backup systems or data centers in geographically diverse locations. In the event of a significant system failure or disaster in one region, operations can be quickly shifted to another location, minimizing downtime and data loss.

The Cross-Cluster Replication (CCR) capability in the Elasticsearch platform can be a key facilitator of operational resilience. CCR allows you to replicate certain data, or identify what data would need to be replicated if needed — reducing replication of entire databases. This functionality is especially valuable when co-locating data to and from the edge. 

CCR enables a variety of mission-critical use cases, such as data recovery and high availability — enabling teams to withstand an outage in a region or data center. It also allows you to colocate data to and from the edge, allowing data to be closer to the user or application server, which reduces latencies that could affect your mission. CCR can also be used to replicate data from a large number of smaller clusters back to a centralized reporting cluster. This is useful when it may not be efficient to query across a large network.

>> Read more about operational resilience

The sheer amount of data used and produced by the DON today poses a significant challenge when teams need to find and access specific information quickly. That challenge is compounded by the fact that data is most often stored in different formats (e.g., images vs. documents vs. maps) and environments (e.g., cloud, on-prem). Not only that, but there are also significant variations in where data is collected, from ship to store, as well as data storage environments ranging from the C4I community to the enterprise. This inevitably leads to information silos, lack of interoperability, and generally complex environments. 

When all your data is accessible via the same query, it’s easier for your teams to find exactly what they need, when and where they need it. Not having to painstakingly dig through mountains of internal documents and spreadsheets and correlate the information from different sources and teams not only saves time, but it also allows professionals to focus their attention on more strategic mission-focused work. Using Elasticsearch as a unified platform to bring together disparate data sources and data sets also dramatically reduces data silos and makes it easier for teams to collaborate with one another, whether at sea or on land.

By leveraging Elasticsearch as a global data mesh that enables you to access mission-critical data wherever it resides, you also unlock the ability to utilize the Elasticsearch platform for additional use cases, such as application performance monitoring, threat hunting, real-time situational awareness, and generative AI.

In an information battlefield where data is both a weapon and a strategic asset, it’s crucial that data of all types and in all locations is easily accessible, findable, and actionable, especially in air-gapped and DDIL environments. We at Elastic stand at the ready to continue to support and partner with the DON to provide a seamless customer experience and top-of-the-line operational resilience.

To talk to an expert directly, contact us.