Search across clusters

Cross-cluster search lets you run a single search request against one or more remote clusters. For example, you can use a cross-cluster search to filter and analyze log data stored on clusters in different data centers.

The following APIs support cross-cluster search:

The following cluster update settings API request adds three remote clusters: cluster_one, cluster_two, and cluster_three.

PUT _cluster/settings
{
  "persistent": {
    "cluster": {
      "remote": {
        "cluster_one": {
          "seeds": [
            "127.0.0.1:9300"
          ]
        },
        "cluster_two": {
          "seeds": [
            "127.0.0.1:9301"
          ]
        },
        "cluster_three": {
          "seeds": [
            "127.0.0.1:9302"
          ]
        }
      }
    }
  }
}

In the search request, you specify data streams and indices on a remote cluster as <remote_cluster_name>:<target>.

The following search API request searches the my-index-000001 index on a single remote cluster, cluster_one.

response = client.search(
  index: 'cluster_one:my-index-000001',
  body: {
    query: {
      match: {
        "user.id": 'kimchy'
      }
    },
    _source: [
      'user.id',
      'message',
      'http.response.status_code'
    ]
  }
)
puts response

GET /cluster_one:my-index-000001/_search
{
  "query": {
    "match": {
      "user.id": "kimchy"
    }
  },
  "_source": ["user.id", "message", "http.response.status_code"]
}

The API returns the following response:

{
  "took": 150,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "failed": 0,
    "skipped": 0
  },
  "_clusters": {
    "total": 1,
    "successful": 1,
    "skipped": 0
  },
  "hits": {
    "total" : {
        "value": 1,
        "relation": "eq"
    },
    "max_score": 1,
    "hits": [
      {
        "_index": "cluster_one:my-index-000001", 
        "_id": "0",
        "_score": 1,
        "_source": {
          "user": {
            "id": "kimchy"
          },
          "message": "GET /search HTTP/1.1 200 1070000",
          "http": {
            "response":
              {
                "status_code": 200
              }
          }
        }
      }
    ]
  }
}

The following search API request searches the my-index-000001 index on three clusters:

response = client.search(
  index: 'my-index-000001,cluster_one:my-index-000001,cluster_two:my-index-000001',
  body: {
    query: {
      match: {
        "user.id": 'kimchy'
      }
    },
    _source: [
      'user.id',
      'message',
      'http.response.status_code'
    ]
  }
)
puts response

GET /my-index-000001,cluster_one:my-index-000001,cluster_two:my-index-000001/_search
{
  "query": {
    "match": {
      "user.id": "kimchy"
    }
  },
  "_source": ["user.id", "message", "http.response.status_code"]
}

The API returns the following response:

{
  "took": 150,
  "timed_out": false,
  "num_reduce_phases": 4,
  "_shards": {
    "total": 3,
    "successful": 3,
    "failed": 0,
    "skipped": 0
  },
  "_clusters": {
    "total": 3,
    "successful": 3,
    "skipped": 0
  },
  "hits": {
    "total" : {
        "value": 3,
        "relation": "eq"
    },
    "max_score": 1,
    "hits": [
      {
        "_index": "my-index-000001", 
        "_id": "0",
        "_score": 2,
        "_source": {
          "user": {
            "id": "kimchy"
          },
          "message": "GET /search HTTP/1.1 200 1070000",
          "http": {
            "response":
              {
                "status_code": 200
              }
          }
        }
      },
      {
        "_index": "cluster_one:my-index-000001", 
        "_id": "0",
        "_score": 1,
        "_source": {
          "user": {
            "id": "kimchy"
          },
          "message": "GET /search HTTP/1.1 200 1070000",
          "http": {
            "response":
              {
                "status_code": 200
              }
          }
        }
      },
      {
        "_index": "cluster_two:my-index-000001", 
        "_id": "0",
        "_score": 1,
        "_source": {
          "user": {
            "id": "kimchy"
          },
          "message": "GET /search HTTP/1.1 200 1070000",
          "http": {
            "response":
              {
                "status_code": 200
              }
          }
        }
      }
    ]
  }
}

By default, a cross-cluster search fails if a remote cluster in the request returns an error or is unavailable. Use the skip_unavailable cluster setting to mark a specific remote cluster as optional for cross-cluster search.

If skip_unavailable is true, a cross-cluster search:

The following cluster update settings API request changes cluster_two's skip_unavailable setting to true.

PUT _cluster/settings
{
  "persistent": {
    "cluster.remote.cluster_two.skip_unavailable": true
  }
}

If cluster_two is disconnected or unavailable during a cross-cluster search, Elasticsearch won’t include matching documents from that cluster in the final results.

Because cross-cluster search involves sending requests to remote clusters, any network delays can impact search speed. To avoid slow searches, cross-cluster search offers two options for handling network delays:

Here’s how cross-cluster search works when you minimize network roundtrips.

Here’s how cross-cluster search works when you don’t minimize network roundtrips.

In 8.0+, Elastic supports searches from a local cluster to a remote cluster running:

Elastic also supports searches from a local cluster running the last minor version of a major version to a remote cluster running any minor version in the following major version. For example, a local 7.17 cluster can search any remote 8.x cluster.

For example, a local 8.0 cluster can search a remote 7.17 or any remote 8.x cluster. However, a search from a local 8.0 cluster to a remote 7.16 or 6.8 cluster is not supported.

Only features that exist across all searched clusters are supported. Using a feature with a remote cluster where the feature is not supported will result in undefined behavior.

A cross-cluster search using an unsupported configuration may still work. However, such searches aren’t tested by Elastic, and their behavior isn’t guaranteed.

The simplest way to ensure your clusters support cross-cluster search is to keep each cluster on the same version of Elasticsearch. If you need to maintain clusters with different versions, you can:

You can still search a remote cluster while performing a rolling upgrade on the local cluster. However, the local coordinating node’s "upgrade from" and "upgrade to" version must be compatible with the remote cluster’s gateway node.

For more information about upgrades, see Upgrading Elasticsearch.