IMPORTANT: No additional bug fixes or documentation updates will be released for this version. For the latest information, see the current release documentation.
« Configure remote clusters with security Configure roles and users for remote clusters »
Elastic Docs ›Elasticsearch Guide [8.8] ›Set up Elasticsearch ›Remote clusters

Connect to remote clusters

edit

Connect to remote clusters

edit

Your local cluster uses the transport interface to establish communication with remote clusters. The coordinating nodes in the local cluster establish long-lived TCP connections with specific nodes in the remote cluster. Elasticsearch requires these connections to remain open, even if the connections are idle for an extended period.

You must have the manage cluster privilege to connect remote clusters.

To add a remote cluster from Stack Management in Kibana:

  1. Select Remote Clusters from the side navigation.
  2. Specify the Elasticsearch endpoint URL, or the IP address or host name of the remote cluster followed by the transport port (defaults to 9300). For example, cluster.es.eastus2.staging.azure.foundit.no:9400 or 192.168.1.1:9300.

Alternatively, use the cluster update settings API to add a remote cluster. You can also use this API to dynamically configure remote clusters for every node in the local cluster. To configure remote clusters on individual nodes in the local cluster, define static settings in elasticsearch.yml for each node.

After connecting remote clusters, configure roles and users for remote clusters.

The following request adds a remote cluster with an alias of cluster_one. This cluster alias is a unique identifier that represents the connection to the remote cluster and is used to distinguish between local and remote indices.

response = client.cluster.put_settings(
  body: {
    persistent: {
      cluster: {
        remote: {
          cluster_one: {
            seeds: [
              '127.0.0.1:9300'
            ]
          }
        }
      }
    }
  }
)
puts response
PUT /_cluster/settings
{
  "persistent" : {
    "cluster" : {
      "remote" : {
        "cluster_one" : {    
          "seeds" : [
            "127.0.0.1:9300" 
          ]
        }
      }
    }
  }
}

The cluster alias of this remote cluster is cluster_one.

Specifies the hostname and transport port of a seed node in the remote cluster.

You can use the remote cluster info API to verify that the local cluster is successfully connected to the remote cluster:

response = client.cluster.remote_info
puts response
GET /_remote/info

The API response indicates that the local cluster is connected to the remote cluster with the cluster alias cluster_one:

{
  "cluster_one" : {
    "seeds" : [
      "127.0.0.1:9300"
    ],
    "connected" : true,
    "num_nodes_connected" : 1,  
    "max_connections_per_cluster" : 3,
    "initial_connect_timeout" : "30s",
    "skip_unavailable" : false, 
    "mode" : "sniff"
  }
}

The number of nodes in the remote cluster the local cluster is connected to.

Indicates whether to skip the remote cluster if searched through cross-cluster search but no nodes are available.

Dynamically configure remote clusters

edit

Use the cluster update settings API to dynamically configure remote settings on every node in the cluster. The following request adds three remote clusters: cluster_one, cluster_two, and cluster_three.

The seeds parameter specifies the hostname and transport port (default 9300) of a seed node in the remote cluster.

The mode parameter determines the configured connection mode, which defaults to sniff. Because cluster_one doesn’t specify a mode, it uses the default. Both cluster_two and cluster_three explicitly use different modes.

response = client.cluster.put_settings(
  body: {
    persistent: {
      cluster: {
        remote: {
          cluster_one: {
            seeds: [
              '127.0.0.1:9300'
            ]
          },
          cluster_two: {
            mode: 'sniff',
            seeds: [
              '127.0.0.1:9301'
            ],
            "transport.compress": true,
            skip_unavailable: true
          },
          cluster_three: {
            mode: 'proxy',
            proxy_address: '127.0.0.1:9302'
          }
        }
      }
    }
  }
)
puts response
PUT _cluster/settings
{
  "persistent": {
    "cluster": {
      "remote": {
        "cluster_one": {
          "seeds": [
            "127.0.0.1:9300"
          ]
        },
        "cluster_two": {
          "mode": "sniff",
          "seeds": [
            "127.0.0.1:9301"
          ],
          "transport.compress": true,
          "skip_unavailable": true
        },
        "cluster_three": {
          "mode": "proxy",
          "proxy_address": "127.0.0.1:9302"
        }
      }
    }
  }
}

You can dynamically update settings for a remote cluster after the initial configuration. The following request updates the compression settings for cluster_two, and the compression and ping schedule settings for cluster_three.

When the compression or ping schedule settings change, all existing node connections must close and re-open, which can cause in-flight requests to fail.

response = client.cluster.put_settings(
  body: {
    persistent: {
      cluster: {
        remote: {
          cluster_two: {
            "transport.compress": false
          },
          cluster_three: {
            "transport.compress": true,
            "transport.ping_schedule": '60s'
          }
        }
      }
    }
  }
)
puts response
PUT _cluster/settings
{
  "persistent": {
    "cluster": {
      "remote": {
        "cluster_two": {
          "transport.compress": false
        },
        "cluster_three": {
          "transport.compress": true,
          "transport.ping_schedule": "60s"
        }
      }
    }
  }
}

You can delete a remote cluster from the cluster settings by passing null values for each remote cluster setting. The following request removes cluster_two from the cluster settings, leaving cluster_one and cluster_three intact:

response = client.cluster.put_settings(
  body: {
    persistent: {
      cluster: {
        remote: {
          cluster_two: {
            mode: nil,
            seeds: nil,
            skip_unavailable: nil,
            "transport.compress": nil
          }
        }
      }
    }
  }
)
puts response
PUT _cluster/settings
{
  "persistent": {
    "cluster": {
      "remote": {
        "cluster_two": {
          "mode": null,
          "seeds": null,
          "skip_unavailable": null,
          "transport.compress": null
        }
      }
    }
  }
}

Statically configure remote clusters

edit

If you specify settings in elasticsearch.yml, only the nodes with those settings can connect to the remote cluster and serve remote cluster requests.

Remote cluster settings that are specified using the cluster update settings API take precedence over settings that you specify in elasticsearch.yml for individual nodes.

In the following example, cluster_one, cluster_two, and cluster_three are arbitrary cluster aliases representing the connection to each cluster. These names are subsequently used to distinguish between local and remote indices.

cluster:
    remote:
        cluster_one:
            seeds: 127.0.0.1:9300
        cluster_two:
            mode: sniff
            seeds: 127.0.0.1:9301
            transport.compress: true      
            skip_unavailable: true        
        cluster_three:
            mode: proxy
            proxy_address: 127.0.0.1:9302

Compression is explicitly enabled for requests to cluster_two.

Disconnected remote clusters are optional for cluster_two.

The address for the proxy endpoint used to connect to cluster_three.
« Configure remote clusters with security Configure roles and users for remote clusters »