Miscellaneous cluster settings

An entire cluster may be set to read-only with the following setting:

There is a limit on the number of shards in a cluster, based on the number of nodes in the cluster. This is intended to prevent a runaway process from creating too many shards which can harm performance and in extreme cases may destabilize your cluster.

If an operation, such as creating a new index, restoring a snapshot of an index, or opening a closed index would lead to the number of shards in the cluster going over this limit, the operation will fail with an error indicating the shard limit. To resolve this, either scale out your cluster by adding nodes, or delete some indices to bring the number of shards below the limit.

If a cluster is already over the limit, perhaps due to changes in node membership or setting changes, all operations that create or open indices will fail.

The cluster shard limit defaults to 1000 shards per non-frozen data node for normal (non-frozen) indices and 3000 shards per frozen data node for frozen indices. Both primary and replica shards of all open indices count toward the limit, including unassigned shards. For example, an open index with 5 primary shards and 2 replicas counts as 15 shards. Closed indices do not contribute to the shard count.

You can dynamically adjust the cluster shard limit with the following setting:

User-defined metadata can be stored and retrieved using the Cluster Settings API. This can be used to store arbitrary, infrequently-changing data about the cluster without the need to create an index to store it. This data may be stored using any key prefixed with cluster.metadata.. For example, to store the email address of the administrator of a cluster under the key cluster.metadata.administrator, issue this request:

resp = client.cluster.put_settings(
    body={
        "persistent": {
            "cluster.metadata.administrator": "sysadmin@example.com"
        }
    },
)
print(resp)

response = client.cluster.put_settings(
  body: {
    persistent: {
      'cluster.metadata.administrator' => 'sysadmin@example.com'
    }
  }
)
puts response

PUT /_cluster/settings
{
  "persistent": {
    "cluster.metadata.administrator": "sysadmin@example.com"
  }
}

The cluster state maintains index tombstones to explicitly denote indices that have been deleted. The number of tombstones maintained in the cluster state is controlled by the following setting:

If Elasticsearch encounters index data that is absent from the current cluster state, those indices are considered to be dangling. For example, this can happen if you delete more than cluster.indices.tombstones.size indices while an Elasticsearch node is offline.

You can use the Dangling indices API to manage this situation.

The settings which control logging can be updated dynamically with the logger. prefix. For instance, to increase the logging level of the indices.recovery module to DEBUG, issue this request:

resp = client.cluster.put_settings(
    body={
        "persistent": {
            "logger.org.elasticsearch.indices.recovery": "DEBUG"
        }
    },
)
print(resp)

response = client.cluster.put_settings(
  body: {
    persistent: {
      'logger.org.elasticsearch.indices.recovery' => 'DEBUG'
    }
  }
)
puts response

PUT /_cluster/settings
{
  "persistent": {
    "logger.org.elasticsearch.indices.recovery": "DEBUG"
  }
}

Plugins can create a kind of tasks called persistent tasks. Those tasks are usually long-lived tasks and are stored in the cluster state, allowing the tasks to be revived after a full cluster restart.

Every time a persistent task is created, the master node takes care of assigning the task to a node of the cluster, and the assigned node will then pick up the task and execute it locally. The process of assigning persistent tasks to nodes is controlled by the following settings: