IMPORTANT: No additional bug fixes or documentation updates will be released for this version. For the latest information, see the current release documentation.
« Apply a policy to a create index request Skipping Rollover »
Elastic Docs ›Elasticsearch Guide [7.4] ›Managing the index lifecycle

Using policies to manage index rollover

edit

Using policies to manage index rollover

edit

The rollover action enables you to automatically roll over to a new index based on the index size, document count, or age. When a rollover is triggered, a new index is created, the write alias is updated to point to the new index, and all subsequent updates are written to the new index.

Rolling over to a new index based on size, document count, or age is preferable to time-based rollovers. Rolling over at an arbitrary time often results in many small indices, which can have a negative impact on performance and resource usage.

You control when the rollover action is triggered by specifying one or more rollover parameters. The rollover is performed once any of the criteria are met. Because the criteria are checked periodically, the index might grow slightly beyond the specified threshold. To control how often the criteria are checked, specify the indices.lifecycle.poll_interval cluster setting.

New indices created via rollover will not automatically inherit the policy used by the old index, and will not use any policy by default. Therefore, it is highly recommended to apply the policy via index template, including a Rollover alias setting, for your indices which specifies the policy you wish to use for each new index.

The rollover action takes the following parameters:

Table 71. rollover Action Parameters

Name Description

max_size

The maximum estimated size the primary shard of the index is allowed to grow to. Defaults to null. Optional.

max_docs

The maximum number of document the index should contain. Defaults to null. Optional.

max_age

The maximum age of the index. Defaults to null. Optional.

These parameters are used to determine when the index is considered "full" and a rollover should be performed. Where multiple criteria are defined the rollover operation will be performed once any of the criteria are met.

The following request defines a policy with a rollover action that triggers when the index size reaches 25GB. The old index is subsequently deleted after 30 days.

Once an index rolls over, index lifecycle management uses the timestamp of the rollover operation rather than the index creation time to evaluate when to move the index to the next phase. For indices that have rolled over, the min_age criteria specified for a phase is relative to the rollover time for indices. In this example, that means the index will be deleted 30 days after rollover, not 30 days from when the index was created.

PUT /_ilm/policy/my_policy
{
  "policy": {
    "phases": {
      "hot": {
        "actions": {
          "rollover": {
            "max_size": "25GB"
          }
        }
      },
      "delete": {
        "min_age": "30d",
        "actions": {
          "delete": {}
        }
      }
    }
  }
}

To use an index lifecycle management policy, you need to specify it in the index template used to create the indices. For example, the following template associates my_policy with indices created from the template my_template.

PUT _template/my_template
{
  "index_patterns": ["test-*"], 
  "settings": {
    "number_of_shards": 1,
    "number_of_replicas": 1,
    "index.lifecycle.name": "my_policy", 
    "index.lifecycle.rollover_alias": "test-alias" 
  }
}

Template applies to all indices with the prefix test-

Associates my_policy with all indices created with this template

Rolls over the write alias test when the rollover action is triggered

To be able to start using the policy for these test-* indexes we need to bootstrap the process by creating the first index.

PUT test-000001 
{
  "aliases": {
    "test-alias":{
      "is_write_index": true 
    }
  }
}

Creates the index called test-000001. The rollover action increments the suffix number for each subsequent index.

Designates this index as the write index for this alias.

When the rollover is performed, the newly-created index is set as the write index for the rolled over alias. Documents sent to the alias are indexed into the new index, enabling indexing to continue uninterrupted.

« Apply a policy to a create index request Skipping Rollover »