IMPORTANT: No additional bug fixes or documentation updates will be released for this version. For the latest information, see the current release documentation.
« URL Decode Processor Getting started with index lifecycle management »
Elastic Docs ›Elasticsearch Guide [6.6]

Managing the index lifecycle

edit

Managing the index lifecycle

edit

This functionality is in beta and is subject to change. The design and code is less mature than official GA features and is being provided as-is with no warranties. Beta features are not subject to the support SLA of official GA features.

Index lifecycle management is currently not supported by leader indices and follower indices.

The index lifecycle management (ILM) APIs enable you to automate how you want to manage your indices over time. Rather than simply performing management actions on your indices on a set schedule, you can base actions on other factors such as shard size and performance requirements.

You control how indices are handled as they age by attaching a lifecycle policy to the index template used to create them. You can update the policy to modify the lifecycle of both new and existing indices.

For time series indices, there are four stages in the index lifecycle:

  • Hot—​the index is actively being updated and queried.
  • Warm—​the index is no longer being updated, but is still being queried.
  • Cold—​the index is no longer being updated and is seldom queried. The information still needs to be searchable, but it’s okay if those queries are slower.
  • Delete—​the index is no longer needed and can safely be deleted.

The lifecycle policy governs how the index transitions through these stages and the actions that are performed on the index at each stage. The policy can specify:

  • The maximum size or age at which you want to roll over to a new index.
  • The point at which the index is no longer being updated and the number of primary shards can be reduced.
  • When to force a merge to permanently delete documents marked for deletion.
  • The point at which the index can be moved to less performant hardware.
  • The point at which the availability is not as critical and the number of replicas can be reduced.
  • When the index can be safely deleted.

For example, if you are indexing metrics data from a fleet of ATMs into Elasticsearch, you might define a policy that says:

  1. When the index reaches 50GB, roll over to a new index.
  2. Move the old index into the warm stage, mark it read only, and shrink it down to a single shard.
  3. After 7 days, move the index into the cold stage and move it to less expensive hardware.
  4. Delete the index once the required 30 day retention period is reached.
« URL Decode Processor Getting started with index lifecycle management »