IMPORTANT: No additional bug fixes or documentation updates will be released for this version. For the latest information, see the current release documentation.
« Rollover index API Split index API »
Elastic Docs ›Elasticsearch Guide [7.3] ›REST APIs ›Index APIs

Shrink index API

edit

Shrink index API

edit

Shrinks an existing index into a new index with fewer primary shards.

POST /twitter/_shrink/shrunk-twitter-index

Request

edit

POST /<index>/_shrink/<target-index>

PUT /<index>/_shrink/<target-index>

Prerequisites

edit

Before you can shrink an index:

  • The index must be read-only.
  • A copy of every shard in the index must reside on the same node.
  • The cluster health status must be green.

These three conditions can be achieved with the following request:

PUT /my_source_index/_settings
{
  "settings": {
    "index.routing.allocation.require._name": "shrink_node_name", 
    "index.blocks.write": true 
  }
}

Forces the relocation of a copy of each shard to the node with name shrink_node_name. See Index-level shard allocation filtering for more options.

Prevents write operations to this index while still allowing metadata changes like deleting the index.

It can take a while to relocate the source index. Progress can be tracked with the _cat recovery API, or the cluster health API can be used to wait until all shards have relocated with the wait_for_no_relocating_shards parameter.

Description

edit

The shrink index API allows you to shrink an existing index into a new index with fewer primary shards. The requested number of primary shards in the target index must be a factor of the number of shards in the source index. For example an index with 8 primary shards can be shrunk into 4, 2 or 1 primary shards or an index with 15 primary shards can be shrunk into 5, 3 or 1. If the number of shards in the index is a prime number it can only be shrunk into a single primary shard. Before shrinking, a (primary or replica) copy of every shard in the index must be present on the same node.

How shrinking works

edit

A shrink operation:

  1. Creates a new target index with the same definition as the source index, but with a smaller number of primary shards.
  2. Hard-links segments from the source index into the target index. (If the file system doesn’t support hard-linking, then all segments are copied into the new index, which is a much more time consuming process. Also if using multiple data paths, shards on different data paths require a full copy of segment files if they are not on the same disk since hardlinks don’t work across disks)
  3. Recovers the target index as though it were a closed index which had just been re-opened.

Shrink an index

edit

To shrink my_source_index into a new index called my_target_index, issue the following request:

POST /my_source_index/_shrink/my_target_index
{
  "settings": {
    "index.routing.allocation.require._name": null, 
    "index.blocks.write": null 
  }
}

Clear the allocation requirement copied from the source index.

Clear the index write block copied from the source index.

The above request returns immediately once the target index has been added to the cluster state — it doesn’t wait for the shrink operation to start.

Indices can only be shrunk if they satisfy the following requirements:

  • the target index must not exist
  • The index must have more primary shards than the target index.
  • The number of primary shards in the target index must be a factor of the number of primary shards in the source index. The source index must have more primary shards than the target index.
  • The index must not contain more than 2,147,483,519 documents in total across all shards that will be shrunk into a single shard on the target index as this is the maximum number of docs that can fit into a single shard.
  • The node handling the shrink process must have sufficient free disk space to accommodate a second copy of the existing index.

The _shrink API is similar to the create index API and accepts settings and aliases parameters for the target index:

POST /my_source_index/_shrink/my_target_index
{
  "settings": {
    "index.number_of_replicas": 1,
    "index.number_of_shards": 1, 
    "index.codec": "best_compression" 
  },
  "aliases": {
    "my_search_indices": {}
  }
}

The number of shards in the target index. This must be a factor of the number of shards in the source index.

Best compression will only take affect when new writes are made to the index, such as when force-merging the shard to a single segment.

Mappings may not be specified in the _shrink request.

Monitor the shrink process

edit

The shrink process can be monitored with the _cat recovery API, or the cluster health API can be used to wait until all primary shards have been allocated by setting the wait_for_status parameter to yellow.

The _shrink API returns as soon as the target index has been added to the cluster state, before any shards have been allocated. At this point, all shards are in the state unassigned. If, for any reason, the target index can’t be allocated on the shrink node, its primary shard will remain unassigned until it can be allocated on that node.

Once the primary shard is allocated, it moves to state initializing, and the shrink process begins. When the shrink operation completes, the shard will become active. At that point, Elasticsearch will try to allocate any replicas and may decide to relocate the primary shard to another node.

Wait for active shards

edit

Because the shrink operation creates a new index to shrink the shards to, the wait for active shards setting on index creation applies to the shrink index action as well.

Path parameters

edit
<index>
(Required, string) Name of the source index to shrink.
<target-index>

(Required, string) Name of the target index to create.

Index names must meet the following criteria:

  • Lowercase only
  • Cannot include \, /, *, ?, ", <, >, |, ` ` (space character), ,, #
  • Indices prior to 7.0 could contain a colon (:), but that’s been deprecated and won’t be supported in 7.0+
  • Cannot start with -, _, +
  • Cannot be . or ..
  • Cannot be longer than 255 bytes (note it is bytes, so multi-byte characters will count towards the 255 limit faster)

Query parameters

edit
wait_for_active_shards

(Optional, string) The number of shard copies that must be active before proceeding with the operation. Set to all or any positive integer up to the total number of shards in the index (number_of_replicas+1). Default: 1, the primary shard.

See Active shards.

timeout
(Optional, time units) Specifies the period of time to wait for a response. If no response is received before the timeout expires, the request fails and returns an error. Defaults to 30s.
master_timeout
(Optional, time units) Specifies the period of time to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error. Defaults to 30s.

Request body

edit
aliases
(Optional, alias object) Index aliases which include the target index. See Update index alias.
settings
(Optional, index setting object) Configuration options for the target index. See Index Settings.
« Rollover index API Split index API »