elasticsearch-shard
editelasticsearch-shard
editIn some cases the Lucene index or translog of a shard copy can become
corrupted. The elasticsearch-shard
command enables you to remove corrupted
parts of the shard if a good copy of the shard cannot be recovered
automatically or restored from backup.
You will lose the corrupted data when you run elasticsearch-shard
. This tool
should only be used as a last resort if there is no way to recover from another
copy of the shard or restore a snapshot.
When Elasticsearch detects that a shard’s data is corrupted, it fails that
shard copy and refuses to use it. Under normal conditions, the shard is
automatically recovered from another copy. If no good copy of the shard is
available and you cannot restore from backup, you can use elasticsearch-shard
to remove the corrupted data and restore access to any remaining data in
unaffected segments.
Stop Elasticsearch before running elasticsearch-shard
.
To remove corrupted shard data use the remove-corrupted-data
subcommand.
There are two ways to specify the path:
-
Specify the index name and shard name with the
--index
and--shard-id
options. -
Use the
--dir
option to specify the full path to the corrupted index or translog files.
Removing corrupted data
editelasticsearch-shard
analyses the shard copy and provides an overview of the
corruption found. To proceed you must then confirm that you want to remove the
corrupted data.
Back up your data before running elasticsearch-shard
. This is a destructive
operation that removes corrupted data from the shard.
$ bin/elasticsearch-shard remove-corrupted-data --index twitter --shard-id 0 WARNING: Elasticsearch MUST be stopped before running this tool. Please make a complete backup of your index before using this tool. Opening Lucene index at /var/lib/elasticsearchdata/nodes/0/indices/P45vf_YQRhqjfwLMUvSqDw/0/index/ >> Lucene index is corrupted at /var/lib/elasticsearchdata/nodes/0/indices/P45vf_YQRhqjfwLMUvSqDw/0/index/ Opening translog at /var/lib/elasticsearchdata/nodes/0/indices/P45vf_YQRhqjfwLMUvSqDw/0/translog/ >> Translog is clean at /var/lib/elasticsearchdata/nodes/0/indices/P45vf_YQRhqjfwLMUvSqDw/0/translog/ Corrupted Lucene index segments found - 32 documents will be lost. WARNING: YOU WILL LOSE DATA. Continue and remove docs from the index ? Y WARNING: 1 broken segments (containing 32 documents) detected Took 0.056 sec total. Writing... OK Wrote new segments file "segments_c" Marking index with the new history uuid : 0pIBd9VTSOeMfzYT6p0AsA Changing allocation id V8QXk-QXSZinZMT-NvEq4w to tjm9Ve6uTBewVFAlfUMWjA You should run the following command to allocate this shard: POST /_cluster/reroute { "commands" : [ { "allocate_stale_primary" : { "index" : "index42", "shard" : 0, "node" : "II47uXW2QvqzHBnMcl2o_Q", "accept_data_loss" : false } } ] } You must accept the possibility of data loss by changing parameter `accept_data_loss` to `true`. Deleted corrupt marker corrupted_FzTSBSuxT7i3Tls_TgwEag from /var/lib/elasticsearchdata/nodes/0/indices/P45vf_YQRhqjfwLMUvSqDw/0/index/
When you use elasticsearch-shard
to drop the corrupted data, the shard’s
allocation ID changes. After restarting the node, you must use the
cluster reroute API to tell Elasticsearch to use the new
ID. The elasticsearch-shard
command shows the request that
you need to submit.
You can also use the -h
option to get a list of all options and parameters
that the elasticsearch-shard
tool supports.