Reindex from a remote cluster
editReindex from a remote cluster
editYou can use reindex from remote to migrate indices from your old cluster to a new 6.x cluster. This enables you move to 6.x from a pre-5.6 cluster without interrupting service.
Elasticsearch provides backwards compatibility support that enables indices from the previous major version to be upgraded to the current major version. Skipping a major version means that you must resolve any backward compatibility issues yourself.
To migrate your indices:
-
Set up a new 6.x cluster alongside your old cluster. Enable it to access your old cluster by adding your old cluster to the
reindex.remote.whitelist
inelasticsearch.yml
:reindex.remote.whitelist: oldhost:9200
The new cluster doesn’t have to start fully-scaled out. As you migrate indices and shift the load to the new cluster, you can add nodes to the new cluster and remove nodes from the old one.
-
For each index that you need to migrate to the 6.x cluster:
-
Create a new index in 6.x with the appropriate mappings and settings. Set the
refresh_interval
to-1
and setnumber_of_replicas
to0
for faster reindexing. -
Reindex from remote to pull documents from the old index into the new 6.x index:
POST _reindex { "source": { "remote": { "host": "http://oldhost:9200", "username": "user", "password": "pass" }, "index": "source", "query": { "match": { "test": "data" } } }, "dest": { "index": "dest" } }
If you run the reindex job in the background by setting
wait_for_completion
tofalse
, the reindex request returns atask_id
you can use to monitor progress of the reindex job with the task API:GET _tasks/TASK_ID
. -
When the reindex job completes, set the
refresh_interval
andnumber_of_replicas
to the desired values (the default settings are30s
and1
). -
Once replication is complete and the status of the new index is
green
, you can delete the old index.
-
Create a new index in 6.x with the appropriate mappings and settings. Set the