Elasticsearch 2.3.0 and 2.2.2 released

Anybody who has worked with Elasticsearch for any length of time knows that the ability to reindex your data is essential. Perhaps you need to improve your field mappings or analysis chain in order to get better search results, or perhaps you want to reindex old data to take advantage of improved data structures like the new geo-points available in Elasticsearch 2.2.0. Whatever the reason, it should be easy to reindex your data. It has long been possible to do so, using the scroll and bulk APIs, but with 2.3.0 it has just become a lot easier.

The new _reindex API will reindex all of the documents in index into a new index with a single command:

POST _reindex
{
  "source": {
    "index": "my_old_index"
  },
  "dest": {
    "index": "my_new_index"
  }
}

Documents can come from one or more indices and can be limited to the subset of documents which match a query. Documents can be transformed during the reindexing process using a script, which even has access to the document metadata like _id and _type.

Elasticsearch 2.3.0 contains the first version of the reindex API, but we already have plans for future improvements such as dynamic throttling, reindex-in-place, reindexing from remote clusters, and more.

A nice side effect of adding the reindex API is that we get the update-by-query API for free. The update-by-query API allows you to update all of the documents which match a query using a script. It is even useful without a script: after adding a multi-field, an empty update-by-query request can be used to back-fill the values for the multi-field for all of the documents already in the index.

You can read more about the reindex and update-by-query APIs in Reindex is coming!.

It can take a long time to reindex terabytes of data. You wouldn’t want to start a reindex job, realise that you had made a mistake, and then be unable to cancel it! It was important to provide a way to monitor and cancel long running tasks before we could safely expose reindexing.

The new task management API allows you to retrieve the status of ongoing reindexing jobs and even cancel them. It not only reports on long running jobs, it reports on all the tasks being executed across your cluster, including searches, indexing requests, shard allocation and so on. Later, we will move other long running tasks like snapshot and restore and delete-by-query to take advantage of the task management framework.

You can read more in the Task Management API documentation.

It can be difficult to know whether you are using deprecated features that will disappear in the next major version. In 2.0, we merged filters with queries and deprecated queries like the filtered query and and and or filters, but these constructs have continued to work throughout the 2.x series. These will be removed in 5.0, along with other deprecated functionality.

To help you figure out whether you are using deprecated constructs anywhere in your application, turn on deprecation logging with the following cluster-settings-update request:

PUT _cluster/settings
{
  "transient": {
    "logger.deprecation": "DEBUG"
  }
}

Elasticsearch will log all usage of deprecated features (for which we have coverage) to the elasticsearch_deprecation.log file in the logs directory. Logging can be disabled again by setting logger.deprecation back to INFO.

Please download Elasticsearch 2.3.0, try it out, and let us know what you think on Twitter (@elastic) or in our forum. You can report any problems on the GitHub issues page.