Elasticsearch 5.0.0-alpha3 released

The Elasticsearch Migration Helper is a site plugin designed to help you to prepare for your migration from Elasticsearch 2.3.x to Elasticsearch 5.0. It comes with three tools:

Instruction for install the Elasticsearch migration helper.

Command line settings and system property settings have been refactored to reduce ambiguity. This is a small change which will impact every Elasticsearch installation, but has an easy migration path. Command line settings are now passed with the -E parameter as follows:

./bin/elasticsearch -Epath.data=/path/to/data

System properties such as JVM heap size settings can be specified in the config/jvm.options file or via the ES_JAVA_OPTS environment variable:

ES_JAVA_OPTS="-Xms2g -Xmx2g" ./bin/elasticsearch

System properties can no longer be passed with the -D command line syntax.

A number of system checks are performed during node startup. Any failing checks will produce a warning when in development mode (bound to localhost) or a hard exception when in production mode (bound to a network interface other than localhost). These failing checks should be fixed before going into production, for the good of your cluster. Sometimes, however, you need to bind to a public interface while still in development so we have added an escape hatch setting which allows you to downgrade these exceptions to warnings. Just set bootstrap.ignore_system_bootstrap_checks to true.

A change to reduce contention in the lock that prevents concurrent updates to the same document provides a 15-20% performance boost when indexing small documents. Another change, reducing the locking required while fsyncing the transaction log, provides another performance boost, especially when running on machines with lots of CPUs but with spinning disks.

The new enabled-by-default scripting language, Painless, has received a huge number of improvements:

Many more improvements will be made to Painless before GA.

In Elasticsearch 2.0 we removed support for dots in fields names because of the ambiguity between dotted fields and object fields. In 5.0, we have resolved this ambiguity and treat dots in field names as though they were an object. In other words, the following two documents are treated as thought they were the same:

PUT my_index/my_type/1
{
  "aaa.bbb.ccc": "some_val",
  "aaa.ddd": "other_val"
}

PUT my_index/my_type/2
{
  "aaa": {
    "bbb": {
      "ccc": "some_val"
    },
    "ddd": "other_val"
  }
}

Figuring out why a shard is unassigned can be tricky. You might need to look at the cluster-reroute API, the shard-stores API, the recovery API, index and cluster settings, and the logs. Now, we have a single API which pulls all of this information together: the Cluster Allocation Explain API.

Sometimes shards fail to be allocated because of some structural problem, like a missing stopwords or synonyms file. Previously, Elasticsearch would keep trying to allocate the shard forever, resulting in an torrent of log messages which would only stop once the disk filled up. Now we try to allocate a shard a maximum of 5 times before giving up. Once the problem has been diagnosed and fixed, shard allocation can be triggered with the Cluster Reroute API.

Please download Elasticsearch 5.0.0-alpha3, try it out, and let us know what you think on Twitter (@elastic) or in our forum. You can report any problems on the GitHub issues page.