Script related changes

The default scripting language for Elasticsearch is now Painless. Painless is a custom-built language with syntax similar to Groovy designed to be fast as well as secure. Many Groovy scripts will be identitical to Painless scripts to help make the transition between languages as simple as possible.

Documentation for Painless can be found at Painless Scripting Language

One common difference to note between Groovy and Painless is the use of parameters — all parameters in Painless must be prefixed with params. now. The following example shows the difference:

Groovy:

{
  "script_score": {
    "script": {
      "lang": "groovy",
      "inline": "Math.log(_score * 2) + my_modifier",
      "params": {
        "my_modifier": 8
      }
    }
  }
}

Painless (my_modifer is prefixed with params):

{
  "script_score": {
    "script": {
      "lang": "painless",
      "inline": "Math.log(_score * 2) + params.my_modifier",
      "params": {
        "my_modifier": 8
      }
    }
  }
}

The script.default_lang setting has been removed. It is no longer possible set the default scripting language. If a different language than painless is used then this should be explicitly specified on the script itself.

For scripts with no explicit language defined, that are part of already stored percolator queries, the default language can be controlled with the script.legacy.default_lang setting.

The deprecated 1.x syntax of defining inline scripts / templates and referring to file or index base scripts / templates have been removed.

The script and params string parameters can no longer be used and instead the script object syntax must be used. This applies for the update api, script sort, script_score function, script query, scripted_metric aggregation and script_heuristic aggregation.

So this usage of inline scripts is no longer allowed:

{
  "script_score": {
    "lang": "groovy",
    "script": "Math.log(_score * 2) + my_modifier",
    "params": {
      "my_modifier": 8
    }
  }
}

and instead this syntax must be used:

{
  "script_score": {
    "script": {
      "lang": "groovy",
      "inline": "Math.log(_score * 2) + my_modifier",
      "params": {
        "my_modifier": 8
      }
    }
  }
}

The script or script_file parameter can no longer be used to refer to file based scripts and templates and instead file must be used.

This usage of referring to file based scripts is no longer valid:

{
  "script_score": {
    "script": "calculate-score",
    "params": {
      "my_modifier": 8
    }
  }
}

This usage is valid:

{
  "script_score": {
    "script": {
      "lang": "groovy",
      "file": "calculate-score",
      "params": {
        "my_modifier": 8
      }
    }
  }
}

The script_id parameter can no longer be used the refer to indexed based scripts and templates and instead id must be used.

This usage of referring to indexed scripts is no longer valid:

{
  "script_score": {
    "script_id": "indexedCalculateScore",
    "params": {
      "my_modifier": 8
    }
  }
}

This usage is valid:

{
  "script_score": {
    "script": {
      "id": "indexedCalculateScore",
      "lang" : "groovy",
      "params": {
        "my_modifier": 8
      }
    }
  }
}

The query field in the template query can no longer be used. This 1.x syntax can no longer be used:

{
    "query": {
        "template": {
            "query": {"match_{{template}}": {}},
            "params" : {
                "template" : "all"
            }
        }
    }
}

and instead the following syntax should be used:

{
    "query": {
        "template": {
            "inline": {"match_{{template}}": {}},
            "params" : {
                "template" : "all"
            }
        }
    }
}

The top level template field in the search template api has been replaced with consistent template / script object syntax. This 1.x syntax can no longer be used:

{
    "template" : {
        "query": { "match" : { "{{my_field}}" : "{{my_value}}" } },
        "size" : "{{my_size}}"
    },
    "params" : {
        "my_field" : "foo",
        "my_value" : "bar",
        "my_size" : 5
    }
}

and instead the following syntax should be used:

{
    "inline" : {
        "query": { "match" : { "{{my_field}}" : "{{my_value}}" } },
        "size" : "{{my_size}}"
    },
    "params" : {
        "my_field" : "foo",
        "my_value" : "bar",
        "my_size" : 5
    }
}

Indexed scripts and templates have been replaced by stored scripts which stores the scripts and templates in the cluster state instead of a dedicate .scripts index.

For the size of stored scripts there is a soft limit of 65535 bytes. If scripts exceed that size then the script.max_size_in_bytes setting can be added to elasticsearch.yml to change the soft limit to a higher value. If scripts are really large, other options like native scripts should be considered.

Previously indexed scripts in the .scripts index will not be used any more as Elasticsearch will now try to fetch the scripts from the cluster state. Upon upgrading to 5.x the .scripts index will remain to exist, so it can be used by a script to migrate the stored scripts from the .scripts index into the cluster state. The current format of the scripts and templates hasn’t been changed, only the 1.x format has been removed.

from elasticsearch import Elasticsearch,helpers

es = Elasticsearch([
	{'host': 'localhost'}
])

for doc in helpers.scan(es, index=".scripts", preserve_order=True):
	es.put_script(lang=doc['_type'], id=doc['_id'], body=doc['_source'])

use Search::Elasticsearch;

my $es     = Search::Elasticsearch->new( nodes => 'localhost:9200');
my $scroll = $es->scroll_helper( index => '.scripts', sort => '_doc');

while (my $doc = $scroll->next) {
  $e->put_script(
    lang => $doc->{_type},
    id   => $doc->{_id},
    body => $doc->{_source}
  );
}

GET _cluster/state?filter_path=metadata.stored_scripts

All the methods related to interacting with indexed scripts have been removed. The Java API methods for interacting with stored scripts have been added under ClusterAdminClient class. The sugar methods that used to exist on the indexed scripts API methods don’t exist on the methods for stored scripts. The only way to provide scripts is by using BytesReference implementation, if a string needs to be provided the BytesArray class should be used.

Prior to 5.0.0, script engines could register multiple languages. The Javascript script engine in particular registered both "lang": "js" and "lang": "javascript". Script engines can now only register a single language. All references to "lang": "js" should be changed to "lang": "javascript" for existing users of the lang-javascript plugin.

Prior to 5.0.0 scripting engines could register multiple extensions. The only engine doing this was the Javascript engine, which registered "js" and "javascript". It now only registers the "js" file extension for on-disk scripts.

The Javascript engine previously registered "js" and "javascript". It now only registers the "js" file extension for on-disk scripts.

The script, script_id and scripting_upsert query string parameters have been removed from the update api.

The TemplateQueryBuilder has been moved to the lang-mustache module. Therefor when using the TemplateQueryBuilder from the Java native client the lang-mustache module should be on the classpath. Also the transport client should load the lang-mustache module as plugin:

TransportClient transportClient = TransportClient.builder()
        .settings(Settings.builder().put("node.name", "node"))
        .addPlugin(MustachePlugin.class)
        .build();
transportClient.addTransportAddress(
        new InetSocketTransportAddress(new InetSocketAddress(InetAddresses.forString("127.0.0.1"), 9300))
);

Also the helper methods in QueryBuilders class that create a TemplateQueryBuilder instance have been removed, instead the constructors on TemplateQueryBuilder should be used.

The template query has been deprecated in favour of the search template api. The template query is scheduled to be removed in the next major version.

The following helper methods have been removed from GeoPoint scripting:

Instead use arcDistance, arcDistanceWithDefault, planeDistance, planeDistanceWithDefault, geohashDistance, geohashDistanceWithDefault and convert from default units (meters) to desired units using the appropriate constance (e.g., multiply by 0.001 to convert to Km).

If you compile too many unique scripts within a small amount of time, Elasticsearch will reject the new dynamic scripts with a circuit_breaking_exception error. By default, up to 15 inline scripts per minute will be compiled. You can change this setting dynamically by setting script.max_compilations_per_minute.

You should watch out for this if you are hard-coding values into your scripts.

Elasticsearch recommends the usage of parameters for efficient script handling. See details here.