Scripting

The scripting module allows to use scripts in order to evaluate custom expressions. For example, scripts can be used to return "script fields" as part of a search request, or can be used to evaluate a custom score for a query and so on.

The scripting module uses by default groovy (previously mvel in 1.3.x and earlier) as the scripting language with some extensions. Groovy is used since it is extremely fast and very simple to use.

GET /_search
{
  "script_fields": {
    "my_field": {
      "script": {
        "inline": "1 + my_var",
        "params": {
          "my_var": 2
        }
      }
    }
  }
}

1 + my_var

GET /_search
{
  "script_fields": {
    "my_field": {
      "script": {
        "file": "my_script",
        "params": {
          "my_var": 2
        }
      }
    }
  }
}

Additional lang plugins are provided to allow to execute scripts in different languages. All places where a script can be used, a lang parameter can be provided to define the language of the script. The following are the supported scripting languages:

To increase security, Elasticsearch does not allow you to specify scripts for non-sandboxed languages with a request. Instead, scripts must be placed in the scripts directory inside the configuration directory (the directory where elasticsearch.yml is). The default location of this scripts directory can be changed by setting path.scripts in elasticsearch.yml. Scripts placed into this directory will automatically be picked up and be available to be used. Once a script has been placed in this directory, it can be referenced by name. For example, a script called calculate-score.groovy can be referenced in a request like this:

$ tree config
config
├── elasticsearch.yml
├── logging.yml
└── scripts
    └── calculate-score.groovy

$ cat config/scripts/calculate-score.groovy
log(_score * 2) + my_modifier

curl -XPOST localhost:9200/_search -d '{
  "query": {
    "function_score": {
      "query": {
        "match": {
          "body": "foo"
        }
      },
      "functions": [
        {
          "script_score": {
            "script": {
              "lang": "groovy",
              "file": "calculate-score",
              "params": {
                "my_modifier": 8
              }
            }
          }
        }
      ]
    }
  }
}'

The name of the script is derived from the hierarchy of directories it exists under, and the file name without the lang extension. For example, a script placed under config/scripts/group1/group2/test.py will be named group1_group2_test.

Elasticsearch allows you to store scripts in an internal index known as .scripts and reference them by id. There are REST endpoints to manage indexed scripts as follows:

Requests to the scripts endpoint look like :

/_scripts/{lang}/{id}

Where the lang part is the language the script is in and the id part is the id of the script. In the .scripts index the type of the document will be set to the lang.

curl -XPOST localhost:9200/_scripts/groovy/indexedCalculateScore -d '{
     "script": "log(_score * 2) + my_modifier"
}'

This will create a document with id: indexedCalculateScore and type: groovy in the .scripts index. The type of the document is the language used by the script.

This script can be accessed at query time by using the id script parameter and passing the script id:

curl -XPOST localhost:9200/_search -d '{
  "query": {
    "function_score": {
      "query": {
        "match": {
          "body": "foo"
        }
      },
      "functions": [
        {
          "script_score": {
            "script": {
              "id": "indexedCalculateScore",
              "lang" : "groovy",
              "params": {
                "my_modifier": 8
              }
            }
          }
        }
      ]
    }
  }
}'

The script can be viewed by:

curl -XGET localhost:9200/_scripts/groovy/indexedCalculateScore

This is rendered as:

'{
     "script": "log(_score * 2) + my_modifier"
}'

Indexed scripts can be deleted by:

curl -XDELETE localhost:9200/_scripts/groovy/indexedCalculateScore

We recommend running Elasticsearch behind an application or proxy, which protects Elasticsearch from the outside world. If users are allowed to run inline scripts (even in a search request) or indexed scripts, then they have the same access to your box as the user that Elasticsearch is running as. For this reason dynamic scripting is allowed only for sandboxed languages by default.

First, you should not run Elasticsearch as the root user, as this would allow a script to access or do anything on your server, without limitations. Second, you should not expose Elasticsearch directly to users, but instead have a proxy application inbetween. If you do intend to expose Elasticsearch directly to your users, then you have to decide whether you trust them enough to run scripts on your box or not.

It is possible to enable scripts based on their source, for every script engine, through the following settings that need to be added to the config/elasticsearch.yml file on every node.

script.inline: true
script.indexed: true

While this still allows execution of named scripts provided in the config, or native Java scripts registered through plugins, it also allows users to run arbitrary scripts via the API. Instead of sending the name of the file as the script, the body of the script can be sent instead or retrieved from the .scripts indexed if previously stored.

There are three possible configuration values for any of the fine-grained script settings:

The default values are the following:

script.inline: sandbox
script.indexed: sandbox
script.file: true

It is also possible to control which operations can execute scripts. The supported operations are:

Plugins can also define custom operations that they use scripts for instead of using the generic plugin category. Those operations can be referred to in the following form: ${pluginName}_${operation}.

The following example disables scripting for update and mapping operations, regardless of the script source, for any engine. Scripts can still be executed from sandboxed languages as part of aggregations, search and plugins execution though, as the above defaults still get applied.

script.update: false
script.mapping: false

Generic settings get applied in order, operation based ones have precedence over source based ones. Language specific settings are supported too. They need to be prefixed with the script.engine.<engine> prefix and have precedence over any other generic settings.

script.engine.groovy.file.aggs: true
script.engine.groovy.file.mapping: true
script.engine.groovy.file.search: true
script.engine.groovy.file.update: true
script.engine.groovy.file.plugin: true
script.engine.groovy.indexed.aggs: true
script.engine.groovy.indexed.mapping: false
script.engine.groovy.indexed.search: true
script.engine.groovy.indexed.update: false
script.engine.groovy.indexed.plugin: false
script.engine.groovy.inline.aggs: true
script.engine.groovy.inline.mapping: false
script.engine.groovy.inline.search: false
script.engine.groovy.inline.update: false
script.engine.groovy.inline.plugin: false

The default scripting language (assuming no lang parameter is provided) is groovy. In order to change it, set the script.default_lang to the appropriate language.

The config/scripts directory is scanned periodically for changes. New and changed scripts are reloaded and deleted script are removed from preloaded scripts cache. The reload frequency can be specified using resource.reload.interval setting, which defaults to 60s. To disable script reloading completely set script.auto_reload_enabled to false.

Sometimes groovy and expressions aren’t enough. For those times you can implement a native script.

The best way to implement a native script is to write a plugin and install it. The plugin documentation has more information on how to write a plugin so that Elasticsearch will properly load it.

To register the actual script you’ll need to implement NativeScriptFactory to construct the script. The actual script will extend either AbstractExecutableScript or AbstractSearchScript. The second one is likely the most useful and has several helpful subclasses you can extend like AbstractLongSearchScript, AbstractDoubleSearchScript, and AbstractFloatSearchScript. Finally, your plugin should register the native script by declaring the onModule(ScriptModule) method.

If you squashed the whole thing into one class it’d look like:

public class MyNativeScriptPlugin extends Plugin {
    @Override
    public String name() {
        return "my-native-script";
    }
    @Override
    public String description() {
        return "my native script that does something great";
    }
    public void onModule(ScriptModule scriptModule) {
        scriptModule.registerScript("my_script", MyNativeScriptFactory.class);
    }

    public static class MyNativeScriptFactory implements NativeScriptFactory {
        @Override
        public ExecutableScript newScript(@Nullable Map<String, Object> params) {
            return new MyNativeScript();
        }
        @Override
        public boolean needsScores() {
            return false;
        }
    }

    public static class MyNativeScript extends AbstractFloatSearchScript {
        @Override
        public float runAsFloat() {
            float a = (float) source().get("a");
            float b = (float) source().get("b");
            return a * b;
        }
    }
}

You can execute the script by specifying its lang as native, and the name of the script as the inline:

curl -XPOST localhost:9200/_search -d '{
  "query": {
    "function_score": {
      "query": {
        "match": {
          "body": "foo"
        }
      },
      "functions": [
        {
          "script_score": {
            "script": {
                "inline": "my_script",
                "lang" : "native"
            }
          }
        }
      ]
    }
  }
}'

Lucene’s expressions module provides a mechanism to compile a javascript expression to bytecode. This allows very fast execution, as if you had written a native script. Expression scripts can be used in script_score, script_fields, sort scripts and numeric aggregation scripts.

See the expressions module documentation for details on what operators and functions are available.

Variables in expression scripts are available to access:

Variables in expression scripts that are of type date may use the following member methods:

The following example shows the difference in years between the date fields date0 and date1:

doc['date1'].getYear() - doc['date0'].getYear()

There are a few limitations relative to other script languages:

In all scripts that can be used in aggregations, the current document’s score is accessible in _score.

see advanced scripting documentation

Most scripting revolve around the use of specific document fields data. The doc['field_name'] can be used to access specific field data within a document (the document in question is usually derived by the context the script is used). Document fields are very fast to access since they end up being loaded into memory (all the relevant field values/tokens are loaded to memory). Note, however, that the doc[...] notation only allows for simple valued fields (can’t return a json object from it) and makes sense only on non-analyzed or single term based fields.

The following data can be extracted from a field:

Stored fields can also be accessed when executing a script. Note, they are much slower to access compared with document fields, as they are not loaded into memory. They can be simply accessed using _fields['my_field_name'].value or _fields['my_field_name'].values.

When using scripting for calculating the score of a document (for instance, with the function_score query), you can access the score using the _score variable inside of a Groovy script.

The source field can also be accessed when executing a script. The source field is loaded per doc, parsed, and then provided to the script for evaluation. The _source forms the context under which the source field can be accessed, for example _source.obj2.obj1.field3.

Accessing _source is much slower compared to using doc but the data is not loaded into memory. For a single field access _fields may be faster than using _source due to the extra overhead of potentially parsing large documents. However, _source may be faster if you access multiple fields or if the source has already been loaded for other purposes.

There are several built in functions that can be used within scripts. They include: