« Painless scripting language Scripts, caching, and search speed »
Elastic Docs ›Elasticsearch Guide [8.15] ›Scripting

How to write scripts

edit

How to write scripts

edit

Wherever scripting is supported in the Elasticsearch APIs, the syntax follows the same pattern; you specify the language of your script, provide the script logic (or source), and add parameters that are passed into the script:

  "script": {
    "lang":   "...",
    "source" | "id": "...",
    "params": { ... }
  }
lang
Specifies the language the script is written in. Defaults to painless.
source, id
The script itself, which you specify as source for an inline script or id for a stored script. Use the stored script APIs to create and manage stored scripts.
params
Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

Write your first script

edit

Painless is the default scripting language for Elasticsearch. It is secure, performant, and provides a natural syntax for anyone with a little coding experience.

A Painless script is structured as one or more statements and optionally has one or more user-defined functions at the beginning. A script must always have at least one statement.

The Painless execute API provides the ability to test a script with simple user-defined parameters and receive a result. Let’s start with a complete script and review its constituent parts.

First, index a document with a single field so that we have some data to work with:

resp = client.index(
    index="my-index-000001",
    id="1",
    document={
        "my_field": 5
    },
)
print(resp)
response = client.index(
  index: 'my-index-000001',
  id: 1,
  body: {
    my_field: 5
  }
)
puts response
const response = await client.index({
  index: "my-index-000001",
  id: 1,
  document: {
    my_field: 5,
  },
});
console.log(response);
PUT my-index-000001/_doc/1
{
  "my_field": 5
}

We can then construct a script that operates on that field and run evaluate the script as part of a query. The following query uses the script_fields parameter of the search API to retrieve a script valuation. There’s a lot happening here, but we’ll break it down the components to understand them individually. For now, you only need to understand that this script takes my_field and operates on it.

resp = client.search(
    index="my-index-000001",
    script_fields={
        "my_doubled_field": {
            "script": {
                "source": "doc['my_field'].value * params['multiplier']",
                "params": {
                    "multiplier": 2
                }
            }
        }
    },
)
print(resp)
response = client.search(
  index: 'my-index-000001',
  body: {
    script_fields: {
      my_doubled_field: {
        script: {
          source: "doc['my_field'].value * params['multiplier']",
          params: {
            multiplier: 2
          }
        }
      }
    }
  }
)
puts response
const response = await client.search({
  index: "my-index-000001",
  script_fields: {
    my_doubled_field: {
      script: {
        source: "doc['my_field'].value * params['multiplier']",
        params: {
          multiplier: 2,
        },
      },
    },
  },
});
console.log(response);
GET my-index-000001/_search
{
  "script_fields": {
    "my_doubled_field": {
      "script": { 
        "source": "doc['my_field'].value * params['multiplier']", 
        "params": {
          "multiplier": 2
        }
      }
    }
  }
}

script object

script source

The script is a standard JSON object that defines scripts under most APIs in Elasticsearch. This object requires source to define the script itself. The script doesn’t specify a language, so it defaults to Painless.

Use parameters in your script

edit

The first time Elasticsearch sees a new script, it compiles the script and stores the compiled version in a cache. Compilation can be a heavy process. Rather than hard-coding values in your script, pass them as named params instead.

For example, in the previous script, we could have just hard coded values and written a script that is seemingly less complex. We could just retrieve the first value for my_field and then multiply it by 2:

"source": "return doc['my_field'].value * 2"

Though it works, this solution is pretty inflexible. We have to modify the script source to change the multiplier, and Elasticsearch has to recompile the script every time that the multiplier changes.

Instead of hard-coding values, use named params to make scripts flexible, and also reduce compilation time when the script runs. You can now make changes to the multiplier parameter without Elasticsearch recompiling the script.

"source": "doc['my_field'].value * params['multiplier']",
"params": {
  "multiplier": 2
}

You can compile up to 150 scripts per 5 minutes by default. For ingest contexts, the default script compilation rate is unlimited.

script.context.field.max_compilations_rate=100/10m

If you compile too many unique scripts within a short time, Elasticsearch rejects the new dynamic scripts with a circuit_breaking_exception error.

Shorten your script

edit

Using syntactic abilities that are native to Painless, you can reduce verbosity in your scripts and make them shorter. Here’s a simple script that we can make shorter:

resp = client.search(
    index="my-index-000001",
    script_fields={
        "my_doubled_field": {
            "script": {
                "lang": "painless",
                "source": "doc['my_field'].value * params.get('multiplier');",
                "params": {
                    "multiplier": 2
                }
            }
        }
    },
)
print(resp)
response = client.search(
  index: 'my-index-000001',
  body: {
    script_fields: {
      my_doubled_field: {
        script: {
          lang: 'painless',
          source: "doc['my_field'].value * params.get('multiplier');",
          params: {
            multiplier: 2
          }
        }
      }
    }
  }
)
puts response
const response = await client.search({
  index: "my-index-000001",
  script_fields: {
    my_doubled_field: {
      script: {
        lang: "painless",
        source: "doc['my_field'].value * params.get('multiplier');",
        params: {
          multiplier: 2,
        },
      },
    },
  },
});
console.log(response);
GET my-index-000001/_search
{
  "script_fields": {
    "my_doubled_field": {
      "script": {
        "lang":   "painless",
        "source": "doc['my_field'].value * params.get('multiplier');",
        "params": {
          "multiplier": 2
        }
      }
    }
  }
}

Let’s look at a shortened version of the script to see what improvements it includes over the previous iteration:

resp = client.search(
    index="my-index-000001",
    script_fields={
        "my_doubled_field": {
            "script": {
                "source": "field('my_field').get(null) * params['multiplier']",
                "params": {
                    "multiplier": 2
                }
            }
        }
    },
)
print(resp)
const response = await client.search({
  index: "my-index-000001",
  script_fields: {
    my_doubled_field: {
      script: {
        source: "field('my_field').get(null) * params['multiplier']",
        params: {
          multiplier: 2,
        },
      },
    },
  },
});
console.log(response);
GET my-index-000001/_search
{
  "script_fields": {
    "my_doubled_field": {
      "script": {
        "source": "field('my_field').get(null) * params['multiplier']",
        "params": {
          "multiplier": 2
        }
      }
    }
  }
}

This version of the script removes several components and simplifies the syntax significantly:

  • The lang declaration. Because Painless is the default language, you don’t need to specify the language if you’re writing a Painless script.
  • The return keyword. Painless automatically uses the final statement in a script (when possible) to produce a return value in a script context that requires one.
  • The get method, which is replaced with brackets []. Painless uses a shortcut specifically for the Map type that allows us to use brackets instead of the lengthier get method.
  • The semicolon at the end of the source statement. Painless does not require semicolons for the final statement of a block. However, it does require them in other cases to remove ambiguity.

Use this abbreviated syntax anywhere that Elasticsearch supports scripts, such as when you’re creating runtime fields.

Store and retrieve scripts

edit

You can store and retrieve scripts from the cluster state using the stored script APIs. Stored scripts reduce compilation time and make searches faster.

Unlike regular scripts, stored scripts require that you specify a script language using the lang parameter.

To create a script, use the create stored script API. For example, the following request creates a stored script named calculate-score.

resp = client.put_script(
    id="calculate-score",
    script={
        "lang": "painless",
        "source": "Math.log(_score * 2) + params['my_modifier']"
    },
)
print(resp)
response = client.put_script(
  id: 'calculate-score',
  body: {
    script: {
      lang: 'painless',
      source: "Math.log(_score * 2) + params['my_modifier']"
    }
  }
)
puts response
const response = await client.putScript({
  id: "calculate-score",
  script: {
    lang: "painless",
    source: "Math.log(_score * 2) + params['my_modifier']",
  },
});
console.log(response);
POST _scripts/calculate-score
{
  "script": {
    "lang": "painless",
    "source": "Math.log(_score * 2) + params['my_modifier']"
  }
}

You can retrieve that script by using the get stored script API.

resp = client.get_script(
    id="calculate-score",
)
print(resp)
response = client.get_script(
  id: 'calculate-score'
)
puts response
const response = await client.getScript({
  id: "calculate-score",
});
console.log(response);
GET _scripts/calculate-score

To use the stored script in a query, include the script id in the script declaration:

resp = client.search(
    index="my-index-000001",
    query={
        "script_score": {
            "query": {
                "match": {
                    "message": "some message"
                }
            },
            "script": {
                "id": "calculate-score",
                "params": {
                    "my_modifier": 2
                }
            }
        }
    },
)
print(resp)
response = client.search(
  index: 'my-index-000001',
  body: {
    query: {
      script_score: {
        query: {
          match: {
            message: 'some message'
          }
        },
        script: {
          id: 'calculate-score',
          params: {
            my_modifier: 2
          }
        }
      }
    }
  }
)
puts response
const response = await client.search({
  index: "my-index-000001",
  query: {
    script_score: {
      query: {
        match: {
          message: "some message",
        },
      },
      script: {
        id: "calculate-score",
        params: {
          my_modifier: 2,
        },
      },
    },
  },
});
console.log(response);
GET my-index-000001/_search
{
  "query": {
    "script_score": {
      "query": {
        "match": {
            "message": "some message"
        }
      },
      "script": {
        "id": "calculate-score", 
        "params": {
          "my_modifier": 2
        }
      }
    }
  }
}

id of the stored script

To delete a stored script, submit a delete stored script API request.

resp = client.delete_script(
    id="calculate-score",
)
print(resp)
response = client.delete_script(
  id: 'calculate-score'
)
puts response
const response = await client.deleteScript({
  id: "calculate-score",
});
console.log(response);
DELETE _scripts/calculate-score

Update documents with scripts

edit

You can use the update API to update documents with a specified script. The script can update, delete, or skip modifying the document. The update API also supports passing a partial document, which is merged into the existing document.

First, let’s index a simple document:

resp = client.index(
    index="my-index-000001",
    id="1",
    document={
        "counter": 1,
        "tags": [
            "red"
        ]
    },
)
print(resp)
response = client.index(
  index: 'my-index-000001',
  id: 1,
  body: {
    counter: 1,
    tags: [
      'red'
    ]
  }
)
puts response
const response = await client.index({
  index: "my-index-000001",
  id: 1,
  document: {
    counter: 1,
    tags: ["red"],
  },
});
console.log(response);
PUT my-index-000001/_doc/1
{
  "counter" : 1,
  "tags" : ["red"]
}

To increment the counter, you can submit an update request with the following script:

resp = client.update(
    index="my-index-000001",
    id="1",
    script={
        "source": "ctx._source.counter += params.count",
        "lang": "painless",
        "params": {
            "count": 4
        }
    },
)
print(resp)
response = client.update(
  index: 'my-index-000001',
  id: 1,
  body: {
    script: {
      source: 'ctx._source.counter += params.count',
      lang: 'painless',
      params: {
        count: 4
      }
    }
  }
)
puts response
const response = await client.update({
  index: "my-index-000001",
  id: 1,
  script: {
    source: "ctx._source.counter += params.count",
    lang: "painless",
    params: {
      count: 4,
    },
  },
});
console.log(response);
POST my-index-000001/_update/1
{
  "script" : {
    "source": "ctx._source.counter += params.count",
    "lang": "painless",
    "params" : {
      "count" : 4
    }
  }
}

Similarly, you can use an update script to add a tag to the list of tags. Because this is just a list, the tag is added even it exists:

resp = client.update(
    index="my-index-000001",
    id="1",
    script={
        "source": "ctx._source.tags.add(params['tag'])",
        "lang": "painless",
        "params": {
            "tag": "blue"
        }
    },
)
print(resp)
response = client.update(
  index: 'my-index-000001',
  id: 1,
  body: {
    script: {
      source: "ctx._source.tags.add(params['tag'])",
      lang: 'painless',
      params: {
        tag: 'blue'
      }
    }
  }
)
puts response
const response = await client.update({
  index: "my-index-000001",
  id: 1,
  script: {
    source: "ctx._source.tags.add(params['tag'])",
    lang: "painless",
    params: {
      tag: "blue",
    },
  },
});
console.log(response);
POST my-index-000001/_update/1
{
  "script": {
    "source": "ctx._source.tags.add(params['tag'])",
    "lang": "painless",
    "params": {
      "tag": "blue"
    }
  }
}

You can also remove a tag from the list of tags. The remove method of a Java List is available in Painless. It takes the index of the element you want to remove. To avoid a possible runtime error, you first need to make sure the tag exists. If the list contains duplicates of the tag, this script just removes one occurrence.

resp = client.update(
    index="my-index-000001",
    id="1",
    script={
        "source": "if (ctx._source.tags.contains(params['tag'])) { ctx._source.tags.remove(ctx._source.tags.indexOf(params['tag'])) }",
        "lang": "painless",
        "params": {
            "tag": "blue"
        }
    },
)
print(resp)
response = client.update(
  index: 'my-index-000001',
  id: 1,
  body: {
    script: {
      source: "if (ctx._source.tags.contains(params['tag'])) { ctx._source.tags.remove(ctx._source.tags.indexOf(params['tag'])) }",
      lang: 'painless',
      params: {
        tag: 'blue'
      }
    }
  }
)
puts response
const response = await client.update({
  index: "my-index-000001",
  id: 1,
  script: {
    source:
      "if (ctx._source.tags.contains(params['tag'])) { ctx._source.tags.remove(ctx._source.tags.indexOf(params['tag'])) }",
    lang: "painless",
    params: {
      tag: "blue",
    },
  },
});
console.log(response);
POST my-index-000001/_update/1
{
  "script": {
    "source": "if (ctx._source.tags.contains(params['tag'])) { ctx._source.tags.remove(ctx._source.tags.indexOf(params['tag'])) }",
    "lang": "painless",
    "params": {
      "tag": "blue"
    }
  }
}

You can also add and remove fields from a document. For example, this script adds the field new_field:

resp = client.update(
    index="my-index-000001",
    id="1",
    script="ctx._source.new_field = 'value_of_new_field'",
)
print(resp)
response = client.update(
  index: 'my-index-000001',
  id: 1,
  body: {
    script: "ctx._source.new_field = 'value_of_new_field'"
  }
)
puts response
const response = await client.update({
  index: "my-index-000001",
  id: 1,
  script: "ctx._source.new_field = 'value_of_new_field'",
});
console.log(response);
POST my-index-000001/_update/1
{
  "script" : "ctx._source.new_field = 'value_of_new_field'"
}

Conversely, this script removes the field new_field:

resp = client.update(
    index="my-index-000001",
    id="1",
    script="ctx._source.remove('new_field')",
)
print(resp)
response = client.update(
  index: 'my-index-000001',
  id: 1,
  body: {
    script: "ctx._source.remove('new_field')"
  }
)
puts response
const response = await client.update({
  index: "my-index-000001",
  id: 1,
  script: "ctx._source.remove('new_field')",
});
console.log(response);
POST my-index-000001/_update/1
{
  "script" : "ctx._source.remove('new_field')"
}

Instead of updating the document, you can also change the operation that is executed from within the script. For example, this request deletes the document if the tags field contains green. Otherwise it does nothing (noop):

resp = client.update(
    index="my-index-000001",
    id="1",
    script={
        "source": "if (ctx._source.tags.contains(params['tag'])) { ctx.op = 'delete' } else { ctx.op = 'none' }",
        "lang": "painless",
        "params": {
            "tag": "green"
        }
    },
)
print(resp)
response = client.update(
  index: 'my-index-000001',
  id: 1,
  body: {
    script: {
      source: "if (ctx._source.tags.contains(params['tag'])) { ctx.op = 'delete' } else { ctx.op = 'none' }",
      lang: 'painless',
      params: {
        tag: 'green'
      }
    }
  }
)
puts response
const response = await client.update({
  index: "my-index-000001",
  id: 1,
  script: {
    source:
      "if (ctx._source.tags.contains(params['tag'])) { ctx.op = 'delete' } else { ctx.op = 'none' }",
    lang: "painless",
    params: {
      tag: "green",
    },
  },
});
console.log(response);
POST my-index-000001/_update/1
{
  "script": {
    "source": "if (ctx._source.tags.contains(params['tag'])) { ctx.op = 'delete' } else { ctx.op = 'none' }",
    "lang": "painless",
    "params": {
      "tag": "green"
    }
  }
}
« Painless scripting language Scripts, caching, and search speed »