« Painless Debugging Using ingest processors in Painless »

› ›

Painless execute API

This functionality is in technical preview and may be changed or removed in a future release. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features.

The Painless execute API runs a script and returns a result.

Request

edit

POST /_scripts/painless/_execute

Description

edit

Use this API to build and test scripts, such as when defining a script for a runtime field. This API requires very few dependencies, and is especially useful if you don’t have permissions to write documents on a cluster.

The API uses several contexts, which control how scripts are executed, what variables are available at runtime, and what the return type is.

Each context requires a script, but additional parameters depend on the context you’re using for that script.

Request body

edit

script

(Required, object) The Painless script to execute.

Properties of script

emit

(Required) Accepts the values from the script valuation. Scripts can call the emit method multiple times to emit multiple values.

The emit method applies only to scripts used in a runtime fields context.

The emit method cannot accept null values. Do not call this method if the referenced fields do not have any values.

Signatures of emit

The signature for emit depends on the type of the field.

`boolean`	`emit(boolean)`
`date`	`emit(long)`
`double`	`emit(double)`
`geo_point`	`emit(double lat, double lon)`
`ip`	`emit(String)`
`long`	`emit(long)`
`keyword`	`emit(String)`

context

(Optional, string) The context that the script should run in. Defaults to painless_test if no context is specified.

Properties of context

painless_test: The default context if no other context is specified. See test context.
filter: Treats scripts as if they were run inside a script query. See filter context.
score: Treats scripts as if they were run inside a script_score function in a function_score query. See score context.

Field contexts

The following options are specific to the field contexts.

Result ordering in the field contexts is not guaranteed.

boolean_field: The context for boolean fields. The script returns a true or false response. See boolean_field context.
date_field: The context for date fields. emit takes a long value and the script returns a sorted list of dates. See date_time context.
double_field: The context for double numeric fields. The script returns a sorted list of double values. See double_field context.
geo_point_field: The context for geo-point fields. emit takes two double parameters, the latitude and longitude values, and the script returns an object in GeoJSON format containing the coordinates for the geo point. See geo_point_field context.
ip_field: The context for ip fields. The script returns a sorted list of IP addresses. See ip_field context.
keyword_field: The context for keyword fields. The script returns a sorted list of string values. See keyword_field context.
long_field: The context for long numeric fields. The script returns a sorted list of long values. See long_field context.
composite_field: The context for composite runtime fields. The script returns a map of values. See composite_field context.

context_setup

(Required, object) Additional parameters for the context.

This parameter is required for all contexts except painless_test, which is the default if no value is provided for context.

Properties of context_setup

document: (Required, string) Document that’s temporarily indexed in-memory and accessible from the script.
index: (Required, string) Index containing a mapping that’s compatible with the indexed document. You may specify a remote index by prefixing the index with the remote cluster alias. For example, remote1:my_index indicates that you want to execute the painless script against the "my_index" index on the "remote1" cluster. This request will be forwarded to the "remote1" cluster if you have configured a connection to that remote cluster.

Wildcards are not accepted in the index expression for this endpoint. The expression *:myindex will return the error "No such remote cluster" and the expression logs* or remote1:logs* will return the error "index not found".

params

(Map, read-only) Specifies any named parameters that are passed into the script as variables.

query

(Optional, object)

This parameter only applies when score is specified as the script context.

Use this parameter to specify a query for computing a score. Besides deciding whether or not the document matches, the query clause also calculates a relevance score in the _score metadata field.

Test context

edit

The painless_test context runs scripts without additional parameters. The only variable that is available is params, which can be used to access user defined values. The result of the script is always converted to a string.

Because the default context is painless_test, you don’t need to specify the context or context_setup.

Request

edit

POST /_scripts/painless/_execute
{
  "script": {
    "source": "params.count / params.total",
    "params": {
      "count": 100.0,
      "total": 1000.0
    }
  }
}

Response

edit

{
  "result": "0.1"
}

Filter context

edit

The filter context treats scripts as if they were run inside a script query. For testing purposes, a document must be provided so that it will be temporarily indexed in-memory and is accessible from the script. More precisely, the _source, stored fields and doc values of such a document are available to the script being tested.

Request

edit

PUT /my-index-000001
{
  "mappings": {
    "properties": {
      "field": {
        "type": "keyword"
      }
    }
  }
}

POST /_scripts/painless/_execute
{
  "script": {
    "source": "doc['field'].value.length() <= params.max_length",
    "params": {
      "max_length": 4
    }
  },
  "context": "filter",
  "context_setup": {
    "index": "my-index-000001",
    "document": {
      "field": "four"
    }
  }
}

Response

edit

{
  "result": true
}

Score context

edit

The score context treats scripts as if they were run inside a script_score function in a function_score query.

Request

edit

PUT /my-index-000001
{
  "mappings": {
    "properties": {
      "field": {
        "type": "keyword"
      },
      "rank": {
        "type": "long"
      }
    }
  }
}

POST /_scripts/painless/_execute
{
  "script": {
    "source": "doc['rank'].value / params.max_rank",
    "params": {
      "max_rank": 5.0
    }
  },
  "context": "score",
  "context_setup": {
    "index": "my-index-000001",
    "document": {
      "rank": 4
    }
  }
}

Response

edit

{
  "result": 0.8
}

Field contexts

edit

The field contexts treat scripts as if they were run inside the runtime_mappings section of a search query. You can use field contexts to test scripts for different field types, and then include those scripts anywhere that they’re supported, such as runtime fields.

Choose a field context based on the data type you want to return.

`boolean_field`

edit

Use the boolean_field field context when you want to return a true or false value from a script valuation. Boolean fields accept true and false values, but can also accept strings that are interpreted as either true or false.

Let’s say you have data for the top 100 science fiction books of all time. You want to write scripts that return a boolean response such as whether books exceed a certain page count, or if a book was published after a specific year.

Consider that your data is structured like this:

PUT /my-index-000001
{
  "mappings": {
    "properties": {
      "name": {
        "type": "keyword"
      },
      "author": {
        "type": "keyword"
      },
      "release_date": {
        "type": "date"
      },
      "page_count": {
        "type": "double"
      }
    }
  }
}

You can then write a script in the boolean_field context that indicates whether a book was published before the year 1972:

POST /_scripts/painless/_execute
{
  "script": {
    "source": """
      emit(doc['release_date'].value.year < 1972);
    """
  },
  "context": "boolean_field",
  "context_setup": {
    "index": "my-index-000001",
    "document": {
      "name": "Dune",
      "author": "Frank Herbert",
      "release_date": "1965-06-01",
      "page_count": 604
    }
  }
}

Because Dune was published in 1965, the result returns as true:

{
  "result" : [
    true
  ]
}

Similarly, you could write a script that determines whether the first name of an author exceeds a certain number of characters. The following script operates on the author field to determine whether the author’s first name contains at least one character, but is less than five characters:

POST /_scripts/painless/_execute
{
  "script": {
    "source": """
      int space = doc['author'].value.indexOf(' ');
      emit(space > 0 && space < 5);
    """
  },
  "context": "boolean_field",
  "context_setup": {
    "index": "my-index-000001",
    "document": {
      "name": "Dune",
      "author": "Frank Herbert",
      "release_date": "1965-06-01",
      "page_count": 604
    }
  }
}

Because Frank is five characters, the response returns false for the script valuation:

{
  "result" : [
    false
  ]
}

`date_time`

edit

Several options are available for using using datetime in Painless. In this example, you’ll estimate when a particular author starting writing a book based on its release date and the writing speed of that author. The example makes some assumptions, but shows to write a script that operates on a date while incorporating additional information.

Add the following fields to your index mapping to get started:

PUT /my-index-000001
{
  "mappings": {
    "properties": {
      "name": {
        "type": "keyword"
      },
      "author": {
        "type": "keyword"
      },
      "release_date": {
        "type": "date"
      },
      "page_count": {
        "type": "long"
      }
    }
  }
}

The following script makes the incredible assumption that when writing a book, authors just write each page and don’t do research or revisions. Further, the script assumes that the average time it takes to write a page is eight hours.

The script retrieves the author and makes another fantastic assumption to either divide or multiply the pageTime value based on the author’s perceived writing speed (yet another wild assumption).

The script subtracts the release date value (in milliseconds) from the calculation of pageTime times the page_count to determine approximately (based on numerous assumptions) when the author began writing the book.

POST /_scripts/painless/_execute
{
  "script": {
    "source": """
      String author = doc['author'].value;
      long pageTime = 28800000;  
      if (author == 'Robert A. Heinlein') {
        pageTime /= 2;           
      } else if (author == 'Alastair Reynolds') {
        pageTime *= 2;           
      }
      emit(doc['release_date'].value.toInstant().toEpochMilli() - pageTime * doc['page_count'].value);
    """
  },
  "context": "date_field",
  "context_setup": {
    "index": "my-index-000001",
    "document": {
      "name": "Revelation Space",
      "author": "Alastair Reynolds",
      "release_date": "2000-03-15",
      "page_count": 585
    }
  }
}

	Eight hours, represented in milliseconds
	Incredibly fast writing from Robert A. Heinlein
	Alastair Reynolds writes space operas at a much slower speed

In this case, the author is Alastair Reynolds. Based on a release date of 2000-03-15, the script calculates that the author started writing Revelation Space on 19 February 1999. Writing a 585 page book in just over one year is pretty impressive!

{
  "result" : [
    "1999-02-19T00:00:00.000Z"
  ]
}

`double_field`

edit

Use the double_field context for numeric data of type double. For example, let’s say you have sensor data that includes a voltage field with values like 5.6. After indexing millions of documents, you discover that the sensor with model number QVKC92Q is under reporting its voltage by a factor of 1.7. Rather than reindex your data, you can fix it with a runtime field.

You need to multiply this value, but only for sensors that match a specific model number.

Add the following fields to your index mapping. The voltage field is a sub-field of the measures object.

PUT my-index-000001
{
  "mappings": {
    "properties": {
      "@timestamp": {
        "type": "date"
      },
      "model_number": {
        "type": "keyword"
      },
      "measures": {
        "properties": {
          "voltage": {
            "type": "double"
          }
        }
      }
    }
  }
}

The following script matches on any documents where the model_number equals QVKC92Q, and then multiplies the voltage value by 1.7. This script is useful when you want to select specific documents and only operate on values that match the specified criteria.

POST /_scripts/painless/_execute
{
  "script": {
    "source": """
      if (doc['model_number'].value.equals('QVKC92Q'))
      {emit(1.7 * params._source['measures']['voltage']);}
      else{emit(params._source['measures']['voltage']);}
    """
  },
  "context": "double_field",
  "context_setup": {
    "index": "my-index-000001",
    "document": {
      "@timestamp": 1516470094000,
      "model_number": "QVKC92Q",
      "measures": {
        "voltage": 5.6
      }
    }
  }
}

The result includes the calculated voltage, which was determined by multiplying the original value of 5.6 by 1.7:

{
  "result" : [
    9.52
  ]
}

`geo_point_field`

edit

Geo-point fields accept latitude-longitude pairs. You can define a geo-point field in several ways, and include values for latitude and longitude in the document for your script.

If you already have a known geo-point, it’s simpler to clearly state the positions of lat and lon in your index mappings.

PUT /my-index-000001/
{
  "mappings": {
    "properties": {
      "lat": {
        "type": "double"
      },
      "lon": {
        "type": "double"
      }
    }
  }
}

You can then use the geo_point_field runtime field context to write a script that retrieves the lat and lon values.

POST /_scripts/painless/_execute
{
  "script": {
    "source": """
      emit(doc['lat'].value, doc['lon'].value);
    """
  },
  "context": "geo_point_field",
  "context_setup": {
    "index": "my-index-000001",
    "document": {
      "lat": 41.12,
      "lon": -71.34
    }
  }
}

Because you’re working with a geo-point field type, the response includes results that are formatted as coordinates.

{
  "result" : [
    {
      "coordinates" : [
        -71.34,
        41.12
      ],
      "type" : "Point"
    }
  ]
}

The emit function for geo-point fields takes two parameters ordered with lat before lon, but the output GeoJSON format orders the coordinates as [ lon, lat ].

`ip_field`

edit

The ip_field context is useful for data that includes IP addresses of type ip. For example, let’s say you have a message field from an Apache log. This field contains an IP address, but also other data that you don’t need.

You can add the message field to your index mappings as a wildcard to accept pretty much any data you want to put in that field.

PUT /my-index-000001/
{
  "mappings": {
    "properties": {
      "message": {
        "type": "wildcard"
      }
    }
  }
}

You can then define a runtime script with a grok pattern that extracts structured fields out of the message field.

The script matches on the %{COMMONAPACHELOG} log pattern, which understands the structure of Apache logs. If the pattern matches, the script emits the value matching the IP address. If the pattern doesn’t match (clientip != null), the script just returns the field value without crashing.

POST /_scripts/painless/_execute
{
  "script": {
    "source": """
      String clientip=grok('%{COMMONAPACHELOG}').extract(doc["message"].value)?.clientip;
      if (clientip != null) emit(clientip);
    """
  },
  "context": "ip_field",
  "context_setup": {
    "index": "my-index-000001",
    "document": {
      "message": "40.135.0.0 - - [30/Apr/2020:14:30:17 -0500] \"GET /images/hm_bg.jpg HTTP/1.0\" 200 24736"
    }
  }
}

The response includes only the IP address, ignoring all of the other data in the message field.

{
  "result" : [
    "40.135.0.0"
  ]
}

`keyword_field`

edit

Keyword fields are often used in sorting, aggregations, and term-level queries.

Let’s say you have a timestamp. You want to calculate the day of the week based on that value and return it, such as Thursday. The following request adds a @timestamp field of type date to the index mappings:

PUT /my-index-000001
{
  "mappings": {
    "properties": {
      "@timestamp": {
        "type": "date"
      }
    }
  }
}

To return the equivalent day of week based on your timestamp, you can create a script in the keyword_field runtime field context:

POST /_scripts/painless/_execute
{
  "script": {
    "source": """
      emit(doc['@timestamp'].value.dayOfWeekEnum.getDisplayName(TextStyle.FULL, Locale.ENGLISH));
    """
  },
  "context": "keyword_field",
  "context_setup": {
    "index": "my-index-000001",
    "document": {
      "@timestamp": "2020-04-30T14:31:43-05:00"
    }
  }
}

The script operates on the value provided for the @timestamp field to calculate and return the day of the week:

{
  "result" : [
    "Thursday"
  ]
}

`long_field`

edit

Let’s say you have sensor data that a measures object. This object includes a start and end field, and you want to calculate the difference between those values.

The following request adds a measures object to the mappings with two fields, both of type long:

PUT /my-index-000001/
{
  "mappings": {
    "properties": {
      "measures": {
        "properties": {
          "start": {
            "type": "long"
          },
          "end": {
           "type": "long"
          }
        }
      }
    }
  }
}

You can then define a script that assigns values to the start and end fields and operate on them. The following script extracts the value for the end field from the measures object and subtracts it from the start field:

POST /_scripts/painless/_execute
{
  "script": {
    "source": """
      emit(doc['measures.end'].value - doc['measures.start'].value);
    """
  },
  "context": "long_field",
  "context_setup": {
    "index": "my-index-000001",
    "document": {
      "measures": {
        "voltage": "4.0",
        "start": "400",
        "end": "8625309"
      }
    }
  }
}

The response includes the calculated value from the script valuation:

{
  "result" : [
    8624909
  ]
}

`composite_field`

edit

Let’s say you have logging data with a raw message field which you want to split in multiple sub-fields that can be accessed separately.

The following request adds a message field to the mappings of type keyword:

PUT /my-index-000001/
{
  "mappings": {
    "properties": {
      "message": {
        "type" : "keyword"
      }
    }
  }
}

You can then define a script that splits such message field into subfields using the grok function:

POST /_scripts/painless/_execute
{
  "script": {
    "source": "emit(grok(\"%{COMMONAPACHELOG}\").extract(doc[\"message\"].value));"
  },
  "context": "composite_field",
  "context_setup": {
    "index": "my-index-000001",
    "document": {
      "timestamp":"2020-04-30T14:31:27-05:00",
      "message":"252.0.0.0 - - [30/Apr/2020:14:31:27 -0500] \"GET /images/hm_bg.jpg HTTP/1.0\" 200 24736"
    }
  }
}

The response includes the values that the script emitted:

{
  "result" : {
    "composite_field.timestamp" : [
      "30/Apr/2020:14:31:27 -0500"
    ],
    "composite_field.auth" : [
      "-"
    ],
    "composite_field.response" : [
      "200"
    ],
    "composite_field.ident" : [
      "-"
    ],
    "composite_field.httpversion" : [
      "1.0"
    ],
    "composite_field.verb" : [
      "GET"
    ],
    "composite_field.bytes" : [
      "24736"
    ],
    "composite_field.clientip" : [
      "252.0.0.0"
    ],
    "composite_field.request" : [
      "/images/hm_bg.jpg"
    ]
  }
}

« Painless Debugging Using ingest processors in Painless »