Ingest processor context

edit

Ingest processor context

edit

Use a Painless script in an ingest processor to modify documents upon insertion.

Variables

params (Map, read-only): User-defined parameters passed in as part of the query.
ctx['_index'] (String): The name of the index.
ctx (Map): Contains extracted JSON in a Map and List structure for the fields that are part of the document.

Side Effects

ctx['_index']: Modify this to change the destination index for the current document.
ctx (Map): Modify the values in the Map/List structure to add, modify, or delete the fields of a document.

Return

void: No expected return value.

API

Both the standard Painless API and Specialized Ingest API are available.

Example

To run this example, first follow the steps in context examples.

The seat data contains:

A date in the format YYYY-MM-DD where the second digit of both month and day is optional.
A time in the format HH:MM* where the second digit of both hours and minutes is optional. The star (*) represents either the String AM or PM.

The following ingest script processes the date and time Strings and stores the result in a datetime field.

String[] dateSplit = ctx.date.splitOnToken("-");                     
String year = dateSplit[0].trim();
String month = dateSplit[1].trim();

if (month.length() == 1) {                                           
    month = "0" + month;
}

String day = dateSplit[2].trim();

if (day.length() == 1) {                                             
    day = "0" + day;
}

boolean pm = ctx.time.substring(ctx.time.length() - 2).equals("PM"); 
String[] timeSplit = ctx.time.substring(0,
        ctx.time.length() - 2).splitOnToken(":");                    
int hours = Integer.parseInt(timeSplit[0].trim());
int minutes = Integer.parseInt(timeSplit[1].trim());

if (pm) {                                                            
    hours += 12;
}

String dts = year + "-" + month + "-" + day + "T" +
        (hours < 10 ? "0" + hours : "" + hours) + ":" +
        (minutes < 10 ? "0" + minutes : "" + minutes) +
        ":00+08:00";                                                 

ZonedDateTime dt = ZonedDateTime.parse(
         dts, DateTimeFormatter.ISO_OFFSET_DATE_TIME);               
ctx.datetime = dt.getLong(ChronoField.INSTANT_SECONDS)*1000L;

	Uses the `splitOnToken` function to separate the date `String` from the seat data into year, month, and day `Strings`. Note The use of the `ctx` ingest processor context variable to retrieve the data from the `date` field.
	Appends the string literal `"0"` value to a single digit month since the format of the seat data allows for this case.
	Appends the string literal `"0"` value to a single digit day since the format of the seat data allows for this case.
	Sets the `boolean type` variable to `true` if the time `String` is a time in the afternoon or evening. Note The use of the `ctx` ingest processor context variable to retrieve the data from the `time` field.
	Uses the `splitOnToken` function to separate the time `String` from the seat data into hours and minutes `Strings`. Note The use of the `substring` method to remove the `AM` or `PM` portion of the time `String`. The use of the `ctx` ingest processor context variable to retrieve the data from the `date` field.
	If the time `String` is an afternoon or evening value adds the integer literal `12` to the existing hours to move to a 24-hour based time.
	Builds a new time `String` that is parsable using existing API methods.
	Creates a `ZonedDateTime` reference type value by using the API method `parse` to parse the new time `String`.
	Sets the datetime field `datetime` to the number of milliseconds retrieved from the API method `getLong`. Note The use of the `ctx` ingest processor context variable to set the field `datetime`. Manipulate each document’s fields with the `ctx` variable as each document is indexed.

Submit the following request:

PUT /_ingest/pipeline/seats
{
  "description": "update datetime for seats",
  "processors": [
    {
      "script": {
        "source": "String[] dateSplit = ctx.date.splitOnToken('-'); String year = dateSplit[0].trim(); String month = dateSplit[1].trim(); if (month.length() == 1) { month = '0' + month; } String day = dateSplit[2].trim(); if (day.length() == 1) { day = '0' + day; } boolean pm = ctx.time.substring(ctx.time.length() - 2).equals('PM'); String[] timeSplit = ctx.time.substring(0, ctx.time.length() - 2).splitOnToken(':'); int hours = Integer.parseInt(timeSplit[0].trim()); int minutes = Integer.parseInt(timeSplit[1].trim()); if (pm) { hours += 12; } String dts = year + '-' + month + '-' + day + 'T' + (hours < 10 ? '0' + hours : '' + hours) + ':' + (minutes < 10 ? '0' + minutes : '' + minutes) + ':00+08:00'; ZonedDateTime dt = ZonedDateTime.parse(dts, DateTimeFormatter.ISO_OFFSET_DATE_TIME); ctx.datetime = dt.getLong(ChronoField.INSTANT_SECONDS)*1000L;"
      }
    }
  ]
}

« Runtime fields context Update context »