Manage data views

To customize the data fields in your data view, you can add runtime fields to the existing documents, add scripted fields to compute data on the fly, and change how Kibana displays the data fields.

Runtime fields are fields that you add to documents after you’ve ingested your data, and are evaluated at query time. With runtime fields, you allow for a smaller index and faster ingest time so that you can use less resources and reduce your operating costs. You can use runtime fields anywhere data views are used, for example, you can explore runtime fields in Discover and create visualizations with runtime fields for your dashboard.

With runtime fields, you can:

For detailed information on how to use runtime fields with Elasticsearch, refer to Runtime fields.

To add runtime fields to your data views, open the data view you want to change, then define the field values by emitting a single value using the Painless scripting language. You can also add runtime fields in Discover and Lens.

Try the runtime field examples on your own using the Sample web logs data.

Return Hello World!:

emit("Hello World!");

Calculate kilobytes from bytes:

emit(doc['bytes'].value / 1024)

Return the string that appears after the last slash in the URL:

def path = doc["url.keyword"].value;
if (path != null) {
    int lastSlashIndex = path.lastIndexOf('/');
    if (lastSlashIndex > 0) {
        emit(path.substring(lastSlashIndex+1));
    return;
    }
}
emit("");

A single runtime field can also produce multiple subfields when the type Composite is selected. The script editor provides default types that can be customized for each subfields.

Return keyword and double type subfields. Note that the first argument for emit is the name of the subfield.

emit('subfield_a', 'Hello');
emit('subfield_b', 42);

Replace null values with None:

def source = doc['referer'].value;
if (source != null) {
	emit(source);
	return;
}
else {
	emit("None");
}

Specify the operating system condition:

def source = doc['machine.os.keyword'].value;
if (source != "") {
	emit(source);
}
else {
	emit("None");
}

Edit the settings for runtime fields, or remove runtime fields from data views.

Scripted fields compute data on the fly from the data in your Elasticsearch indices. The data is shown on the Discover tab as part of the document data, and you can use scripted fields in your visualizations. You query scripted fields with the Kibana query language, and can filter them using the filter bar. The scripted field values are computed at query time, so they aren’t indexed and cannot be searched using the Kibana default query language.

When you define a scripted field in Kibana, you have a choice of the Lucene expressions or the Painless scripting language.

You can reference any single value numeric field in your expressions, for example:

doc['field_name'].value

For more information on scripted fields and additional examples, refer to Using Painless in Kibana scripted fields

The following code snippets demonstrate how an example scripted field called computed_values on the Kibana Sample Data Logs data view could be migrated to either a runtime field or an ES|QL query, highlighting the differences between each approach.

In the scripted field example, variables are created to track all values the script will need to access or return. Since scripted fields can only return a single value, the created variables must be returned together as an array at the end of the script.

def hour_of_day = $('@timestamp', ZonedDateTime.parse('1970-01-01T00:00:00Z')).getHour();
def time_of_day = '';

if (hour_of_day >= 22 || hour_of_day < 5)
  time_of_day = 'Night';
else if (hour_of_day < 12)
  time_of_day = 'Morning';
else if (hour_of_day < 18)
  time_of_day = 'Afternoon';
else
  time_of_day = 'Evening';

def response_int = Integer.parseInt($('response.keyword', '200'));
def response_category = '';

if (response_int < 200)
  response_category = 'Informational';
else if (response_int < 300)
  response_category = 'Successful';
else if (response_int < 400)
  response_category = 'Redirection';
else if (response_int < 500)
  response_category = 'Client Error';
else
  response_category = 'Server Error';

return [time_of_day, response_category];

Unlike scripted fields, runtime fields do not need to return a single value and can emit values at any point in the script, which will be combined and returned as a multi-value field. This allows for more flexibility in the script logic and removes the need to manually manage an array of values.

def hour_of_day = $('@timestamp', ZonedDateTime.parse('1970-01-01T00:00:00Z')).getHour();

if (hour_of_day >= 22 || hour_of_day < 5)
  emit('Night');
else if (hour_of_day < 12)
  emit('Morning');
else if (hour_of_day < 18)
  emit('Afternoon');
else
  emit('Evening');

def response_int = Integer.parseInt($('response.keyword', '200'));

if (response_int < 200)
  emit('Informational');
else if (response_int < 300)
  emit('Successful');
else if (response_int < 400)
  emit('Redirection');
else if (response_int < 500)
  emit('Client Error');
else
  emit('Server Error');

Alternatively, ES|QL can be used to skip the need for data view management entirely and simply compute the values you need at query time. ES|QL supports computing multiple field values in a single query, using computed values with its rich set of commands and functions, and even aggregations against computed values. This makes it an excellent solution for one-off queries and realtime data analysis.

FROM kibana_sample_data_logs
  | EVAL hour_of_day = DATE_EXTRACT("HOUR_OF_DAY", @timestamp)
  | EVAL time_of_day = CASE(
      hour_of_day >= 22 OR hour_of_day < 5, "Night",
      hour_of_day < 12, "Morning",
      hour_of_day < 18, "Afternoon",
      "Evening"
    )
  | EVAL response_int = TO_INTEGER(response)
  | EVAL response_category = CASE(
      response_int < 200, "Informational",
      response_int < 300, "Successful",
      response_int < 400, "Redirection",
      response_int < 500, "Client Error",
      "Server Error"
    )
  | EVAL computed_values = MV_APPEND(time_of_day, response_category)
  | DROP hour_of_day, time_of_day, response_int, response_category

For more information about scripted fields in Elasticsearch, refer to Scripting.

Kibana uses the same field types as Elasticsearch, however, some Elasticsearch field types are unsupported in Kibana. To customize how Kibana displays data fields, use the formatting options.

String fields support String and Url formatters.

The String field formatter enables you to apply transforms to the field.

Supported transformations include:

You can specify the following types to the Url field formatter:

To customize URL field formats, use templates. An URL template enables you to add values to a partial URL. To add the contents of the field to a fixed URL, use the {{value}} string.

For example, when:

The resulting URL replaces {{value}} with the user ID from the field.

The {{value}} template string URL-encodes the contents of the field. When a field encoded into a URL contains non-ASCII characters, the characters are replaced with a % character and the appropriate hexadecimal code. For example, field contents users/admin result in the URL template adding users%2Fadmin.

When the formatter type is Image, the {{value}} template string specifies the name of an image at the specified URI.

When the formatter type is Audio, the {{value}} template string specifies the name of an audio file at the specified URI.

To pass unescaped values directly to the URL, use the {{rawValue}} string.

A Label template enables you to specify a text string that appears instead of the raw URL. You can use the {{value}} template string normally in label templates. You can also use the {{url}} template string to display the formatted URL.

Date fields support Date, String, and Url formatters.

The Date formatter enables you to choose the display format of date stamps using the moment.js standard format definitions.

The String field formatter enables you to apply transforms to the field.

Supported transformations include:

You can specify the following types to the Url field formatter:

To customize URL field formats, use templates. An URL template enables you to add values to a partial URL. To add the contents of the field to a fixed URL, use the {{value}} string.

For example, when:

The resulting URL replaces {{value}} with the user ID from the field.

The {{value}} template string URL-encodes the contents of the field. When a field encoded into a URL contains non-ASCII characters, the characters are replaced with a % character and the appropriate hexadecimal code. For example, field contents users/admin result in the URL template adding users%2Fadmin.

When the formatter type is Image, the {{value}} template string specifies the name of an image at the specified URI.

When the formatter type is Audio, the {{value}} template string specifies the name of an audio file at the specified URI.

To pass unescaped values directly to the URL, use the {{rawValue}} string.

A Label template enables you to specify a text string that appears instead of the raw URL. You can use the {{value}} template string normally in label templates. You can also use the {{url}} template string to display the formatted URL.

Geographic point fields support the String formatter.

The String field formatter enables you to apply transforms to the field.

Supported transformations include:

Numeric fields support Bytes, Color, Duration, Histogram, Number, Percentage, String, and Url formatters.

The Bytes, Number, and Percentage formatters enable you to choose the display formats of numbers in the field using the Elastic numeral pattern syntax that Kibana maintains.

The Histogram formatter is used only for the histogram field type. When you use the Histogram formatter, you can apply the Bytes, Number, or Percentage format to aggregated data.

You can specify the following types to the Url field formatter:

To customize URL field formats, use templates. An URL template enables you to add values to a partial URL. To add the contents of the field to a fixed URL, use the {{value}} string.

For example, when:

The resulting URL replaces {{value}} with the user ID from the field.

The {{value}} template string URL-encodes the contents of the field. When a field encoded into a URL contains non-ASCII characters, the characters are replaced with a % character and the appropriate hexadecimal code. For example, field contents users/admin result in the URL template adding users%2Fadmin.

When the formatter type is Image, the {{value}} template string specifies the name of an image at the specified URI.

When the formatter type is Audio, the {{value}} template string specifies the name of an audio file at the specified URI.

To pass unescaped values directly to the URL, use the {{rawValue}} string.

A Label template enables you to specify a text string that appears instead of the raw URL. You can use the {{value}} template string normally in label templates. You can also use the {{url}} template string to display the formatted URL.

The String field formatter enables you to apply transforms to the field.

Supported transformations include:

The Duration field formatter displays the numeric value of a field in the following increments:

You can specify these increments with up to 20 decimal places for input and output formats.

The Color field formatter enables you to specify colors with ranges of values for a number field.

When you select the Color formatter, click Add Color, then specify the Range, Text color, and Background color.