Creating API objects from JSON data

This feature was added in version 7.17.2

A common workflow during application development with Elasticsearch is to use the Kibana Developer Console to interactively prepare and test queries, aggregations, index mappings and other complex API calls. This results in working JSON snippets that you may want to use in your application.

As translating these JSON snippets to Java code can be time-consuming and error-prone, most of the data classes in the Java API Client can be loaded from JSON text: object builders have withJson() methods that populate the builder from raw JSON. This also allows you to combine dynamically loaded JSON with programmatic construction of objects.

Under the hood, the withJson() methods call the object’s deserializer. The JSON text’s structure and value types therefore have to be correct for the target data structure. Using withJson() keeps the strong typing guarantees of the Java API Client.

Examples

edit

Loading an index definition from a resource file

edit

Consider a resource file some-index.json containing an index definition:

{
  "mappings": {
    "properties": {
      "field1": { "type": "text" }
    }
  }
}

You can create an index from that definition as follows:

InputStream input = this.getClass()
    .getResourceAsStream("some-index.json"); 

CreateIndexRequest req = CreateIndexRequest.of(b -> b
    .index("some-index")
    .withJson(input) 
);

boolean created = client.indices().create(req).acknowledged();

	open an input stream for the JSON resource file.
	populate the index creation request with the resource file contents.

Ingesting documents from JSON files

edit

Similarly, you can read documents to be stored in Elasticsearch from data files:

FileReader file = new FileReader(new File(dataDir, "document-1.json"));

IndexRequest<JsonData> req; 

req = IndexRequest.of(b -> b
    .index("some-index")
    .withJson(file)
);

client.index(req);

when calling withJson() on data structures that have generic type parameters, these generic types will be considered to be JsonData.

Creating a search request combining JSON and programmatic construction

edit

You can combine withJson() with regular calls to setter methods. The example below loads the query part of a search request from a String and programmatically adds an aggregation.

Reader queryJson = new StringReader(
    "{" +
    "  \"query\": {" +
    "    \"range\": {" +
    "      \"@timestamp\": {" +
    "        \"gt\": \"now-1w\"" +
    "      }" +
    "    }" +
    "  }" +
    "}");

SearchRequest aggRequest = SearchRequest.of(b -> b
    .withJson(queryJson) 
    .aggregations("max-cpu", a1 -> a1 
        .dateHistogram(h -> h
            .field("@timestamp")
            .calendarInterval(CalendarInterval.Hour)
        )
        .aggregations("max", a2 -> a2
            .max(m -> m.field("host.cpu.usage"))
        )
    )
    .size(0)
);

Map<String, Aggregate> aggs = client
    .search(aggRequest, Void.class) 
    .aggregations();

	loads the query from the JSON string.
	adds the aggregation.
	since this is an aggregation we don’t care about result documents and set their target class to `Void`, meaning they will just be ignored. Note that setting `size` to zero actually prevents any document from being returned.

Creating a search request from multiple JSON snippets

edit

The withJson() methods are partial deserializers: the properties loaded from the JSON will set property values or replace the previous ones, but will not reset other properties not found in the JSON input. You can use this to combine multiple JSON snippets to build complex search requests. In the example below, we combine separate definitions of a query that selects some documents and an aggregation that is run on the results of this query.

Reader queryJson = new StringReader(
    "{" +
    "  \"query\": {" +
    "    \"range\": {" +
    "      \"@timestamp\": {" +
    "        \"gt\": \"now-1w\"" +
    "      }" +
    "    }" +
    "  }," +
    "  \"size\": 100" + 
    "}");

Reader aggregationJson = new StringReader(
    "{" +
    "  \"size\": 0, " + 
    "  \"aggregations\": {" +
    "    \"hours\": {" +
    "      \"date_histogram\": {" +
    "        \"field\": \"@timestamp\"," +
    "        \"interval\": \"hour\"" +
    "      }," +
    "      \"aggregations\": {" +
    "        \"max-cpu\": {" +
    "          \"max\": {" +
    "            \"field\": \"host.cpu.usage\"" +
    "          }" +
    "        }" +
    "      }" +
    "    }" +
    "  }" +
    "}");

SearchRequest aggRequest = SearchRequest.of(b -> b
    .withJson(queryJson) 
    .withJson(aggregationJson) 
    .ignoreUnavailable(true) 
);

Map<String, Aggregate> aggs = client
    .search(aggRequest, Void.class)
    .aggregations();

	set max number of returned document to 100 for queries.
	we do not want any matching document in aggregations.
	loads the query part of the request.
	loads the aggregation part of the request (overwrites `size` from the query).
	additional request properties set programmatically.

Notice that order matters when the JSON snippets have some common properties: just as when setting property values programmatically, the last value that is set for a property overwrites the previous one.

The source code for the examples above can be found in the Java API Client tests.

« Object life cycles and thread safety Exceptions »