API conventions

edit

The Java API Client uses a very consistent code structure, using modern code patterns that make complex requests easier to write and complex responses easier to process. This page explains these so that you quickly feel at home.

Package structure and namespace clients

edit

The Elasticsearch API is large and is organized into feature groups, as can be seen in the Elasticsearch API documentation.

The Java API Client follows this structure: feature groups are called “namespaces”, and each namespace is located in a subpackage of co.elastic.clients.elasticsearch.

Each of the namespace clients can be accessed from the top level Elasticsearch client. The only exceptions are the “search” and “document” APIs which are located in the core subpackage and can be accessed on the main Elasticsearch client object.

The snippet below shows how to use the indices namespace client to create an index (the lambda syntax is explained below):

// Create the "products" index
ElasticsearchClient client = ...
client.indices().create(c -> c.index("products"));

Namespace clients are very lightweight objects that can be created on the fly.

Method naming conventions

edit

Classes in the Java API Client contain two kinds of methods and properties:

  • Methods and properties that are part of the API, such as ElasticsearchClient.search() or SearchResponse.maxScore(). They are derived from their respective names in the Elasticsearch JSON API using the standard Java camelCaseNaming convention.
  • Methods and properties that are part of the framework on which the Java API Client is built, such as Query._kind(). These methods and properties are prefixed with an underscore to both avoid any naming conflicts with API names, and as an easy way to distinguish the API from the framework.

Immutable objects, builders and builder lambdas

edit

All data types in the Java API Client are immutable. Object creation uses the builder pattern that was popularized in Effective Java in 2008.

ElasticsearchClient client = ...
CreateIndexResponse createResponse = client.indices().create(
    new CreateIndexRequest.Builder()
        .index("my-index")
        .aliases("foo",
            new Alias.Builder().isWriteIndex(true).build()
        )
        .build()
);

Note that a builder should not be reused after its build() method has been called.

Although this works nicely, having to instantiate builder classes and call the build() method is a bit verbose. So every property setter in the Java API Client also accepts a lambda expression that takes a newly created builder as a parameter and returns a populated builder. The snippet above can also be written as:

ElasticsearchClient client = ...
CreateIndexResponse createResponse = client.indices()
    .create(createIndexBuilder -> createIndexBuilder
        .index("my-index")
        .aliases("foo", aliasBuilder -> aliasBuilder
            .isWriteIndex(true)
        )
    );

This approach allows for much more concise code, and also avoids importing classes (and even remembering their names) since types are inferred from the method parameter signature.

Note in the above example that builder variables are only used to start a chain of property setters. The names of these variables are therefore unimportant and can be shortened to improve readability:

ElasticsearchClient client = ...
CreateIndexResponse createResponse = client.indices()
    .create(c -> c
        .index("my-index")
        .aliases("foo", a -> a
            .isWriteIndex(true)
        )
    );

Builder lambdas become particularly useful with complex nested queries like the one below, taken from the intervals query API documentation.

This example also highlights a useful naming convention for builder parameters in deeply nested structures. For lambda expressions with a single argument, Kotlin provides the implicit it parameter and Scala allows use of _. This can be approximated in Java by using an underscore prefix followed by a number representing the depth level (i.e. _0, _1, and so on). Not only does this remove the need to create throw-away variable names, but it also improves code readability. Correct indentation also allows the structure of the query to stand out.

ElasticsearchClient client = ...
SearchResponse<SomeApplicationData> results = client
    .search(_0 -> _0
        .query(_1 -> _1
            .intervals(_2 -> _2
                .field("my_text")
                .allOf(_3 -> _3
                    .ordered(true)
                    .intervals(_4 -> _4
                        .match(_5 -> _5
                            .query("my favorite food")
                            .maxGaps(0)
                            .ordered(true)
                        )
                    )
                    .intervals(_4 -> _4
                        .anyOf(_5 -> _5
                            .intervals(_6 -> _6
                                .match(_7 -> _7
                                    .query("hot water")
                                )
                            )
                            .intervals(_6 -> _6
                                .match(_7 -> _7
                                    .query("cold porridge")
                                )
                            )
                        )
                    )
                )
            )
        ),
    SomeApplicationData.class 
);

Search results will be mapped to SomeApplicationData instances to be readily available to the application.

Lists and maps

edit

Additive builder setters

edit

Properties of type List and Map are exposed by object builders as a set of overloaded additive-only methods that update the property value, by appending to lists and adding new entries to maps (or replacing existing ones).

Object builders create immutable objects, and this also applies to list and map properties that are made immutable at object construction time.

// Prepare a list of index names
List<String> names = Arrays.asList("idx-a", "idx-b", "idx-c");

// Prepare cardinality aggregations for fields "foo" and "bar"
Map<String, Aggregation> cardinalities = new HashMap<>();
cardinalities.put("foo-count", Aggregation.of(a -> a.cardinality(c -> c.field("foo"))));
cardinalities.put("bar-count", Aggregation.of(a -> a.cardinality(c -> c.field("bar"))));

// Prepare an aggregation that computes the average of the "size" field
final Aggregation avgSize = Aggregation.of(a -> a.avg(v -> v.field("size")));

SearchRequest search = SearchRequest.of(r -> r
    // Index list:
    // - add all elements of a list
    .index(names)
    // - add a single element
    .index("idx-d")
    // - add a vararg list of elements
    .index("idx-e", "idx-f", "idx-g")

    // Sort order list: add elements defined by builder lambdas
    .sort(s -> s.field(f -> f.field("foo").order(SortOrder.Asc)))
    .sort(s -> s.field(f -> f.field("bar").order(SortOrder.Desc)))

    // Aggregation map:
    // - add all entries of an existing map
    .aggregations(cardinalities)
    // - add a key/value entry
    .aggregations("avg-size", avgSize)
    // - add a key/value defined by a builder lambda
    .aggregations("price-histogram",
        a -> a.histogram(h -> h.field("price")))
);

List and map values are never null

edit

The Elasticsearch API has a lot of optional properties. For single-valued properties, the Java API Client represents missing optional values as null. Applications therefore have to null-check optional values before using them.

For lists and maps however, applications often only care about whether they’re empty or not, or even just iterate on their content. Using null values is then cumbersome. To avoid this, Java API Client collection properties are never null, and missing optional collections are returned as an empty collection.

If you ever need to distinguish between a missing (undefined) optional collection and an effectively-empty collection returned by Elasticsearch, the ApiTypeHelper class provides a utility method to distinguish them:

NodeStatistics stats = NodeStatistics.of(b -> b
    .total(1)
    .failed(0)
    .successful(1)
);

// The `failures` list was not provided.
// - it's not null
assertNotNull(stats.failures());
// - it's empty
assertEquals(0, stats.failures().size());
// - and if needed we can know it was actually not defined
assertFalse(ApiTypeHelper.isDefined(stats.failures()));

Variant types

edit

The Elasticsearch API has a lot of variant types: queries, aggregations, field mappings, analyzers, and so on. Finding the correct class name in such large collections can be challenging.

The Java API Client builders make this easy: the builders for variant types, such as Query, have methods for each of the available implementations. We’ve seen this in action above with intervals (a kind of query) and allOf, match and anyOf (various kinds of intervals).

This is because variant objects in the Java API Client are implementations of a “tagged union”: they contain the identifier (or tag) of the variant they hold and the value for that variant. For example, a Query object can contain an IntervalsQuery with tag intervals, a TermQuery with tag term, and so on. This approach allows writing fluent code where you can let the IDE completion features guide you to build and navigate complex nested structures:

Variant builders have setter methods for every available implementation. They use the same conventions as regular properties and accept both a builder lambda expression and a ready-made object of the actual type of the variant. Here’s an example to build a term query:

Query query = new Query.Builder()
    .term(t -> t                          
        .field("name")                    
        .value(v -> v.stringValue("foo"))
    )
    .build();                             

Choose the term variant to build a term query.

Build the terms query with a builder lambda expression.

Build the Query that now holds a TermQuery object of kind term.

Variant objects have getter methods for every available implementation. These methods check that the object actually holds a variant of that kind and return the value downcasted to the correct type. They throw an IllegalStateException otherwise. This approach allows writing fluent code to traverse variants.

assertEquals("foo", query.term().value().stringValue());

Variant objects also provide information on the variant kind they currently hold:

  • with is methods for each of the variant kinds: isTerm(), isIntervals(), isFuzzy(), etc.
  • with a nested Kind enumeration that defines all variant kinds.

This information can then be used to navigate down into specific variants after checking their actual kind:

if (query.isTerm()) { 
    doSomething(query.term());
}

switch(query._kind()) { 
    case Term:
        doSomething(query.term());
        break;
    case Intervals:
        doSomething(query.intervals());
        break;
    default:
        doSomething(query._kind(), query._get()); 
}

Test if the variant is of a specific kind.

Test a larger set of variant kinds.

Get the kind and value held by the variant object.

Blocking and asynchronous clients

edit

API clients come in two flavors: blocking and asynchronous. All methods on asynchronous clients return a standard CompletableFuture.

Both flavors can be used at the same time depending on your needs, sharing the same transport object:

ElasticsearchTransport transport = ...

// Synchronous blocking client
ElasticsearchClient client = new ElasticsearchClient(transport);

if (client.exists(b -> b.index("products").id("foo")).value()) {
    logger.info("product exists");
}

// Asynchronous non-blocking client
ElasticsearchAsyncClient asyncClient =
    new ElasticsearchAsyncClient(transport);

asyncClient
    .exists(b -> b.index("products").id("foo"))
    .thenAccept(response -> {
        if (response.value()) {
            logger.info("product exists");
        }
    });

Exceptions

edit

Client methods can throw two kinds of exceptions:

  • Requests that were received by the Elasticsearch server but that were rejected (validation error, server internal timeout exceeded, etc) will produce an ElasticsearchException. This exception contains details about the error, provided by Elasticsearch.
  • Requests that failed to reach the server (network error, server unavailable, etc) will produce a TransportException. That exception’s cause is the exception thrown by the lower-level implementation. In the case of the RestClientTransport it will be a ResponseException that contains the low level HTTP response.

Object life cycles

edit

There are five kinds of objects in the Java API Client with different life cycles:

Object mapper
Stateless and thread-safe, but can be costly to create. It’s usually a singleton that is created at application startup and used to create the transport.
Transport
Thread-safe, holds network resources through the underlying HTTP client. A transport object is associated with an Elasticsearch cluster and has to be explicitly closed to release the underlying resources such as network connections.
Clients
Immutable, stateless and thread-safe. These are very lightweight objects that just wrap a transport and provide API endpoints as methods.
Builders
Mutable, non thread-safe. Builders are transient objects that should not be reused after calling build().
Requests & other API objects
Immutable, thread-safe. If your application uses the same request or same parts of a request over and over, these objects can be prepared in advance and reused across multiple calls over multiple clients with different transports.