Elasticsearch .NET client evolution: From NEST to Elastic.Clients.Elasticsearch

Introduction to .NET client and NEST

In the .NET world, integration with Elasticsearch has long been facilitated by the NEST library, which serves as a robust interface for developers to interact with Elasticsearch's powerful search and analytics capabilities. NEST, born out of a need for a native .NET client for Elasticsearch, quickly gained popularity among developers for its rich feature set and seamless integration capabilities.

For nearly 14 years and only 8 months after Elasticsearch's first commit NEST has been faithfully tracking Elasticsearch releases.

Transitioning from NEST to Elastic.Clients.Elasticsearch

As Elasticsearch evolved, maintaining NEST's complex codebase became increasingly difficult. We recognized the need for a more sustainable approach to client development and went on a journey to redesign the .NET client from the ground up. It took us almost a year to release a first beta version and another year to get close to supporting every single server endpoint. One of the most difficult decisions was to reduce the scope of the library in order to prioritize maintainability instead.

Given the size of the Elasticsearch API surface today, it is no longer practical to maintain over 450 endpoints and nearly 3000 types (requests, responses, queries, aggregations, etc.) by hand. To ensure consistent, accurate, and timely alignment between language clients and Elasticsearch, the 8.x clients, and many of the associated types are now automatically code-generated from a shared specification. This is a common solution to maintaining alignment between client and server among SDKs and libraries, such as those for Azure, AWS and the Google Cloud Platform.

The Elasticsearch specification was created over 8 years ago by exporting the type mappings from NEST and through the hard work of the clients team we can now use the same specification to create a new .NET client (and clients for multiple other languages like Java, Go, etc.).

With the release of version 8.13, the deprecation of NEST was officially announced. As Elasticsearch transitions to Elastic.Clients.Elasticsearch, NEST will gradually phase out, reaching its end-of-life at the close of the year. Developers are strongly encouraged to commence migration efforts early to ensure a smooth transition and mitigate any potential disruptions. Embracing Elastic.Clients.Elasticsearch not only ensures compatibility with the latest server features but also future-proofs applications against deprecated functionality.

Elastic.Clients.Elasticsearch: features and changes overview

Switching to the v8 client Elastic.Clients.Elasticsearch enables access to all the new features of Elasticsearch 8 and also brings numerous modernizations to the library itself but also implies a reduction in convenience features compared to its predecessor. Some of the new core features include the query language ES|QL, modern machine learning (ML) capabilities and improved diagnostics in the form of OpenTelemetry-compatible activities. Starting with version 8.13, Elastic.Clients.Elasticsearch supports almost all server features of Elasticsearch 8.

An important breaking change, for example, is related to aggregations. In NEST, the fluent API usage looks like this:

s => s
.Aggregations(aggs => aggs
    .Children<CommitActivity>("name_of_child_agg", child => child
        .Aggregations(childAggs => childAggs
            .Average("average_per_child", avg => avg.Field(p => p.ConfidenceFactor))
            .Max("max_per_child", max => max.Field(p => p.ConfidenceFactor))
            .Min("min_per_child", min => min.Field(p => p.ConfidenceFactor))
        )
    )
)

while the v8 client requires the following syntax:

s => s
.Aggregations(aggs => aggs
	.Add("name_of_child_agg", agg => agg
		.Children(_ => {})
		.Aggregations(childAggs => childAggs
			.Add("average_per_child", agg => agg.Avg(avg => avg.Field(p => p.ConfidenceFactor)))
			.Add("max_per_child", agg => agg.Max(max => max.Field(p => p.ConfidenceFactor)))
			.Add("min_per_child", agg => agg.Min(min => min.Field(p => p.ConfidenceFactor)))
		)
	)
)

Migrating from NEST v7 to .NET client v8

A comprehensive migration guide is available here: Migration guide: From NEST v7 to .NET Client v8.

Additional resources

Want to get Elastic certified? Find out when the next Elasticsearch Engineer training is running!

Elasticsearch is packed with new features to help you build the best search solutions for your use case. Dive into our sample notebooks to learn more, start a free cloud trial, or try Elastic on your local machine now.