Product

Elasticsearch Python client now supports async I/O

By
Seth Michael Larson

With the increasing popularity of Python web frameworks supporting asynchronous I/O like FastAPI, Starlette, and soon in Django 3.1, there has been a growing demand for native async I/O support in the Python Elasticsearch client.  Async I/O is exciting because your application can use system resources efficiently compared to a traditional multi-threaded application, which leads to better performance on I/O-heavy workloads, like when serving a web application.

The wait is over! Native async / await support is now available in version 7.8.0 of the package, alongside support for all the shiny new Elasticsearch 7.8 APIs. To follow along with the below examples you’ll need Python 3.6 or later installed locally and an Elasticsearch 7.x cluster running.

Installing and using the Elasticsearch Python client

Install the package using the [async] extra to install an additional dependency, Aiohttp, which is used to make HTTP requests to your Elasticsearch instances: 

$ python -m pip install elasticsearch[async]

After being successfully installed, you can now access an async native Elasticsearch client from Python. 

To show async in action, I’ll demonstrate using IPython since an event loop is available right away, allowing me to call await from the REPL. I’ll connect to an Elasticsearch instance hosted in Elastic Cloud using the cloud_id parameter for this quick demonstration: 

$ ipython 
In [1]: from elasticsearch import AsyncElasticsearch 
In [2]: es = AsyncElasticsearch( 
   ...:   cloud_id=”async-example:cGVla2Fib28h...”, 
   ...:   http_auth=(“elastic”, “changeme”) 
   ...: ) 
In [3]: es.info()                                                          
Out[3]: <coroutine object info at 0x7f2a59bb6fc0> 
In [4]: # Returns a coroutine! We need to use 'await'                        
In [5]: await es.info()                                                    
Out[5]: 
{ 
  "name": "instance-0000000000", 
  "cluster_name": "7e6a94f376ed4666a13547cf6431335e", 
  "cluster_uuid": "QrA1xavJQZqnBmBXoLSk5A", 
  "version": { 
 "number": "7.8.0", 
 "build_flavor": "default", 
 "build_type": "docker", 
 "build_hash": "757314695644ea9a1dc2fecd26d1a43856725e65", 
 "build_date": "2020-06-14T19:35:50.234439Z", 
 "build_snapshot": False, 
 "lucene_version": "8.5.1", 
 "minimum_wire_compatibility_version": "6.8.0", 
 "minimum_index_compatibility_version": "6.0.0-beta1" 
  }, 
  "tagline": "You Know, for Search" 
}

Success! 

Start taking advantage of async

There’s additional documentation on the asynchronous client including information on using bulk helpers with async, a section on common scenarios you may encounter, and an example async web application project integrating Elasticsearch and Elastic APM. 

And if you haven’t tried Elastic APM yet, spin up a free trial of Elasticsearch Service today, instrument the APM Python agent, and make sure your application is spending its time where it should be.

If you’re interested in learning more about how the Python Elasticsearch client works, you can tune into the upcoming Introduction into the Python Elasticsearch Client webinar on August 5th that’ll include the basics, a discussion of the architecture, and lots of time for Q&A.