DevRel newsletter — April 2024

Hello from the Elastic DevRel team! Welcome to our latest newsletter blog edition, where you'll find information on upcoming events in your region, catch up on content, and stay up to date with product updates. In this edition, we’ll cover the release of Elasticsearch and Elastic Stack 8.13, which are packed with significant enhancements that are set to elevate your Elastic experience.

Elasticsearch and the Elastic Stack 8.13 are available

The 8.13 release brings performance improvements in Elasticsearch and Kibana, Apache Lucene 9.10, and new features:

• Improved query parallelization by communicating between threads on different segments (Large data sets with multiple segments and high dimensionality will likely experience 2x to 3x improvements.)

• Simplified kNN vector search by making k and num_candidates optional in the kNN section

• int8_flat and flat vector field index types for scenarios where brute force is the better choice than HNSW

• Multiple results from the same document with nested vectors for documents that have many vectors per field per document

• Detect data streams that are stuck in their data stream lifecycle with the DSL health indicator

• Detect misconfigured or invalid repositories with the enhanced repository_integrity indicator

• ES|QL in the Java client

• ES|QL support in the Data Visualizer

• Anomaly detection and AIOps: usability enhancements

• Elastic Integration Filter allows you to transfer the execution of integration-specific transformations from Elasticsearch to Logstash before sending them to Elasticsearch

• Elastic Agent support for Kafka is now GA

Elastic Search 8.13 includes an expanded toolkit for relevance including the GA of new vector search configurations, the GA of the synonyms API, and the tech preview of Learning to Rank. It also offers enhanced tools for managing connectors to sync private data for search applications, the tech preview of Elastic’s integration with Cohere, and an ELSER query optimization.

Elastic Observability 8.13 includes AI Assistant enhancements, such as the GA of support for Amazon Bedrock and the tech preview of editing and visualizing ES|QL queries, in addition to the GA of SLO enhancements.

Elastic Security 8.13 includes a refined Cloud Security Posture Management (CSPM) benchmark rules experience, automated endpoint response actions, and a suite of enhancements to help users continue to accelerate their security program.

Upgrade to Elastic Stack 8.13.2, which contains important security and bug fixes or to 7.17.20 if you’re still on the previous major version.

Cohere integration: Elasticsearch’s inference API adds support for Cohere embeddings, and the seamless integration lets you use Cohere’s Rerank 3 model on any existing Elasticsearch retrieval flow without requiring any significant code changes.

Text expansion performance: Follow Kathleen DeRusso to learn how to improve text expansion performance in ELSER using token pruning, which results in a 3–4x improvement in 99th percentile latency.

ES|QL in Python and PHP: The Elasticsearch Query Language (ES|QL) provides a powerful way to filter, transform, and analyze data stored in Elasticsearch. Learn how to get from ES|QL to Pandas dataframes in Python and from ES|QL to PHP objects. 

Retrieval augmented generation (RAG): Srikanth Manvi uses LlamaIndex and a locally running Mistral LLM to implement a Q&A experience. Ashish Tiwari builds a RAG system with Google’s Gemma and Hugging Face.

Scaling ML inference pipelines: Iulia Feroli takes a look at potential bottlenecks and ways to mitigate growing pains around Elasticsearch inference pipelines when dealing with more complex workloads, from downloading the model to monitoring it.

Enrichment of documents: Depending on your use case, you can decide to do it within Elasticsearch through ingest pipelines or runtime fields, centralized in Logstash, or from the edge with Elastic Agent. Follow along with David Pilato’s three-part series.

Elastic distributions for OpenTelemetry SDKs: Steve Gordon shares Elastic’s plans to expand adoption for and commitment to OpenTelemetry with the introduction of Elastic distributions of the OpenTelemetry language SDKs, which will complement the existing Elastic APM agents.

Universal Profiling: Watch Florian Lehner’s introduction to Universal Profiling video covering the use case, how it works under the hood, and how to use it in Kibana.

Data engineering pipeline: Implement a real-time sentiment analysis project with TCP Socket, Apache Spark, OpenAI, Kafka, and Elasticsearch by Rafael VM.

AI chatbot: Mike Richter explains how he built an AI chatbot with Elasticsearch and Azure AI for New York City Park events in about 3 hours. You can find the code for it on GitHub.

DevOpsDays Aracaju — April 20 

Meetup in St. Louis: Transforming Underutilized Media Assets into Valuable Resources — April 23

Haystack, Charlottesville — April 23–24 

Meetup in New York City: Powering an AI Research Assistant for Financial Services + Semantic Search & RAG — April 24

Meetup in Mexico City: Elastic & NowBit en CDMX — April 25 

Meetup in San Francisco: Elasticsearch in the United States House — May 1

Meetup in Boston: Scaling Logstash with BGP + Elasticsearch Query Language ES|QL — May 1 

Virtual Meetup: Aggregations, the Elasticsearch Group By — May 15 

Meetup in Silicon Valley: OTel Collector for log collection + Elasticsearch in the United States House — May 15

Cloud Summit Cerrado 2024, Goiânia — May 15–16

PyCon US, Pittsburgh — May 15–23

Microsoft Build, Seattle — May 21–23

Devoxx Greece, Athens — April 18–20

Commit Conference, Madrid — April 19–20

Meetup in Amsterdam: PyLadies Meetup @ Elastic — Introduction to Retrieval-Augmented Generation (RAG) — April 23 

AWS Summit London — April 24 

Meetup in Brussels: SRE and Observability with Elastic on GCP — Elastic & Google — April 24

Meetup in London: Lightning Talks with Aspiring Women Speakers — May 1 

Meetup in Copenhagen: Happy Hour Meetup with Elastic Security — May 1 

Devoxx UK, London — May 8–10

Javeloper, online — May 9 

JCON Europe, Cologne — May 13–16

Meetup in Zurich: Migros & Elastic: A Recipe for Success — May 15

PHPday 2024, Verona — May 16–17

GIDS Developer Summit India, Bengaluru — April 23–26 

Meetup in Tokyo: 第59回Elasticsearch勉強会 2024.4.24 #elasticsearchjp — April 24 

Global Azure Chennai — April 27

Meetup in Chongqing: Elasticsearch New Piped Query Language — ES|QL — April 27

Join your local Elastic meetup group for the latest news on upcoming events. If you’re interested in presenting at a meetup, send an email to meetups@elastic.co.

The release and timing of any features or functionality described in this post remain at Elastic's sole discretion. Any features or functionality not currently available may not be delivered on time or at all.