DevRel newsletter — July 2024

Hello from the Elastic DevRel team! In this newsletter blog, you’ll find information on upcoming events in your region, catch up on the latest content, and get a deep dive into the Elastic distribution for OpenTelemetry (OTel).

Elastic distribution for OpenTelemetry: Elastic introduces a distribution of the OpenTelemetry agents for Java, .NET, Python, and Node.js.

Background information and common questions:

• What is OpenTelemetry?
  OTel is a vendor-neutral observability framework and toolkit that supports telemetry signals, such as traces, metrics, and logs in applications and distributed microservice-based architectures. By adopting OpenTelemetry, software code can be instrumented in a vendor-agnostic fashion with telemetry signals exported in a standardized format to one or more vendor backends, such as Elastic APM.

• What is an OpenTelemetry distribution?
  It is a customized version of one or more OpenTelemetry components. Each distribution extends the core functionality offered by the component while adhering to its API and existing features, utilizing built-in extension points.

• Why would you prefer Elastic’s distribution over vanilla OpenTelemetry?
  It supports vendor-neutral instrumentation and no Elastic-specific configuration in code while adding features of Elastic APM that are currently missing in OpenTelemetry. Elastic’s distribution uses the recommended defaults when using Elastic Observability as a backend, and it uses OpenTelemetry APIs to further customize defaults.

Additional features in the OpenTelemetry distribution for Java: The agent brings together the best aspects of the proprietary Elastic Java agent and the OpenTelemetry Java agent. This distribution enhances the vanilla OpenTelemetry Java agent with a set of additional features realized through extensions while still being a fully compatible drop-in replacement.

• Uncover code latency with inferred spans: The wall clock time profiling-based inferred spans help us pinpoint the exact methods that contribute the most to the overall latency of the request. It follows the naming pattern Class#method. Elastic provides the inferred spans feature as an extension for the native OTel Java SDK, which you can use independently.

• Combining Universal Profiling with traces: Universal Profiling now integrates seamlessly with OpenTelemetry traces, marking a significant advancement in performance troubleshooting. This innovation allows for precise identification of code bottlenecks within the context of distributed traces, streamlining root cause analysis. By correlating continuous profiling data with tracing insights, Elastic Observability enhances visibility into cloud costs and environmental impacts at both service and transaction levels, empowering efficient and environmentally conscious development practices.

• Span stack traces: The Elastic OpenTelemetry Java agent includes span stack traces, gathering stack information for spans that last longer than a specified minimum time. This provides clear details on where each span starts in the code.

semantic_text: Learn about the new semantic_text field type and semantic query with Carlos Delgado and Mike Pellegrini — making semantic search in Elasticsearch just as simple as keyword search.

Playground: Follow Joe McElroy and Serena Chou as they explain how to use Kibana’s new Playground to experiment with retrieval augmented generation (RAG) applications using Elasticsearch. 

GPT models and Azure OpenAI consumption: Gain insight into OpenAI model inference and track metrics, such as token count and total request costs, from David Hope. This data is gathered via the OpenAI API and the OpenTelemetry Python library, which instrument external calls.

Semantic and geo search: Learn from Madhusudhan Konda on how to combine multiple sparse, dense, and geo fields to build more expressive queries and the best way to do it.

Elasticsearch vs. OpenSearch: Ugo Sangiorgi compares Elasticsearch and OpenSearch for vector search performance. Spoiler alert: The results show that Elasticsearch is up to 12x faster than OpenSearch for vector search.

OpenTelemetry cloud resource attributes: Sylvain Juge explains how cloud resource attributes can be used and configured in Java applications using OpenTelemetry SDK/API and instrumentation agents.

Data denormalization, semantic search, and generative AI: Check out Tim Brophy’s and Dani Aschwanden’s online banking prototype repo, which demonstrates how Elastic can improve both customer-facing and internal bank visibility and analytics.

Elasticsearch basics: Jessica Garson wrote some thoughts on when to choose between an index-level API, a query, an aggregation, or ES|QL in Elasticsearch.

RAG: Lloyd Chan contributed a Jupyter Notebook showing how to build a RAG application with Gemma, Elasticsearch, and Hugging Face to the Hugging Face generative AI cookbook.

• Woody Walton explains why Large Language Models (LLMs) and vector databases need a search engine to perform optimally and efficiently.
• Jamie Lindsell talks about the integration between Cohere and Elastic, going over Cohere’s journey in using Elasticsearch to build an internal AI chat assistant.
• Take a deeper look into AI search with Sunile Manjee.

• Aman Saxena outlines how to improve Elasticsearch performance with GoLang.
• Garvit Jain explains how to scale ELK Stack ingestion with Kafka.
• Vakhtang Matskeplishvili shares tips and tricks for mastering the Elastic Stack with APIs.

Meetup in Rio de Janeiro: Elastic Meetup no Rio: AIOps e Segurança — July 18

New York City: DevRelCon — July 18-19

Lima: KCD Lima 2024 — July 20 

Meetup in Rio de Janeiro: Elastic & Tech in Rio: Esquenta DevOpsDays Rio 2024 — August 22

Meetup in Toronto: Elastic best practices and more — July 24

Virtual Meetup: Scaling Logstash with BGP — July 25

Meetup in Cleveland: Elastic User Group Meetup — July 25

Cleveland: PyOhio — July 27-28

Meetup in São Paulo: Elastic Meetup na iColabora Brasil — July 31

Mexico City: Summit de Soluciones Elastic y Generative AI — August 1

Meetup in Mexico City: Generative AI y Elastic - Amor a primera vista — August 6 

Virtual Meetup: The Machine Learning Tree — August 7

São Paulo: AWS Summit 2024 — August 15

Virtual Meetup (Portuguese): 4º Meetup Virtual Elastic Brasil 2024 — August 21

Virtual conference: Testμ Conference — August 21-23

Meetup in São Paulo: Elastic & Red Hat em São Paulo — August 22

Rio de Janeiro: DevOpsDays Rio 2024 — August 24

Virtual Meetup (Spanish): 4º Meetup Elastic Latin America 2024 — August 27

Meetup in New York: Elastic User Group Meetup — August 28

Maceió: Roga DX 2024 — August 30-31

Virtual Meetup: Our Search Odyssey: Powering Auction Discovery with Elasticsearch — July 31

Meetup in Stockholm: Elasticsearch Query Language (ES|QL) — August 29

Dublin: ShipItCon — August 30

Meetup in Auckland: Elastic 101, Gen AI and More! — July 23

Meetup in Shenzhen: Elasticsearch new piped query language - ESQL — July 27 

Pune: AWS Community Day — August 3

Shanghai: The Global Open Source Technology conference — Aug 15-16 

Join your local Elastic meetup group for the latest news on upcoming events. If you’re interested in presenting at a meetup, send an email to meetups@elastic.co.

The release and timing of any features or functionality described in this post remain at Elastic's sole discretion. Any features or functionality not currently available may not be delivered on time or at all.