Upstream OpenTelemetry Collectors and language SDKs
Note
This is one of several approaches you can use to integrate Elastic with OpenTelemetry. To compare approaches and choose the best approach for your use case, refer to Use OpenTelemetry with APM.
Elastic natively supports the OpenTelemetry protocol (OTLP). This means trace data and metrics collected from your applications and infrastructure can be sent directly to Elastic.
- Send data to Elastic from an upstream OpenTelemetry Collector
- Send data to Elastic from an upstream OpenTelemetry language SDK
Send data from an upstream OpenTelemetry Collector
Connect your OpenTelemetry Collector instances to Elastic Observability using the OTLP exporter:
receivers:
# ...
otlp:
processors:
# ...
memory_limiter:
check_interval: 1s
limit_mib: 2000
batch:
exporters:
logging:
loglevel: warn
otlp/elastic:
# Elastic https endpoint without the "https://" prefix
endpoint: "${ELASTIC_APM_SERVER_ENDPOINT}" <5>
headers:
# Elastic API key
Authorization: "ApiKey ${ELASTIC_APM_API_KEY}" <6>
service:
pipelines:
traces:
receivers: [otlp]
exporters: [logging, otlp/elastic]
metrics:
receivers: [otlp]
exporters: [logging, otlp/elastic]
logs:
receivers: [otlp]
exporters: [logging, otlp/elastic]
You're now ready to export traces and metrics from your services and applications.
Tip
When using the OpenTelemetry Collector, you should always prefer sending data via the OTLP
exporter.
Using other methods, like the elasticsearch
exporter, will bypass all of the validation and data processing that Elastic performs.
In addition, your data will not be viewable in your Observability project if you use the elasticsearch
exporter.
Send data from an upstream OpenTelemetry SDK
Note
This document outlines how to send data directly from an upstream OpenTelemetry SDK to APM Server, which is appropriate when getting started. However, in many cases you should use the OpenTelemetry SDK to send data to an OpenTelemetry Collector that processes and exports data to APM Server. Read more about when and how to use a collector in the OpenTelemetry documentation.
To export traces and metrics to Elastic, instrument your services and applications with the OpenTelemetry API, SDK, or both. For example, if you are a Java developer, you need to instrument your Java app with the OpenTelemetry agent for Java. See the OpenTelemetry Instrumentation guides to download the OpenTelemetry agent or SDK for your language.
Define environment variables to configure the OpenTelemetry agent or SDK and enable communication with Elastic APM. For example, if you are instrumenting a Java app, define the following environment variables:
export OTEL_RESOURCE_ATTRIBUTES=service.name=checkoutService,service.version=1.1,deployment.environment=production
export OTEL_EXPORTER_OTLP_ENDPOINT=https://apm_server_url:8200
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=ApiKey an_apm_api_key"
export OTEL_METRICS_EXPORTER="otlp" \
export OTEL_LOGS_EXPORTER="otlp" \
java -javaagent:/path/to/opentelemetry-javaagent-all.jar \
-classpath lib/*:classes/ \
com.mycompany.checkout.CheckoutServiceServer
OTEL_RESOURCE_ATTRIBUTES | Fields that describe the service and the environment that the service runs in. See resource attributes for more information. |
OTEL_EXPORTER_OTLP_ENDPOINT | Elastic URL. The host and port that Elastic listens for APM events on. |
OTEL_EXPORTER_OTLP_HEADERS | Authorization header that includes the Elastic APM API key: |
OTEL_METRICS_EXPORTER | Metrics exporter to use. See exporter selection for more information. |
OTEL_LOGS_EXPORTER | Logs exporter to use. See exporter selection for more information. |
You are now ready to collect traces and metrics before verifying metrics and visualizing metrics.
Proxy requests to Elastic
Elastic supports both the (OTLP/gRPC) and (OTLP/HTTP) protocol on the same port as Elastic APM agent requests. For ease of setup, we recommend using OTLP/HTTP when proxying or load balancing requests to Elastic.
If you use the OTLP/gRPC protocol, requests to Elastic must use either HTTP/2 over TLS or HTTP/2 Cleartext (H2C). No matter which protocol is used, OTLP/gRPC requests will have the header: "Content-Type: application/grpc"
.
When using a layer 7 (L7) proxy like AWS ALB, requests must be proxied in a way that ensures requests to Elastic follow the rules outlined above. For example, with ALB you can create rules to select an alternative backend protocol based on the headers of requests coming into ALB. In this example, you'd select the gRPC protocol when the "Content-Type: application/grpc"
header exists on a request.
For more information on how to configure an AWS ALB to support gRPC, see this AWS blog post: Application Load Balancer Support for End-to-End HTTP/2 and gRPC.
For more information on how Elastic services gRPC requests, see Muxing gRPC and HTTP/1.1.
Next steps
- Collect metrics
- Add Resource attributes
- Learn about the limitations of this integration