Data streamsedit

Data stream naming schemeedit

Elastic Agent uses data streams to store append-only time series data across multiple indices while giving users a single named resource for requests. If you’re new to data streams, see the Fleet and Elastic Agent Guide to learn more.

apm input data is divided into three types:

Traces

Traces are comprised of spans and transactions. Traces are stored in the following data stream:

  • Application traces: traces-apm-<namespace>
Metrics

Metrics include application-based metrics and basic system metrics. Metrics are stored in the following data streams:

  • Application defined metrics: metrics-apm.app.<service.name>-<namespace>
  • APM internal metrics: metrics-apm.internal-<namespace>
  • APM profiling metrics: metrics-apm.profiling-<namespace>
Logs

Logs include application error events and application logs. Logs are stored in the following data streams:

  • APM error/exception logging: logs-apm.error-<namespace>

Service namesedit

The APM integration maps an instrumented service’s name–defined in each APM agent’s configuration–to the index that its application defined metrics are stored in Elasticsearch. Service names therefore must follow index naming rules:

  • Service names are case-insensitive and must be unique. For example, you cannot have a service named Foo and another named foo.
  • Special characters will be removed from service names and replaced with underscores (_). Special characters include:

    '\\', '/', '*', '?', '"', '<', '>', '|', ' ', ',', '#', ':', '-'

Namespaceedit

There is no recommendation for what to use as your namespace; it’s intentionally flexible which allows greater control over how your data is indexed. For example, you might create namespaces for each of your environments, like dev, prod, production, etc. Or, you might create namespaces that correspond to strategic business units within your organization.