Troubleshooting

This section describes common problems you might encounter with the APM app. To add to this page, please consider opening a pull request with your proposed changes.

If your issue is potentially related to other components of the APM ecosystem, don’t forget to check our other troubleshooting guides or discussion forum:

This section can help with any of the following:

These problems are likely to be caused by a missing index template or ingest pipeline. By default, Fleet sets up these and other APM assets when the APM integration is installed. Try reinstalling the APM integration by navigating to Integrations > Elastic APM > Manage in Fleet > Settings > Reinstall Elastic APM.

Because assets cannot be applied to indices retroactively, after reinstalling the APM integration you must either wait for the index to rollover or force a rollover. To force a rollover, use the rollover API to target the relevant APM data streams.

Transaction names are defined in each APM Agent; when an Agent supports a framework, it includes logic for naming the transactions that the framework creates. In some cases though, like when using an Agent’s API to create custom transactions, it is up to the user to define a pattern for transaction naming. When transactions are named incorrectly, each unique URL can be associated with a unique transaction group—causing an explosion in the number of transaction groups per service, and leading to inaccuracies in the APM app.

To fix a large number of unique transaction names, you need to change how you are using the Agent API to name your transactions. To do this, ensure you are not naming based on parameters that can change. For example, user ids, product ids, order numbers, query parameters, etc., should be stripped away, and commonality should be found between your unique URLs.

Let’s look at an example from the RUM Agent documentation. Here are a few URLs you might find on Elastic.co:

// Blog Posts
https://www.elastic.co/blog/reflections-on-three-years-in-the-elastic-public-sector
https://www.elastic.co/blog/say-heya-to-the-elastic-search-awards
https://www.elastic.co/blog/and-the-winner-of-the-elasticon-2018-training-subscription-drawing-is

// Documentation
https://www.elastic.co/guide/en/elastic-stack/current/index.html
https://www.elastic.co/guide/en/apm/get-started/current/index.html
https://www.elastic.co/guide/en/infrastructure/guide/current/index.html

These URLs, like most, include unique names. If we named transactions based on each unique URL, we’d end up with the problem described above—a very large number of different transaction names. Instead, we should strip away the unique information and group our transactions based on common information. In this case, that means naming all blog transactions, /blog, and all documentation transactions, /guide.

If you feel like you’d be losing valuable information by following this naming convention, don’t fret! You can always add additional metadata to your transactions using {apm-guide-ref-v}/metadata.html#labels-fields[labels] (indexed) or {apm-guide-ref-v}/metadata.html#custom-fields[custom context] (non-indexed).

After ensuring you’ve correctly named your transactions, you might still see an error in the APM app related to too many transaction names. If this is the case, you can increase the default number of transaction groups displayed in the APM app by configuring xpack.apm.ui.transactionGroupBucketSize.

More information

While this can happen with any APM Agent, it typically occurs with the RUM Agent. For more information on how to correctly set transaction.name in the RUM Agent, see custom initial page load transaction names.

The RUM Agent can also set the transaction.name when observing for transaction events. See apm.observe() for more information.

If your problem is occurring in a different Agent, the tips above still apply. See the relevant Agent API documentation to adjust how you’re naming your transactions.

The transaction overview will only display helpful information when the transactions in your services are named correctly. If you’re seeing "GET unknown route" or "unknown route" in the APM app, it could be a sign that something isn’t working as it should.

Elastic APM Agents come with built-in support for popular frameworks out-of-the-box. This means, among other things, that the Agent will try to automatically name HTTP requests. As an example, the Node.js Agent uses the route that handled the request, while the Java Agent uses the Servlet name.

"Unknown route" indicates that the Agent can’t determine what to name the request, perhaps because the technology you’re using isn’t supported, the Agent has been installed incorrectly, or because something is happening to the request that the Agent doesn’t understand.

To resolve this, you’ll need to head over to the relevant Agent documentation. Specifically, view the Agent’s supported technologies page. You can also use the Agent’s public API to manually set a name for the transaction.

In Elasticsearch, index templates are used to define settings and mappings that determine how fields should be analyzed. The recommended index templates for APM are installed by Fleet when the Elastic APM integration is installed. These templates, by default, enable and disable indexing on certain fields.

As an example, some agents store cookie values in http.request.cookies. Since http.request has disabled dynamic indexing, and http.request.cookies is not declared in a custom mapping, the values in http.request.cookies are not indexed and thus not searchable.

Ensure an APM data view exists As a first step, you should ensure the correct data view exists. In Kibana, go to Stack Management > Data views. You should see the APM data view—​the default is traces-apm*,apm-*,logs-apm*,apm-*,metrics-apm*,apm-*. If you don’t, the data view doesn’t exist. To fix this, navigate to the APM app in Kibana and select Add data. In the APM tutorial, click Load Kibana objects to create the APM data view.

If creating an APM data view doesn’t solve the problem, see No APM data found for further troubleshooting.

Ensure a field is searchable There are two things you can do to if you’d like to ensure a field is searchable:

If the service map is not showing an expected connection between the client and server, it’s likely because you haven’t configured distributedTracingOrigins.

This setting is necessary, for example, for cross-origin requests. If you have a basic web application that provides data via an API on localhost:4000, and serves HTML from localhost:4001, you’d need to set distributedTracingOrigins: ['https://localhost:4000'] to ensure the origin is monitored as a part of distributed tracing. In other words, distributedTracingOrigins is consulted prior to the agent adding the distributed tracing traceparent header to each request.