Troubleshooting
editTroubleshooting
editSomething isn’t quite working as expected? Here are some guidelines how to find out what’s going wrong.
Don’t worry if you can’t figure out what the problem is. Open a topic in the APM discuss forum and we will help you out. If you do so, please attach your debug logs so that we can analyze the problem.
Also, please check if your stack is compatible with the currently supported technologies.
Logging
editThere are several logging related configuration options.
The most important one is log_level.
Set the log level to DEBUG or even TRACE to get more information about the behavior of the agent.
The TRACE log level is even more verbose than DEBUG and prints a stack trace every time a transaction or span is started or stopped.
Also, the JSON documents which are sent to the APM server will be logged on TRACE level.
Please always post the whole content of your log files when asking for help.
When the agent starts up, you should see logs similar to these:
[main] INFO co.elastic.apm.configuration.StartupInfo - Starting Elastic APM (unknown version) on Java 10 (Oracle Corporation) Mac OS X 10.13.6 [main] DEBUG co.elastic.apm.configuration.StartupInfo - service_name: 'elastic-apm-test' (source: Java System Properties) [main] DEBUG co.elastic.apm.configuration.StartupInfo - log_level: 'DEBUG' (source: Java System Properties)
Make sure to execute some requests to your application before posting your log files. Each request should at least add some lines similar to these in the logs:
[main] DEBUG co.elastic.apm.impl.ElasticApmTracer - startTransaction '' e0d6b9f84cf758a30a303f4fb607a9a6 {
[main] DEBUG co.elastic.apm.impl.ElasticApmTracer - startSpan '' e0d6b9f84cf758a30a303f4fb607a9a6:1 {
[main] DEBUG co.elastic.apm.impl.ElasticApmTracer - } endSpan 'SELECT' e0d6b9f84cf758a30a303f4fb607a9a6:1
[main] DEBUG co.elastic.apm.impl.ElasticApmTracer - } endTransaction 'GET' e0d6b9f84cf758a30a303f4fb607a9a6
If you don’t see anything in your logs, the technology stack you are using is probably not supported.
After that, you should see logs indicating that the agent has successfully sent data to the APM server:
[apm-reporter] DEBUG co.elastic.apm.report.ApmServerHttpPayloadSender - Sending payload with 1 elements to APM server http://localhost:8200 [apm-reporter] DEBUG co.elastic.apm.report.ApmServerHttpPayloadSender - APM server responded with status code 202
If the APM server responds with a 400,
it could indicate JSON validation errors.
In that case,
please turn on TRACE logging and send us the logged JSON payload.
Debugging
editSometimes reading the logs is just not enough to debug a problem. As the agent is OpenSource and released on Maven Central, debugging the agent code is really easy.
In order for your IDE to download the sources, first declare a dependency to the agent.
The agent is added via the -javaagent flag.
So you have to make sure that you declare the dependency in a way that the agent is not on the classpath twice.
For example,
when you are developing a web application,
make sure that the agent is not packaged in your war in WEB-INF/lib.
In the debug window of your IDE,
make sure that elastic-apm-agent does not appear in the -classpath.
pom.xml.
<dependency>
<groupId>co.elastic.apm</groupId>
<artifactId>elastic-apm-agent</artifactId>
<version>${elastic-apm.version}</version>
<scope>provided</scope>
</dependency>
Even when setting the scope to provided,
IntelliJ sometimes adds the agent on the classpath anyway.
One workaround is to set the scope to test.
build.gradle.
compileOnly "co.elastic.apm:elastic-apm-agent:$elasticApmVersion"
Common problems
editThere is no data in the Kibana APM UI
editThe most common source of this problem are connection issues between the agent and the APM server.
If the APM server does not receive data from the agent,
check if the agent is able to establish a connection to the server.
In the agent logs,
look out for logs containing Elastic APM server is available
and Elastic APM server is not available.
If you see the message Elastic APM server is not available,
the agent has problems connecting to the APM server.
Check the setting of server_urls and make sure the agent is able to connect to the server.
Try to execute curl -v <apm-server-url>/healthcheck from the machine the agent is running on.
The server should respond with a 200 status code.
If the APM server does not respond successfully, have a look at the APM server logs to verify that the server is actually running. Also make sure to configure your firewalls so that the host the agent runs on can open HTTP connections to the APM server.