A newer version is available. For the latest information, see the current release documentation.
« Breaking Changes Upgrade to APM Server version 7.0 »
Elastic Docs › ›Troubleshoot ›Upgrade APM Server

Upgrade to APM Server version 6.5

edit

Upgrade to APM Server version 6.5

edit

If you’re an advanced user, are implementing a custom agent, or provide custom data to Elasticsearch, there are some changes in APM Server version 6.5 that you should be aware of.

None of the changes in APM Server v6.5 are breaking changes, and the APM Server is still compatible with your currently implemented agents. However, to take advantage of new features, it’s recommended you read about the changes below and update accordingly. If you’d like to continue using the deprecated Intake API and Elasticsearch Schema, you can view previous documentation.

Upgrade Steps

edit
  1. Upgrade the Elastic Stack: Elasticsearch, Kibana, and APM Server.
  2. Upgrade your APM Agent.

Check the agent/server compatibility matrix for more details on which APM agents are compatible with APM Server version 6.5.

You must upgrade the APM Server before upgrading your APM Agent. New agents do not support APM Server <6.5 and will not work if you upgrade in the wrong order.

Intake API Changes

edit

The Intake API is what we call the internal protocol that APM agents use to talk to the APM Server. This API has been redesigned in 6.5 to increase the memory efficiency and predictability of the APM Server and APM agents.

Notably, most of the endpoints associated with this API have been deprecated, and replaced with a new endpoint: intake/v2/events.

Similarly, the RUM endpoint has been deprecated and replaced with a new endpoint: intake/v2/rum/events.

Details about these changes, and the information you can send to the new endpoint, is available in the Events API documentation.

Metadata
edit

Metadata is now sent to the APM Server only once per request. APM Server will retain this information and apply it to the applicable documents when it writes to Elasticsearch.

Metadata should be sent to the new intake/v2/events endpoint. See the metadata documentation for more information.

Error
edit

The error schema adds three new properties, and deprecates two others.

  • deprecate pattern of error.id, change format to hex encoded 64 random bits
  • deprecate transaction.id
  • add transaction_id (required, hex encoded 64 random bits)
  • add parent_id (optional, hex encoded 64 random bits)
  • add trace_id (optional, hex encoded 128 random bits)

Error events should be sent to the new intake/v2/events endpoint. See the error schema documentation for more information.

Transaction
edit

The transaction schema adds two new properties, and deprecates two others.

  • deprecate pattern of transaction.id, change format to hex encoded 64 random bits
  • deprecate spans (don’t forbid them, just don’t process them)
  • add parent_id (optional, hex encoded 64 random bits string)
  • add trace_id (required, hex encoded 128 random bits)

Transaction events should be sent to the new intake/v2/events endpoint. See the transaction schema documentation for more information.

Span
edit

The span schema adds three new properties, removes two, and changes one.

  • change pattern of id to hex encoded 64 random bits
  • deprecate pattern of transaction_id
  • deprecate use of parent
  • add parent_id (required, hex encoded 64 random bits)
  • add trace_id (required, hex encoded 128 random bits)
  • add timestamp

Span events should be sent to the new intake/v2/events endpoint. See the span schema documentation for more information.

Healthcheck Endpoint
edit

The Healthcheck endpoint, previously /healthcheck, has been deprecated. The server information api has replaced it.

Assets Endpoint
edit

The assets endpoint, /v1/rum/sourcemaps has been deprecated, and replaced with the endpoint: /assets/v1/sourcemaps. The data format remains unchanged.

Metricsets Endpoint
edit

The Metricsets endpoint, previously /v1/metrics, has also integrated with the new intake endpoint at /intake/v2/events. There are two changes to be aware of:

  • Timestamps are now UTC based and formatted as microseconds since Unix epoch.
  • The metrics.enabled setting has been deprecated and only applies to v1/metrics. Metricsets can’t be disabled in v2.

See the metricset schema for more information.

Server Configuration Changes

edit

There are some configuration changes that are worth mentioning.

Deprecated configuration options:

  • max_unzipped_size
  • max_request_queue_time

New configuration options:

  • max_event_size with a default value: 307200 bytes. You may notice this default value is much smaller than the deprecated max_unzipped_size default value. This is because max_unzipped_size referred to the whole request, while max_event_size only refers to a single event within the request.

Notable options:

  • read_timeout and write_timeout remain at 30 seconds, but if you have a load balancer in front of the APM server you may notice connections are a little bit longer lived than they used to be.

If you are updating APM Server from version 6.3 or earlier, you may not be using an updated apm-server.yml configuration file. Update your apm-server.yml configuration file to take advantage of new configuration options.

RUM configuration changes:

  • event_rate.limit has replaced the deprecated rate_limit. In v1 of the intake API, the RUM rate limiter was bound to the number of requests per second, per IP. In v2, the rate limiter has changed to be bound to the number of events sent per second, per IP.

Elasticsearch Schema Changes

edit

The Elasticsearch schema defines how APM data is stored in Elasticsearch. There have been a number of changes to the Elasticsearch schema for 6.5.

An important change to note is the addition of the trace and parent keys, which have been added to errors, transactions, and spans. Both only hold a field id. These new keys are essential to taking advantage of APM’s new distributed tracing feature.

Error
edit

The Elasticsearch error schema adds two new keys:

  • add trace.id
  • add parent.id

View the sample Elasticsearch error document for more information.

Transaction
edit

The Elasticsearch transaction schema adds two new keys:

  • add trace.id
  • add parent.id

View the sample Elasticsearch transaction document for more information.

Span
edit

The Elasticsearch span schema adds three new keys, and deprecates two:

  • add trace.id
  • add parent.id
  • add hex_id
  • deprecate parent long
  • deprecate id long

View the sample Elasticsearch span document for more information.

« Breaking Changes Upgrade to APM Server version 7.0 »