Troubleshoot Beats upgrade issues

edit

Troubleshoot Beats upgrade issues

edit

This section describes common problems you might encounter when upgrading to Beats 7.x.

You can avoid some of these problems by reading Upgrade from 6.x to 7.x before upgrading Beats.

Missing fields or too many fields in the index

edit

You may have run the Beat before loading the required index template. To clean up and start again:

Delete the index that was created when you ran the Beat. For example:
```
DELETE metricbeat-7.17.26-2019.04.02*
```
Be careful when using wildcards to delete indices. Make sure the pattern matches only the indices you want to delete. The example shown here deletes all data indexed into the metricbeat-7.17.26 indices on 2019.04.02.
If you want the index to work with 6.x dashboards, turn on the compatibility layer. See Enable the compatibility layer for Elastic Common Schema (ECS) fields.
Delete the index template that was loaded earlier. For example:
```
DELETE /_index_template/metricbeat-7.17.26
```
Because the index template was loaded without the compatibility layer enabled, the required aliases were not created.
Load the correct index template. See Upgrade the Elasticsearch index template.
Restart Beats.

User is not authorized

edit

Because index lifecycle management is on by default in 7.0, you might encounter new errors related to user authorization when you run version 7.0 against an Elasticsearch cluster that supports index lifecycle management.

`[cluster:monitor/main] is unauthorized for user`

edit

Problem: The Beats user is unable to send monitoring information.

Solution: Grant the monitor cluster privilege.

`[cluster:admin/ilm/put] is unauthorized for user`

edit

Problem: The Beats user is not authorized to load ILM policies.

Solution: Grant the manage_ilm cluster privilege.

`[indices:admin/template/put] is unauthorized for user`

edit

Problem: Automatic index template loading is required when ILM is enabled, but the user is not authorized to manage index templates.

Solution: Grant the manage_index_templates cluster privilege.

`[indices:admin/aliases] is unauthorized for user`

edit

Problem: The Beats user is unable to set up aliases needed by the compatibility layer.

Solution: Grant the manage privilege on the Beats indices.

`[indices:data/write/bulk] is unauthorized for user`

edit

Problem: The Beats user is unable to write events to Elasticsearch.

Solution: Grant the index privilege on the Beats indices.

6.x dashboards not showing data from 7.0 shippers

edit

You might have run the Beat without turning on the compatibility layer. See Enable the compatibility layer for Elastic Common Schema (ECS) fields then clean up your environment as described under Missing fields or too many fields in the index.

Data parsed by Logstash not appearing in 7.0 dashboards

edit

You might be writing to an index that doesn’t match the index pattern used by Beats. See How to use versioned index templates when the output is not Elasticsearch.

"shard failed" error when viewing Beats dashboards in Kibana

edit

After upgrading to Elasticsearch 7.0, any indices created by Beats 6.6 or older will not work in Kibana dashboards until the index.query.default_field setting is added to each index. Indices created in Beats 6.7 or later are unaffected.

Background: Starting in Elasticsearch 7.0, some query types, such as Simple Query String, have a limit to the number of fields they will query against. Because Beats indices often contain more fields than this cap, the index.query.default_field index setting is used to inform Elasticsearch which fields to use by default when no field is specified for a query.

To add the setting to the index, you can use the 7.0 Upgrade Assistant, or add the setting manually.

To add the setting manually, first identify the list of fields that you want to set as the default search fields, then specify the list of fields in the default_field setting.

For example, here’s a snippet that shows how to add default search fields to a Metricbeat 6.6 index. This example is truncated. Full examples for Metricbeat and Filebeat are available in this file.

PUT /metricbeat-6.6.2-2019.04.09/_settings
{
  "index": {
    "query": {
      "default_field": [
        "aerospike.namespace.name",
        "aerospike.namespace.node.host",
        "aerospike.namespace.node.name",
        "apache.status.hostname",
        "beat.hostname",
        "beat.name",
        "beat.timezone",
        "beat.version",
        ...
      ]
    }
  }
}

« Upgrade from 6.x to 7.x Breaking changes »