Upgrading the Elasticsearch Mapping Template

By default, the Elasticsearch mapping template that comes with the Beats is not re-installed if it already exists. This means that if the new Beats version adds new fields (e.g. a new module in Metricbeat), they won’t be reflected in the Elasticsearch mapping, which can result in visualization problems or incorrect data.

You can set the output.elasticsearch.template.overwrite option to true to make the Beats install the new mapping template even if a template with the same name already exists. You can set this option in the configuration file or as a CLI flag, for example:

packetbeat -e -E output.elasticsearch.template.overwrite=true

However, the mapping template is applied on index creation, so it won’t affect the current index if it was already created with the old version.

You can force the start of a new index by renaming the old index before starting the new version of the Beat. As of Elasticsearch 5.0, this can be done via the re-index API:

POST /_reindex
{
  "source": {
    "index": "packetbeat-2016.09.20"
  },
  "dest": {
    "index": "packetbeat-2016.09.20-old"
  }
}
DELETE /packetbeat-2016.09.20

Note that the reindex API command can take a long time, depending on the size of the index. It is recommended that you stop the Beat during this time, so the order of operations should be:

If downtime is not acceptable, another possible approach is to configure a different index pattern in the new Beat version, but this will likely require adjustments to your Kibana dashboards.