Upgrading Logstash and Elasticsearch to 2.0

If you are using Elasticsearch as an output, and wish to upgrade to Elasticsearch 2.0, please be aware of breaking changes before you upgrade. In addition, the following steps needs to be performed after upgrading to Elasticsearch 2.0:

Mapping changes: Users may have custom template changes, so by default a Logstash upgrade will leave the template as is. Even if you don’t have a custom template, Logstash will not overwrite an existing template by default.

There is one known issue (removal of path) with using GeoIP filter that needs a manual update to the template.

Note: If you have custom template changes, please make sure to save it and merge any changes. You can get the existing template by running:

curl -XGET localhost:9200/_template/logstash

Add the following option to your Logstash config:

output {
	elasticsearch {
		template_overwrite => true
	}
}

Restart Logstash.

Dots in fields: Elasticsearch 2.0 does not allow field names to contain the . character. Further details about this change here. Some plugins already have been updated to compensate for this breaking change, including logstash-filter-metrics and logstash-filter-elapsed. These plugin updates are available for Logstash 2.0. To upgrade to the latest version of these plugins, the command is:

bin/logstash-plugin update <plugin_name>

Multiline Filter: If you are using the Multiline Filter in your configuration and upgrade to Logstash 2.0, you will get an error. Make sure to explicitly set the number of filter workers (-w) to 1. You can set the number of workers by passing a command line flag such as:

bin/logstash -w 1