NOTE: You are looking at documentation for an older release. For the latest information, see the current release documentation.

« Elasticsearch for Apache Hadoop version 6.0.1 Elasticsearch for Apache Hadoop version 6.0.0-rc2 »

› ›

Elasticsearch for Apache Hadoop version 6.0.0

Elasticsearch for Apache Hadoop version 6.0.0

Breaking Changes

Elasticsearch on YARN integration has been removed #1001 #1027
Using Hadoop versions 1.x is deprecated as of 5.5.0 #1001
ES-Hadoop will support Scala 2.11 by default in 6.0.0 #1001
New timestamp formatting #985

New Features

Spark

Spark 2.0 Stream Interfaces #818

Serialization

Support Elasticsearch "join" types #1012

Enhancements

Spark

Spark 2.1.1 and 2.2.0 Support #978 #1013
Spark JavaRDD give partial documents #946

Scripting

Allow users to use file based scripts during updates #918
Support script id/file/inline options #538

REST

Remove uses of features deprecated in 5.0 #881
Remove unmatched format specifier from Restservice #1029

Serialization

Support nested collections of Java Bean classes in Spark #1021
Not able to use kryoserializer for writing data into Elasticsearch #1019
JacksonJsonGenerator.getParentPath will always be empty #1006

Pig

Using es.mapping.exclude/include and still getting StrictDynamicMappingException on excluded fields #1015

Build

Elasticsearch is no longer started as an embedded process when running tests.
New buildSrc layout for integration testing and one-click-builds.
Fix implicit conversion warnings in Spark #973 #980
Fix remaining Scala warnings #970 #1011

Testing

Convert tests to use Painless #987

Bug Fixes

Spark

Add scaled and half float to schemautils #982 #981
Support Date / Timestamp objects #934 #931
When overwriting data in a Spark Dataframe, the Dataframe will not attempt to delete data if the index already does not exist.
version mismatch exception #1070

REST

Scroll Id’s are malformed for ES 1.x #1002
ElasticsearchIllegalArgumentException[Malformed scrollId []] caused by es.scroll.limit #992
Scroll Id regression on Elasticsearch 2.4.5 #1034
A new set of logging commands added for SSL debugging have been reduced in severity to avoid spamming streaming users.

Documentation

fixed reference to maven dependency https://github.com/elastic/elasticsearch-hadoop/pull/1016#1016]
wrong variable saved in java streaming example #961
Unclear Docs (about multi-resource writes) #990
Adding comment on breaking change #1048
Document best practices for stack upgrades that use ES-Hadoop #1053

« Elasticsearch for Apache Hadoop version 6.0.1 Elasticsearch for Apache Hadoop version 6.0.0-rc2 »