IMPORTANT: No additional bug fixes or documentation updates will be released for this version. For the latest information, see the current release documentation.
« Elasticsearch for Apache Hadoop version 6.0.1 Elasticsearch for Apache Hadoop version 6.0.0-rc2 »
Elastic Docs ›Elasticsearch for Apache Hadoop [6.3] ›Release Notes

Elasticsearch for Apache Hadoop version 6.0.0

edit

Elasticsearch for Apache Hadoop version 6.0.0

edit

November 14, 2017

Breaking Changes

edit
  • Elasticsearch on YARN integration has been removed #1001 #1027
  • Using Hadoop versions 1.x is deprecated as of 5.5.0 #1001
  • ES-Hadoop will support Scala 2.11 by default in 6.0.0 #1001
  • New timestamp formatting #985

New Features

edit
Spark
  • Spark 2.0 Stream Interfaces #818
Serialization
  • Support Elasticsearch "join" types #1012

Enhancements

edit
Spark
  • Spark 2.1.1 and 2.2.0 Support #978 #1013
  • Spark JavaRDD give partial documents #946
Scripting
  • Allow users to use file based scripts during updates #918
  • Support script id/file/inline options #538
REST
  • Remove uses of features deprecated in 5.0 #881
  • Remove unmatched format specifier from Restservice #1029
Serialization
  • Support nested collections of Java Bean classes in Spark #1021
  • Not able to use kryoserializer for writing data into Elasticsearch #1019
  • JacksonJsonGenerator.getParentPath will always be empty #1006
Pig
  • Using es.mapping.exclude/include and still getting StrictDynamicMappingException on excluded fields #1015
Build
  • Elasticsearch is no longer started as an embedded process when running tests.
  • New buildSrc layout for integration testing and one-click-builds.
  • Fix implicit conversion warnings in Spark #973 #980
  • Fix remaining Scala warnings #970 #1011
Testing
  • Convert tests to use Painless #987

Bug Fixes

edit
Spark
  • Add scaled and half float to schemautils #982 #981
  • Support Date / Timestamp objects #934 #931
  • When overwriting data in a Spark Dataframe, the Dataframe will not attempt to delete data if the index already does not exist.
  • version mismatch exception #1070
REST
  • Scroll Id’s are malformed for ES 1.x #1002
  • ElasticsearchIllegalArgumentException[Malformed scrollId []] caused by es.scroll.limit #992
  • Scroll Id regression on Elasticsearch 2.4.5 #1034
  • A new set of logging commands added for SSL debugging have been reduced in severity to avoid spamming streaming users.

Documentation

edit
« Elasticsearch for Apache Hadoop version 6.0.1 Elasticsearch for Apache Hadoop version 6.0.0-rc2 »