WARNING: Version 6.1 has passed its EOL date.

This documentation is no longer being maintained and may be removed. If you are running this version, we strongly advise you to upgrade. For the latest information, see the current release documentation.

« Elasticsearch for Apache Hadoop version 6.0.1 Elasticsearch for Apache Hadoop version 6.0.0-rc2 »
Elastic Docs ›Elasticsearch for Apache Hadoop [6.1] ›Release Notes

Elasticsearch for Apache Hadoop version 6.0.0

edit

Elasticsearch for Apache Hadoop version 6.0.0

edit

November 14, 2017

Breaking Changes

edit
  • Elasticsearch on YARN integration has been removed #1001 #1027
  • Using Hadoop versions 1.x is deprecated as of 5.5.0 #1001
  • ES-Hadoop will support Scala 2.11 by default in 6.0.0 #1001
  • New timestamp formatting #985

New Features

edit
Spark
  • Spark 2.0 Stream Interfaces #818
Serialization
  • Support Elasticsearch "join" types #1012

Enhancements

edit
Spark
  • Spark 2.1.1 and 2.2.0 Support #978 #1013
  • Spark JavaRDD give partial documents #946
Scripting
  • Allow users to use file based scripts during updates #918
  • Support script id/file/inline options #538
REST
  • Remove uses of features deprecated in 5.0 #881
  • Remove unmatched format specifier from Restservice #1029
Serialization
  • Support nested collections of Java Bean classes in Spark #1021
  • Not able to use kryoserializer for writing data into Elasticsearch #1019
  • JacksonJsonGenerator.getParentPath will always be empty #1006
Pig
  • Using es.mapping.exclude/include and still getting StrictDynamicMappingException on excluded fields #1015
Build
  • Elasticsearch is no longer started as an embedded process when running tests.
  • New buildSrc layout for integration testing and one-click-builds.
  • Fix implicit conversion warnings in Spark #973 #980
  • Fix remaining Scala warnings #970 #1011
Testing
  • Convert tests to use Painless #987

Bug Fixes

edit
Spark
  • Add scaled and half float to schemautils #982 #981
  • Support Date / Timestamp objects #934 #931
  • When overwriting data in a Spark Dataframe, the Dataframe will not attempt to delete data if the index already does not exist.
  • version mismatch exception #1070
REST
  • Scroll Id’s are malformed for ES 1.x #1002
  • ElasticsearchIllegalArgumentException[Malformed scrollId []] caused by es.scroll.limit #992
  • Scroll Id regression on Elasticsearch 2.4.5 #1034
  • A new set of logging commands added for SSL debugging have been reduced in severity to avoid spamming streaming users.

Documentation

edit
« Elasticsearch for Apache Hadoop version 6.0.1 Elasticsearch for Apache Hadoop version 6.0.0-rc2 »