WARNING: Version 5.4 of Elasticsearch has passed its EOL date.

This documentation is no longer being maintained and may be removed. If you are running this version, we strongly advise you to upgrade. For the latest information, see the current release documentation.

« 5.0.0-alpha2 Release Notes 5.0.0-alpha1 Release Notes (Changes previously released in 2.x) »
Elastic Docs ›Elasticsearch Reference [5.4] ›Release Notes

5.0.0-alpha1 Release Notes

edit

5.0.0-alpha1 Release Notes

edit

The changes listed below have been released for the first time in Elasticsearch 5.0.0-alpha1. Changes in this release which were first released in the 2.x series are listed in 5.0.0-alpha1 Release Notes (Changes previously released in 2.x).

Also see Breaking changes in 5.0.

This is an alpha release and is intended for testing purposes only. Indices created in this version will not be compatible with Elasticsearch 5.0.0 GA. Upgrading 5.0.0-alpha1 to any other version is not supported.

Breaking changes

edit
Aliases
  • make get alias expand to open and closed indices by default #15954 (issue: #14982)
  • Remove deprecated indices.get_aliases #13906
Allocation
  • Remove DisableAllocationDecider #13313
CAT API
  • Add raw recovery progress to cat recovery API #17064 (issue: #17022)
  • Remove host from cat nodes API #16656 (issues: #12959, #16575)
  • Using the accept header in the request instead of content-type in _cat API. #14421 (issue: #14195)
CRUD
  • Remove object notation for core types. #15684 (issue: #15388)
Cache
  • Remove deprecated query cache settings #15592
Cluster
Core
Engine
  • Remove index.compound_on_flush setting and default to true #15594 (issue: #10778)
Fielddata
  • Remove "uninverted" and "binary" fielddata support for numeric and boolean fields. #14082
Index APIs
Internal
  • Remove support for pre 2.0 indices #13799
Mapping
  • Change the field mapping index time boost into a query time boost. #16900
  • Deprecate string in favor of text/keyword. #16877
  • Term vector APIs should no longer update mappings #16285
  • Remove the format option of the _source field. #15398
  • Remove transform #13657 (issue: #12674)
Parent/Child
  • Removed total score mode in favour for sum score mode. #17174 (issues: #13470, #17083)
  • Removed pre 2.x parent child implementation #13376
Percolator
  • Change the percolate api to not dynamically add fields to mapping #16077 (issue: #15751)
Plugins
  • Rename bin/plugin in bin/elasticsearch-plugin #16454
  • Remove multicast plugin #16326 (issue: #16310)
  • Plugins: Remove site plugins #16038
Query DSL
  • Function score query: remove deprecated support for boost_factor #13510
  • Remove support for deprecated queries. #13418 (issue: #13326)
REST
  • Limit the accepted length of the _id #16036 (issue: #16034)
Search
Search Refactoring
  • Remove "query" query and fix related parsing bugs #14304 (issue: #13326)
Settings
  • Remove ability to specify arbitrary node attributes with node. prefix #17402 (issue: #17280)
  • Enforce discovery.zen.minimum_master_nodes is set when bound to a public ip #17288
  • Prevent index level setting from being configured on a node level #17144 (issue: #16799)
  • Remove support for node.client setting #16963 (issue: #16565)
  • Remove es.max-open-files flag #16757 (issues: #16506, #483)
  • Enforce node level limits if node is started in production env #16733 (issue: #16727)
  • Make settings validation strict #16365
  • Remove the ability to fsync on every operation and only schedule fsync task if really needed #16257 (issue: #16152)
  • Script settings #16197
  • Remove index.flush_on_close entirely #15977
  • Restore chunksize of 512kb on recovery and remove configurability #15235 (issue: #15161)
  • Remove ancient deprecated and alternative recovery settings #15234
  • Fix ping timeout settings inconsistencies #13701 (issue: #6579)
Similarities
  • Renames default similarity into classic #15446 (issue: #15102)
Snapshot/Restore
  • Fail closing or deleting indices during a full snapshot #17021 (issue: #16321)
Stats
  • Modify load average format #15932 (issue: #15907)
  • Reintroduce five-minute and fifteen-minute load averages on Linux #15907 (issues: #12049, #14741)
  • Add system CPU percent to OS stats #14741
Term Vectors
  • Remove DFS support from TermVector API #16452
Translog
  • Drop support for simple translog and hard-wire buffer to 8kb #15574
  • Simplify translog-based flush settings #15573
Warmers
  • Remove query warmers and the warmer API. #15614 (issue: #15607)

Breaking Java changes

edit
Aggregations
  • getKeyAsString and key_as_string should be the same for terms aggregation on boolean field #15393
Allocation
  • Simplify shard balancer interface #17028 (issue: #8954)
Analysis
  • Simplify Analysis registration and configuration #14355
Cache
  • Refactor IndicesRequestCache to make it testable. #16610
  • Fold IndexCacheModule into IndexModule #14293
Core
  • Remove es.useLinkedTransferQueue #16786
Discovery
  • Include pings from client nodes in master election #17329 (issue: #17325)
Internal
  • Remove duplicate getters from DiscoveryNode and DiscoveryNodes #17410 (issue: #16963)
  • Cli: Switch to jopt-simple #17024 (issue: #11564)
  • Replace ContextAndHeaders with a ThreadPool based ThreadLocal implementation #15776
  • Remove NodeBuilder #15354
  • Fix IndexSearcherWrapper interface to not depend on the EngineConfig #14654
  • Cleanup query parsing and remove IndexQueryParserService #14452
  • Remove circular dependency between IndicesService and IndicesStore #14285
  • Remove guice injection from IndexStore and friends #14279
  • Replace IndicesLifecycle with a per-index IndexEventListener #14217 (issue: #13259)
  • Simplify similarity module and friends #13942
  • Refactor SearchRequest to be parsed on the coordinating node #13859
Java API
  • Remove the count api #14166 (issue: #13928)
  • IdsQueryBuilder to accept only non null ids and types #13937
Mapping
  • [Mapping] Several MappingService cleanups #16133 (issue: #15924)
Network
  • Remove ability to disable Netty gathering writes #16774 (issue: #7811)
Parent/Child
  • Cleanup ParentFieldMapper #16045
  • Several other parent/child cleanups #13470
Plugins
  • Enforce isolated mode for all plugins #17276
  • Change the inner structure of the plugins zip #16453
  • Don’t use guice for QueryParsers #15761
  • Remove guice from the index level #14518
  • Remove shard-level injector #13881
Query DSL
  • Remove the MissingQueryBuilder which was deprecated in 2.2.0. #15364 (issue: #14112)
  • Remove NotQueryBuilder #14204 (issue: #13761)
Search
  • Query refactoring: split parse phase into fromXContent and toQuery for all queries #13788 (issue: #10217)
Search Refactoring
  • Refactored inner hits parsing and intoduced InnerHitBuilder #17291
  • Remove deprecated parameter from field sort builder. #16573 (issue: #16127)
  • Remove support for query_binary and filter_binary #14433 (issue: #14308)
  • Validate query api: move query parsing to the coordinating node #14384
Settings
  • Move remaining settings in NettyHttpServerTransport to the new infra #16531
  • Replace IndexSettings annotation with a full-fledged class #14251
Store
  • Standardize state format type for global and index level metadata #17123
Suggesters

Deprecations

edit
Plugin Mapper Attachment
  • Deprecate mapper-attachments plugin #16948 (issue: #16910)
Search

New features

edit
Allocation
  • Add API to explain why a shard is or isn’t assigned #17305 (issue: #14593)
Discovery
  • Add two phased commit to Cluster State publishing #13062
Ingest
  • Merge feature/ingest branch into master branch #16049 (issue: #14049)
Mapping
Percolator
  • index the query terms from the percolator query #13646 (issue: #12664)
Plugin Ingest Attachment
Plugin Mapper Attachment
  • Migrate mapper attachments plugin to main repository #14605
Plugin Repository HDFS
Query DSL
  • Adds a rewrite phase to queries on the shard level #16870 (issue: #9526)
Reindex API
  • Merge reindex to master #16861
Scripting
  • Exceptions and Infinite Loop Checking #15936
  • Added a new scripting language (PlanA) #15136 (issue: #13084)
Search
  • Add search_after parameter in the SearchAPI #16125 (issue: #8192)
Settings
  • Add infrastructure to transactionally apply and reset dynamic settings #15278
Stats
Suggesters
  • Add document-oriented completion suggester #14410 (issue: #10746)
Task Manager
  • Add task cancellation mechanism #16320
  • Make the Task object available to the action caller #16033
  • Task Management: Add framework for registering and communicating with tasks #15347 (issue: #15117)

Enhancements

edit
Aggregations
  • Add tests and documentation for using time_zone in date range aggregation #16955 (issue: #10130)
  • Fixes serialisation of Ranges #16674
Allocation
  • Write shard state metadata as soon as shard is created / initializing #16625 (issue: #14739)
  • Reuse existing allocation id for primary shard allocation #16530 (issue: #14739)
  • Remove version in ShardRouting (now obsolete) #16243 (issue: #14739)
  • Prefer nodes that previously held primary shard for primary shard allocation #16096 (issue: #14739)
  • Extend reroute with an option to force assign stale primary shard copies #15708 (issue: #14739)
  • Allocate primary shards based on allocation IDs #15281 (issue: #14739)
  • Persist currently started allocation IDs to index metadata #14964 (issue: #14739)
  • Use ObjectParser to parse AllocationID #14962 (issue: #14831)
  • Persist allocation ID with shard state metadata on nodes #14831 (issue: #14739)
Analysis
  • Improve error message if resource files have illegal encoding #17237 (issue: #17212)
CAT API
  • Expose http address in cat/nodes #16770
  • [cat/recovery] Make recovery time a TimeValue() #16743 (issue: #9209)
  • :CAT API: remove space at the end of a line #15250 (issue: #9464)
CRUD
  • CRUD: Allow to get and set ttl as a time value/string #15047
Cache
Cluster
  • Cluster Health should run on applied states, even if waitFor=0 #17440
  • Resolve index names to Index instances early #17048
  • Remove DiscoveryNode#shouldConnectTo method #16898 (issue: #16815)
  • Fail demoted primary shards and retry request #16415 (issue: #14252)
  • Illegal shard failure requests #16275
  • Shard failure requests for non-existent shards #16089 (issue: #14252)
  • Add handling of channel failures when starting a shard #16041 (issue: #15895)
  • Wait for new master when failing shard #15748 (issue: #14252)
  • Master should wait on cluster state publication when failing a shard #15468 (issue: #14252)
  • Split cluster state update tasks into roles #14899 (issue: #13627)
  • Add timeout mechanism for sending shard failures #14707 (issue: #14252)
  • Add listener mechanism for failures to send shard failed #14295 (issue: #14252)
Core
  • Remove PROTOTYPE from BulkItemResponse.Failure #17433 (issue: #17086)
  • Throw an exception if Writeable.Reader reads null #17332
  • Remove PROTOTYPE from RescorerBuilders #17330
  • Port Primary Terms to master #17044 (issues: #14062, #14651, #17038)
  • Use index UUID to lookup indices on IndicesService #17001
  • Add -XX+AlwaysPreTouch JVM flag #16937
  • Add max size virtual memory check #16935
  • Use and test relative time in TransportBulkAction #16916
  • Bump Elasticsearch version to 5.0.0-SNAPSHOT #16862
  • Assert that we can write in all data-path on startup #16745
  • Add G1GC check on startup #16737 (issue: #10740)
  • Shards with heavy indexing should get more of the indexing buffer #14121
  • Remove and ban ImmutableMap #13939 (issue: #13224)
  • Finish banning ImmutableSet #13820 (issue: #13224)
  • Removes and bans ImmutableSet #13754 (issue: #13224)
  • Remove and ban ImmutableMap#entrySet #13724
  • Forbid ForwardingSet #13720 (issue: #13224)
Discovery
  • Add a dedicate queue for incoming ClusterStates #13303 (issue: #13062)
Engine
  • Remove writeLockTimeout from InternalEngine #16930
  • Don’t guard IndexShard#refresh calls by a check to isRefreshNeeded #16118
  • Never call a listerner under lock in InternalEngine #15786
  • Use System.nanoTime() to initialize Engine.lastWriteNanos #14321
  • Flush big merges automatically if shard is inactive #14275
  • Remove Engine.Create #13955
  • Remove the disabled autogenerated id optimization from InternalEngine #13857
Exceptions
  • Fix typos in exception/assert/log messages in core module. #16649
  • Add field names to several mapping errors #16508 (issue: #16378)
  • Add serialization support for more important IOExceptions #15766
  • Adds exception objects to log messages. #14827 (issue: #10021)
  • Add stack traces to logged exceptions where missing #13825 (issue: #10021)
  • Remove reflection hacks from ElasticsearchException #13796
  • Rename QueryParsingException to a more generic ParsingException #13631
  • Add *Exception(Throwable cause) constructors/ call where appropriate #13544 (issue: #10021)
Geo
  • Fix a potential parsing problem in GeoDistanceSortParser #17111
  • Geo: Add validation of shapes to ShapeBuilders #15551 (issue: #14416)
  • Make remaining ShapeBuilders implement Writeable #15010 (issue: #14416)
  • Geo: Remove internal translated flag from LineStringBuilder #14969
  • Make PointBuilder, CircleBuilder & EnvelopeBuilder implement Writable #14933 (issue: #14416)
  • Merging BaseLineString and BasePolygonBuilder with subclass #14887 (issue: #14482)
  • Moving static factory methods to ShapeBuilders #14529
  • Remove InternalLineStringBuilder and InternalPolygonBuilder #14482 (issue: #14416)
Highlighting
  • Switch Highlighting to ObjectParser #17363
  • Use HighlightBuilder in SearchSourceBuilder #15376 (issue: #15044)
  • Joint parsing of common global Hightlighter and subfield parameters #15368 (issue: #15285)
  • Enable HighlightBuilder to create SearchContextHighlight #15324
  • Add fromXContent method to HighlightBuilder #15157
Ingest
  • add automatic type conversion support to ConvertProcessor #17263 (issue: #17139)
  • Give the foreach processor access to the rest of the document #17172 (issue: #17147)
  • Added ingest statistics to node stats API #16915
  • Add ingest_took to bulk response #16876
  • Add ingest info to node info API, which contains a list of available processors #16865
  • Use diffs for ingest metadata in cluster state #16847
  • hide null-valued metadata fields from WriteableIngestDocument#toXContent #16557
  • Ingest: use bulk thread pool for bulk request processing (was index before) #16539 (issue: #16503)
  • Add foreach processor #16432
  • revert PipelineFactoryError handling with throwing ElasticsearchParseException in ingest pipeline creation #16355
  • Add processor tags to on_failure metadata in ingest pipeline #16324 (issue: #16202)
  • catch processor/pipeline factory exceptions and return structured error responses #16276 (issue: #16010)
  • Ingest: move get/put/delete pipeline methods to ClusterAdminClient #16242
  • Geoip processor: remove redundant latitude and longitude fields and make location an object with lat and lon subfields #16173
Internal
  • Remove PROTOTYPE from MLT.Item #17481 (issue: #17085)
  • Remove PROTOTYPE from VersionType #17480 (issue: #17085)
  • Remove PROTOTYPEs from highlighting #17466 (issue: #17085)
  • Remove PROTOTYPEs from ingest #17434 (issue: #17085)
  • Start to rework query registration #17424
  • Factor out slow logs into Search and IndexingOperationListeners #17398
  • Remove PROTOTYPE from Suggesters #17370
  • Remove PROTOTYPE from SortBuilders #17337 (issue: #17085)
  • Remove PROTOTYPE from ShapeBuilders #17336 (issue: #17085)
  • Replace FieldStatsProvider with a method on MappedFieldType. #17334
  • Stop using PROTOTYPE in NamedWriteableRegistry #17284 (issue: #17085)
  • Support scheduled commands in current context #17077
  • Thread limits #17003
  • Remove leniency from segments info integrity checks #16985 (issue: #16973)
  • Rename SearchServiceTransportAction to SearchTransportService #16880
  • Decouple the TransportService and ClusterService #16872 (issue: #16788)
  • Refactor bootstrap checks #16844 (issues: #16733, #16835)
  • Add LifecycleRunnable #16752
  • Hot inlined methods in your area #16725
  • Move IndicesQueryCache and IndicesRequestCache into IndicesService #16603
  • Forbid use of java.security.MessageDigest#clone() #16543 (issue: #16479)
  • Make IndicesWarmer a private class of IndexService #16470
  • Simplify IndicesFieldDataCache and detach from guice #16469
  • Uppercase ells (L) in long literals #16329 (issue: #16279)
  • ShardId equality and hash code inconsistency #16319 (issue: #16217)
  • Ensure all resources are closed on Node#close() #16316 (issue: #13685)
  • Make index uuid available in Index, ShardRouting & ShardId #16217
  • Move RefreshTask into IndexService and use since task per index #15933
  • Make IndexingMemoryController private to IndicesService #15877
  • Cleanup IndexingOperationListeners infrastructure #15875
  • Remove and forbid use of j.u.c.ThreadLocalRandom #15862 (issue: #15294)
  • Fix IntelliJ query builder type inference issues #15429
  • Remove and forbid use of Collections#shuffle(List) and Random#<init>() #15299 (issue: #15287)
  • Remove and forbid use of the type-unsafe empty Collections fields #15187
  • Move IndicesService.canDeleteShardContent to use IndexSettings #15150 (issue: #15059)
  • Simplify MonitorService construction and detach from guice #15035
  • Use Supplier for StreamInput#readOptionalStreamable #14806
  • Add variable-length long encoding #14780
  • Extend usage of IndexSetting class #14731 (issue: #14251)
  • Fold SimilarityModule into IndexModule #14284
  • Move to lucene BoostQuery #14264
  • Use built-in method for computing hash code of longs #14213
  • Refactor ShardFailure listener infrastructure #14206
  • Add methods for variable-length encoding integral arrays #14087
  • Fold IndexAliasesService into IndexService #14044
  • Remove unneeded Module abstractions #13944
  • Query refactoring: simplify IndexQueryParserService parse methods #13938 (issue: #13859)
  • Remove and forbid use of com.google.common.collect.Iterators #13916 (issue: #13224)
  • Remove and forbid use of com.google.common.collect.ImmutableCollection #13909 (issue: #13224)
  • Remove and forbid use of com.google.common.io.Resources #13908 (issue: #13224)
  • Remove and forbid use of com.google.common.hash.* #13907 (issue: #13224)
  • Remove and forbid use of com.google.common.net.InetAddresses #13905 (issue: #13224)
  • Remove and forbid use of com.google.common.collect.EvictingQueue #13903 (issue: #13224)
  • Replace Guava cache with simple concurrent LRU cache #13879
  • Remove ClusterSerivce and IndexSettingsService dependency from IndexShard #13853
  • Start making RecoverySourceHandler unittestable #13840
  • Remove IndexService dep. from IndexShard #13797
  • Remove ES internal deletion policies in favour of Lucenes implementations #13794
  • Move ShardTermVectorService to be on indices level as TermVectorService #13786
  • Move ShardPercolateService creation into IndexShard #13777
  • Remove ExpressionScriptCompilationException and ExpressionScriptExecutionException #13742
  • Reduced the number of ClusterStateUpdateTask variants #13735
  • Add a BaseParser helper for stream parsing #13615
  • Remove and forbid use of com.google.common.primitives.Ints #13596 (issue: #13224)
  • Remove and forbid use of com.google.common.math.LongMath #13575 (issue: #13224)
  • Remove and forbid use of com.google.common.base.Joiner #13572 (issue: #13224)
  • Replace and ban next batch of Guava classes #13562 (issue: #13224)
  • Remove and forbid use of com.google.common.collect.Iterables #13559 (issue: #13224)
  • Replace LoadingCache usage with a simple ConcurrentHashMap #13552 (issue: #13224)
  • Use Supplier instead of Reflection #13545
  • Remove and forbid use of com.google.common.base.Preconditions #13540 (issue: #13224)
  • Remove and forbid use of guava Function, Charsets, Collections2 #13533 (issue: #13224)
  • Remove and forbid use of com.google.common.collect.ImmutableSortedMap #13525 (issue: #13224)
  • Remove and forbid use of several com.google.common.util. classes #13524 (issue: #13224)
  • Cleanup SearchRequest & SearchRequestBuilder #13518
  • Remove and forbid use of com.google.common.collect.Queues #13498 (issue: #13224)
  • Remove and forbid use of com.google.common.base.Preconditions#checkNotNull #13493 (issue: #13224)
  • Remove and forbid use of com.google.common.collect.Sets #13463 (issue: #13224)
  • Remove and forbid use of com.google.common.collect.Maps #13438 (issue: #13224)
  • Remove use of underscore as an identifier #13353
  • Remove and forbid the use of com.google.common.base.Predicate(s)? #13349 (issues: #13224, #13314)
  • This commit removes com.google.common.io #13302 (issue: #13224)
Java API
  • Remove copy constructors from request classes and TransportMessage type #16640 (issue: #15776)
Mapping
  • Add a soft limit on the mapping depth. #17400
  • Disable fielddata on text fields by defaults. #17386
  • Add limit to total number of fields in mapping #17357
  • Make parseMultiField part of parseField. #17313
  • Automatically add a sub keyword field to string dynamic mappings. #17188
  • Remove friction from the mapping changes in 5.0. #16991
  • Rework norms parameters for 5.0. #16987
  • Moved dynamic field handling in doc parsing to end of parsing #16798
  • Remove the MapperBuilders utility class. #16609
  • Make the index property a boolean. #16161
  • Remove the ability to enable doc values with the fielddata.format setting. #16147
  • Be stricter about parsing boolean values in mappings. #16146
  • Fix default doc values to be enabled when a field is not indexed. #16141
  • Dynamically map floating-point numbers as floats instead of doubles. #15319 (issue: #13851)
  • Simplify MetaDataMappingService. #15217
  • Remove MergeMappingException. #15177
Packaging
  • Added RPM metadata #17477
  • Elasticsearch ownership for data, logs, and configs #17197 (issue: #12688)
  • Fail early on JDK with compiler bug #16418 (issues: #16097, #16362)
  • Make security non-optional #16176
  • Remove RuntimePermission("accessDeclaredMembers") #15378
  • Remove Guava as a dependency #14055 (issue: #13224)
  • Remove Guava as a dependency #14054 (issue: #13224)
Percolator
  • Add scoring support to the percolator query #17385 (issue: #13827)
  • Add query extract support for the blended term query and the common terms query #17347
  • Add support for several span queries in ExtractQueryTermsService #17323
  • Add support for TermsQuery in ExtractQueryTermsService #17316
  • Replace percolate APIs with a percolator query #16349 (issues: #10741, #11264, #13176, #13978, #4317, #7297)
Plugin Analysis Kuromoji
  • Add nbest options and NumberFilter #17173
Plugin Discovery EC2
  • Add support for proxy authentication for s3 and ec2 #15293 (issue: #15268)
Plugin Ingest Attachment
  • Minor attachment processor improvements #16574
Plugin Lang Painless
Plugin Mapper Attachment
  • minor attachments cleanups: IDE test support and EPUB format #14626
Plugin Repository Azure
  • Support global repositories.azure. settings #15141 (issue: #13776)
  • Add timeout settings (default to 5 minutes) #15080 (issue: #14277)
  • Remove AbstractLegacyBlobContainer #14650 (issue: #13434)
Plugin Repository HDFS
  • merge current hdfs improvements to master #15588
Plugin Repository S3
  • Check that S3 setting buffer_size is always lower than chunk_size #17274 (issue: #17244)
Plugins
  • PluginManager: Add xpack as official plugin #17227
  • CliTool: Cleanup and document Terminal #16443
  • Plugin cli: Improve maven coordinates detection #16384 (issue: #16376)
  • Enforce plugin zip does not contain zip entries outside of the plugin dir #16361
  • CliTool: Allow unexpected exceptions to propagate #16359
  • Reduce complexity of plugin cli #16336
  • Remove Plugin.onIndexService. #15029 (issue: #14896)
  • Open up QueryCache and SearcherWrapper extension points #14303
Query DSL
  • An exists query on an object should query a single term. #17186 (issue: #17131)
  • Function Score Query: make parsing stricter #16617 (issue: #16583)
  • Parsers should throw exception on unknown objects #14255 (issue: #10974)
  • UNICODE_CHARACTER_CLASS fix #11598 (issue: #10146)
Query Refactoring
  • Add infrastructure to rewrite query builders #16599
  • Switch geo validation to enum #13672 (issue: #13608)
REST
  • More robust handling of CORS HTTP Access Control #16092
  • Add option to exclude based on paths in XContent #16017
Recovery
  • Recover broken IndexMetaData as closed #17187
  • Relocation source should be marked as relocating before starting recovery to primary relocation target #16500
  • Operation counter for IndexShard #15956 (issue: #15900)
  • Primary relocation handoff #15900 (issue: #15532)
  • Remove recovery threadpools and throttle outgoing recoveries on the master #15372
  • Refactor StoreRecoveryService to be a simple package private util class #13766
Reindex API
  • Make reindex throttling dynamic #17262
  • Throttling support for reindex #17039
  • Add ingest pipeline support to reindex #16932
Scripting
  • Skipping hidden files compilation for script service #16286 (issue: #15269)
  • Rename Plan A to Painless #16245
  • Add plumbing for script compile-time parameters #15464
  • Factor mustache → modules/lang-mustache #15328
Search
  • Add a soft limit on the number of shards that can be queried in a single search request. #17396
  • Type filters should not have a performance impact when there is a single type. #17350
  • Store _all payloads on 1 byte instead of 4. #16899
  • Refuse to load fields from _source when using the fields option and support wildcards. #15017 (issues: #10783, #14489)
  • Add response into ClearScrollResponse #13835 (issue: #13817)
  • Shuffle shards for _only_nodes + support multiple specifications like cluster API #12575 (issues: #12546, #12700)
Search Refactoring
  • Remove RescoreParseElement #17441
  • Remove HighlighterParseElement #17303
  • Move top level parsing of sort element to SortBuilder #17248
  • Switch to using refactored SortBuilder instead of using BytesReference in serialization #17205 (issues: #17146, #17257)
  • Add build() method to SortBuilder implementations #17146 (issue: #10217)
  • Refactoring of Suggestions #17096 (issue: #10217)
  • Move sort order field up into SortBuilder #17035
  • Moves SortParser:parse(…​) to only require QueryShardContext #16999 (issue: #15178)
  • Change internal representation of suggesters #16873
  • Make GeoDistanceSortBuilder serializable, 2nd try #16572 (issues: #15178, #16151)
  • Move missing() from SortBuilder interface to class #16225 (issues: #15178, #16151)
  • Remove deprecated parameters from ScriptSortBuilder #16153 (issue: #15178)
  • Refactor GeoSortBuilder #16151 (issue: #15178)
  • Refactor FieldSortBuilder #16127 (issue: #15178)
  • Make sort order enum writable. #16124 (issue: #15178)
  • Make DistanceUnit writable. #16122 (issue: #15178)
  • RescoreBuilder: Add parsing and creating of RescoreSearchContext #16014 (issue: #15559)
  • Make RescoreBuilder and nested QueryRescorer Writable #15953 (issue: #15559)
  • Explain api: move query parsing to the coordinating node #14270
  • Switch query parsers to use ParseField #14249 (issue: #8964)
  • Refactoring of Aggregations #14136
Settings
  • Add guard against null-valued settings #17310 (issue: #17292)
  • Useful error message for null property placeholder #17293 (issue: #17292)
  • Archive cluster level settings if unknown or broken #17246
  • Improve error message if setting is not found #17230
  • Improve upgrade experience of node level index settings #17223 (issue: #17187)
  • Settings with complex matchers should not overlap #16754
  • Moves GCE settings to the new infra #16722 (issue: #16720)
  • Add filtering support within Setting class #16629 (issue: #16598)
  • Migrate AWS settings to new settings infrastructure #16602 (issue: #16293)
  • Remove gateway.initial_meta and always rely on min master nodes #16446
  • Rewrite SettingsFilter to be immutable #16425
  • Simplify azure settings #16363
  • Convert PageCacheRecycler settings #16341
  • Monitor settings #16313
  • Cut over tribe node settings to new settings infra #16311
  • Convert multcast plugin settings to the new infra #16295
  • Convert request.headers.* to the new settings infra #16292
  • Migrate Azure settings to new settings infrastructure #16291
  • Validate logger settings and allow them to be reset via API #16289
  • Switch NodeEnvironment’s settings to new settings #16273
  • Simplify AutoCreateIndex and add more tests #16270
  • Convert several pending settings #16269
  • Migrate query caching settings to the new settings infra. #16267
  • Convert action.auto_create_index and action.master.force_local to the new settings infra #16263
  • Convert cluster.routing.allocation.type and processors to the new settings infra. #16238
  • Validate tribe node settings on startup #16237
  • Move node.client, node.data, node.master, node.local and node.mode to new settings infra #16230
  • Moved http settings to the new settings infrastructure #16188
  • Migrate network service to the new infra #16187
  • Convert client.transport settings to new infra #16183
  • Move discovery.* settings to new Setting infrastructure #16182
  • Change over to o.e.common.settings.Setting for http settings #16181
  • Convert "path.*" and "pidfile" to new settings infra #16180
  • Migrate repository settings to the new settings API #16178
  • Convert "indices.*" settings to new infra. #16177
  • Migrate gateway settings to the new settings API. #16175
  • Convert several node and test level settings #16172
  • Run Metadata upgrade tool on every version #16168
  • Check for invalid index settings on metadata upgrade #16156
  • Validate the settings key if it’s simple chars separated by . #16120
  • Validate known global settings on startup #16091
  • Cut over all index scope settings to the new setting infrastrucuture #16054 (issues: #12790, #12854, #16032, #6732)
  • Remove updatability of index.flush_on_close #15964 (issue: #15955)
  • Move all dynamic settings and their config classes to the index level #15955 (issue: #6732)
  • Always require units for bytes and time settings #15948 (issue: #11437)
  • Make MetaData parsing less lenient. #15828
  • Move async translog sync logic into IndexService #15584
  • Remove index.merge.scheduler.notify_on_failure and default to true #15572 (issue: #15570)
  • Remove cache concurrency level settings that no longer apply #14210 (issues: #13224, #13717, #7836)
Similarities
  • Defining a global default similarity #16682 (issue: #16594)
Stats
Store
Task Manager
  • Add ability to group tasks by common parent #17341
  • Add start time and duration to tasks #16829
  • Combine node name and task id into single string task id #16744
  • Add task status #16356 (issue: #16344)
  • Extend tracking of parent tasks to master node, replication and broadcast actions #15931
Translog
  • Remove ChannelReference and simplify Views #15898
  • Simplify TranslogWriter to always write to a stream #15771
  • Remove TranslogService and fold it into synchronous IndexShard API #13707

Bug fixes

edit
Aggregations
  • Fixes the defaults for keyed in the percentiles aggregations #17217
  • Correct typo in class name of StatsAggregator #15264 (issue: #14730)
Allocation
  • Replica shards must be failed before primary shards #15686
CRUD
  • Prevent TransportReplicationAction to route request based on stale local routing table #16274 (issues: #12573, #12574)
  • Resolves the conflict between alias routing and parent routing by applying the alias routing and ignoring the parent routing. #15371 (issue: #3068)
Cluster
  • Shard state action channel exceptions #16057 (issue: #15748)
Core
  • Handle RejectedExecution gracefully in TransportService during shutdown #16965
Geo
  • Fix multi-field support for GeoPoint types #15702 (issue: #15701)
  • Enforce distance in distance query is > 0 [ISSUE] #15135
Ingest
  • Handle regex parsing errors in Gsub and Grok Processors #17260
  • add on_failure exception metadata to ingest document for verbose simulate #16562
  • The IngestDocument copy constructor should make a deep copy #16248 (issue: #16246)
Internal
  • Enable unmap hack for java 9 #16986 (issue: #1)
  • Fix issues with failed cache loads #14315
  • Allow parser to move on the START_OBJECT token when parsing search source #14145
  • Ensure searcher is release if wrapping fails #14107
  • Avoid deadlocks in Cache#computeIfAbsent #14091 (issue: #14090)
Java API
  • Fix potential NPE in SearchSourceBuilder #16905 (issue: #16902)
Mapping
  • Make dynamic template parsing less lenient. #17249
  • Fix dynamic mapper when its parent already has an update #17065
  • Fix copy_to when the target is a dynamic object field. #15216 (issues: #111237, #11237)
  • Preserve existing mappings on batch mapping updates #15130 (issues: #14899, #15129)
Packaging
  • Set MAX_OPEN_FILES to 65536 #17431 (issue: #17430)
  • [windows] Service command still had positional start command #17391
  • Do not pass double-dash arguments on startup #17087 (issue: #17084)
Percolator
  • Let PercolatorQuery’s explain use the two phase iterator #17315 (issue: #17314)
Plugin Store SMB
  • Fix calling ensureOpen() on the wrong directory (master forwardport) #16395 (issue: #16383)
Plugins
  • CliTool: Messages printed in Terminal should have percent char escaped #16367
Query DSL
  • Resolve string dates and date math to millis before evaluating for rewrite in range query #17239
  • constant_score query should throw error on more than one filter #17135 (issue: #17126)
  • Single IPv4 addresses in IP field term queries #16068 (issue: #16058)
  • Make strategy optional in GeoShapeQueryBuilder readFrom and writeTo #13963
Query Refactoring
  • Query refactoring: set has_parent & has_child types context properly #13863
  • Make sure equivalent geohashCellQueries are equal after toQuery called #13792
Recovery
  • Invoke IndexingOperationListeners also when recovering from store or remote #17406
  • Prevent interruption while store checks lucene files for consistency #16308
  • Mark shard as recovering on the cluster state thread #14276 (issues: #13766, #14115)
Search
Settings
  • Register bootstrap settings #16513
  • Add settings filtering to node info requests #16445
  • Ban write access to system properties #14914
Task Manager
  • Take filterNodeIds into consideration while sending task requests to nodes #17081
Translog
  • Move translog recover outside of the engine #17422
  • Mark shard active during recovery; push settings after engine finally inits #16250 (issues: #14121, #16209)

Upgrades

edit
Core
  • Upgrade to lucene-6.0.0-f0aa4fc. #17075
  • upgrade to lucene 6.0.0-snapshot-bea235f #16964
  • Upgrade to Jackson 2.7.1 #16801 (issue: #16294)
Ingest
  • Update MaxMind geoip2 version to 2.6 #16837 (issue: #16801)
Internal
  • Bump master (3.0-snapshot) to java 8 #13314
Search Templates
  • Update mustache.java to version 0.9.1 #14053 (issue: #13224)
« 5.0.0-alpha2 Release Notes 5.0.0-alpha1 Release Notes (Changes previously released in 2.x) »