- Elasticsearch Guide: other versions:
- Getting Started
- Set up Elasticsearch
- Installing Elasticsearch
- Configuring Elasticsearch
- Important Elasticsearch configuration
- Important System Configuration
- Bootstrap Checks
- Heap size check
- File descriptor check
- Memory lock check
- Maximum number of threads check
- Max file size check
- Maximum size virtual memory check
- Maximum map count check
- Client JVM check
- Use serial collector check
- System call filter check
- OnError and OnOutOfMemoryError checks
- Early-access check
- G1GC check
- All permission check
- Starting Elasticsearch
- Stopping Elasticsearch
- Adding nodes to your cluster
- Installing X-Pack
- Set up X-Pack
- Configuring X-Pack Java Clients
- X-Pack Settings
- Bootstrap Checks for X-Pack
- Upgrade Elasticsearch
- API Conventions
- Document APIs
- Search APIs
- Aggregations
- Metrics Aggregations
- Avg Aggregation
- Weighted Avg Aggregation
- Cardinality Aggregation
- Extended Stats Aggregation
- Geo Bounds Aggregation
- Geo Centroid Aggregation
- Max Aggregation
- Min Aggregation
- Percentiles Aggregation
- Percentile Ranks Aggregation
- Scripted Metric Aggregation
- Stats Aggregation
- Sum Aggregation
- Top Hits Aggregation
- Value Count Aggregation
- Bucket Aggregations
- Adjacency Matrix Aggregation
- Auto-interval Date Histogram Aggregation
- Intervals
- Children Aggregation
- Composite Aggregation
- Date Histogram Aggregation
- Date Range Aggregation
- Diversified Sampler Aggregation
- Filter Aggregation
- Filters Aggregation
- Geo Distance Aggregation
- GeoHash grid Aggregation
- Global Aggregation
- Histogram Aggregation
- IP Range Aggregation
- Missing Aggregation
- Nested Aggregation
- Range Aggregation
- Reverse nested Aggregation
- Sampler Aggregation
- Significant Terms Aggregation
- Significant Text Aggregation
- Terms Aggregation
- Pipeline Aggregations
- Avg Bucket Aggregation
- Derivative Aggregation
- Max Bucket Aggregation
- Min Bucket Aggregation
- Sum Bucket Aggregation
- Stats Bucket Aggregation
- Extended Stats Bucket Aggregation
- Percentiles Bucket Aggregation
- Moving Average Aggregation
- Moving Function Aggregation
- Cumulative Sum Aggregation
- Bucket Script Aggregation
- Bucket Selector Aggregation
- Bucket Sort Aggregation
- Serial Differencing Aggregation
- Matrix Aggregations
- Caching heavy aggregations
- Returning only aggregation results
- Aggregation Metadata
- Returning the type of the aggregation
- Metrics Aggregations
- Indices APIs
- Create Index
- Delete Index
- Get Index
- Indices Exists
- Open / Close Index API
- Shrink Index
- Split Index
- Rollover Index
- Put Mapping
- Get Mapping
- Get Field Mapping
- Types Exists
- Index Aliases
- Update Indices Settings
- Get Settings
- Analyze
- Index Templates
- Indices Stats
- Indices Segments
- Indices Recovery
- Indices Shard Stores
- Clear Cache
- Flush
- Refresh
- Force Merge
- cat APIs
- Cluster APIs
- Query DSL
- Mapping
- Analysis
- Anatomy of an analyzer
- Testing analyzers
- Analyzers
- Normalizers
- Tokenizers
- Standard Tokenizer
- Letter Tokenizer
- Lowercase Tokenizer
- Whitespace Tokenizer
- UAX URL Email Tokenizer
- Classic Tokenizer
- Thai Tokenizer
- NGram Tokenizer
- Edge NGram Tokenizer
- Keyword Tokenizer
- Pattern Tokenizer
- Char Group Tokenizer
- Simple Pattern Tokenizer
- Simple Pattern Split Tokenizer
- Path Hierarchy Tokenizer
- Path Hierarchy Tokenizer Examples
- Token Filters
- Standard Token Filter
- ASCII Folding Token Filter
- Flatten Graph Token Filter
- Length Token Filter
- Lowercase Token Filter
- Uppercase Token Filter
- NGram Token Filter
- Edge NGram Token Filter
- Porter Stem Token Filter
- Shingle Token Filter
- Stop Token Filter
- Word Delimiter Token Filter
- Word Delimiter Graph Token Filter
- Multiplexer Token Filter
- Conditional Token Filter
- Predicate Token Filter Script
- Stemmer Token Filter
- Stemmer Override Token Filter
- Keyword Marker Token Filter
- Keyword Repeat Token Filter
- KStem Token Filter
- Snowball Token Filter
- Phonetic Token Filter
- Synonym Token Filter
- Synonym Graph Token Filter
- Compound Word Token Filters
- Reverse Token Filter
- Elision Token Filter
- Truncate Token Filter
- Unique Token Filter
- Pattern Capture Token Filter
- Pattern Replace Token Filter
- Trim Token Filter
- Limit Token Count Token Filter
- Hunspell Token Filter
- Common Grams Token Filter
- Normalization Token Filter
- CJK Width Token Filter
- CJK Bigram Token Filter
- Delimited Payload Token Filter
- Keep Words Token Filter
- Keep Types Token Filter
- Exclude mode settings example
- Classic Token Filter
- Apostrophe Token Filter
- Decimal Digit Token Filter
- Fingerprint Token Filter
- Minhash Token Filter
- Remove Duplicates Token Filter
- Character Filters
- Modules
- Index Modules
- Ingest Node
- Pipeline Definition
- Ingest APIs
- Accessing Data in Pipelines
- Conditional Execution in Pipelines
- Handling Failures in Pipelines
- Processors
- Append Processor
- Bytes Processor
- Convert Processor
- Date Processor
- Date Index Name Processor
- Dissect Processor
- Drop Processor
- Dot Expander Processor
- Fail Processor
- Foreach Processor
- Grok Processor
- Gsub Processor
- Join Processor
- JSON Processor
- KV Processor
- Lowercase Processor
- Pipeline Processor
- Remove Processor
- Rename Processor
- Script Processor
- Set Processor
- Set Security User Processor
- Split Processor
- Sort Processor
- Trim Processor
- Uppercase Processor
- URL Decode Processor
- SQL Access
- Monitor a cluster
- Rolling up historical data
- Set up a cluster for high availability
- Secure a cluster
- Overview
- Configuring security
- Encrypting communications in Elasticsearch
- Encrypting communications in an Elasticsearch Docker Container
- Enabling cipher suites for stronger encryption
- Separating node-to-node and client traffic
- Configuring an Active Directory realm
- Configuring a file realm
- Configuring an LDAP realm
- Configuring a native realm
- Configuring a PKI realm
- Configuring a SAML realm
- Configuring a Kerberos realm
- FIPS 140-2
- Security settings
- Security files
- Auditing settings
- How security works
- User authentication
- Built-in users
- Internal users
- Realms
- Realm chains
- Active Directory user authentication
- File-based user authentication
- LDAP user authentication
- Native user authentication
- PKI user authentication
- SAML authentication
- Kerberos authentication
- Integrating with other authentication systems
- Enabling anonymous access
- Controlling the user cache
- Configuring SAML single-sign-on on the Elastic Stack
- User authorization
- Auditing security events
- Encrypting communications
- Restricting connections with IP filtering
- Cross cluster search, tribe, clients, and integrations
- Tutorial: Getting started with security
- Tutorial: Encrypting communications
- Troubleshooting
- Can’t log in after upgrading to 6.5.4
- Some settings are not returned via the nodes settings API
- Authorization exceptions
- Users command fails due to extra arguments
- Users are frequently locked out of Active Directory
- Certificate verification fails for curl on Mac
- SSLHandshakeException causes connections to fail
- Common SSL/TLS exceptions
- Common Kerberos exceptions
- Common SAML issues
- Internal Server Error in Kibana
- Setup-passwords command fails due to connection failure
- Failures due to relocation of the configuration files
- Limitations
- Alerting on Cluster and Index Events
- Command line tools
- How To
- Testing
- Glossary of terms
- X-Pack APIs
- Info API
- Cross-cluster replication APIs
- Explore API
- Licensing APIs
- Migration APIs
- Machine learning APIs
- Add events to calendar
- Add jobs to calendar
- Close jobs
- Create calendar
- Create datafeeds
- Create filter
- Create jobs
- Delete calendar
- Delete datafeeds
- Delete events from calendar
- Delete filter
- Delete forecast
- Delete jobs
- Delete jobs from calendar
- Delete model snapshots
- Find file structure
- Flush jobs
- Forecast jobs
- Get calendars
- Get buckets
- Get overall buckets
- Get categories
- Get datafeeds
- Get datafeed statistics
- Get influencers
- Get jobs
- Get job statistics
- Get machine learning info
- Get model snapshots
- Get scheduled events
- Get filters
- Get records
- Open jobs
- Post data to jobs
- Preview datafeeds
- Revert model snapshots
- Start datafeeds
- Stop datafeeds
- Update datafeeds
- Update filter
- Update jobs
- Update model snapshots
- Rollup APIs
- Security APIs
- Authenticate
- Change passwords
- Clear cache
- Clear roles cache
- Create or update application privileges
- Create or update role mappings
- Create or update roles
- Create or update users
- Delete application privileges
- Delete role mappings
- Delete roles
- Delete users
- Disable users
- Enable users
- Get application privileges
- Get role mappings
- Get roles
- Get token
- Get users
- Has privileges
- Invalidate token
- SSL certificate
- Watcher APIs
- Definitions
- Release Highlights
- Breaking changes
- Release Notes
- Elasticsearch version 6.5.4
- Elasticsearch version 6.5.3
- Elasticsearch version 6.5.2
- Elasticsearch version 6.5.1
- Elasticsearch version 6.5.0
- Elasticsearch version 6.4.3
- Elasticsearch version 6.4.2
- Elasticsearch version 6.4.1
- Elasticsearch version 6.4.0
- Elasticsearch version 6.3.2
- Elasticsearch version 6.3.1
- Elasticsearch version 6.3.0
- Elasticsearch version 6.2.4
- Elasticsearch version 6.2.3
- Elasticsearch version 6.2.2
- Elasticsearch version 6.2.1
- Elasticsearch version 6.2.0
- Elasticsearch version 6.1.4
- Elasticsearch version 6.1.3
- Elasticsearch version 6.1.2
- Elasticsearch version 6.1.1
- Elasticsearch version 6.1.0
- Elasticsearch version 6.0.1
- Elasticsearch version 6.0.0
- Elasticsearch version 6.0.0-rc2
- Elasticsearch version 6.0.0-rc1
- Elasticsearch version 6.0.0-beta2
- Elasticsearch version 6.0.0-beta1
- Elasticsearch version 6.0.0-alpha2
- Elasticsearch version 6.0.0-alpha1
- Elasticsearch version 6.0.0-alpha1 (Changes previously released in 5.x)
IMPORTANT: No additional bug fixes or documentation updates
will be released for this version. For the latest information, see the
current release documentation.
Elasticsearch version 6.0.0-beta1
editElasticsearch version 6.0.0-beta1
editBreaking Changes
edit- Aggregations
- Aliases
- Highlighting
-
- Remove the postings highlighter and make unified the default highlighter choice #25028
- Index APIs
- Indexed Scripts/Templates
-
- Scripting: Remove search template actions #25717
- Ingest
- Java REST Client
- Packaging
-
- Remove support for ES_INCLUDE #25804
- Setup: Change default heap to 1G #25695
- Use config directory to find jvm.options #25679 (issue: #23004)
- Remove implicit 32-bit support #25435
- Remove default path settings #25408 (issue: #25357)
- Remove path.conf setting #25392 (issue: #25357)
- Honor masking of systemd-sysctl.service #24234 (issues: #21899, #806)
- Plugin Analysis ICU
- Plugin Discovery Azure Classic
- Plugin Repository GCS
-
- GCS Repository: Remove specifying credential file on disk #24727
- Plugins
-
- Make plugin loading stricter #25405
- Query DSL
- REST
- Scripting
- Search
- Security
-
- A new bootstrap check enforces that TLS/SSL is required for inter-node communication when running in production mode. See Encrypting Communications.
-
A new bootstrap check enforces that HTTPS is used by the built-in token
service when running in
production mode.
To disable the token service, set
xpack.security.authc.token.enabled
tofalse
in yourelasticsearch.yml
. See Token service settings.
- Settings
- Similarities
Breaking Java Changes
edit- Aggregations
- Internal
- Java API
- Java High Level REST Client
- Java REST Client
- Plugin Delete By Query
-
- Move DeleteByQuery and Reindex requests into core #24578
- Query DSL
- REST
Deprecations
edit- Index APIs
- Indexed Scripts/Templates
- Percolator
-
- Deprecate percolate query’s document_type parameter. #25199
- Scripting
-
- Scripting: Change keys for inline/stored scripts to source/id #25127
- Scripting: Deprecate native scripts #24692 (issue: #19966)
- Scripting: Deprecate index lookup #24691 (issue: #19359)
- Deprecate Fine Grain Settings for Scripts #24573 (issue: #24532)
- Scripting: Deprecate file script settings #24555 (issue: #21798)
- Scripting: Deprecate file scripts #24552 (issue: #21798)
- Settings
- Tribe Node
New Features
editEnhancements
edit- Aggregations
-
- Add strict parsing of aggregation ranges #25769
- Adds rewrite phase to aggregations #25495 (issue: #17676)
- Tweak AggregatorBase.addRequestCircuitBreakerBytes #25162 (issue: #24511)
- Add superset size to Significant Term REST response #24865
- Add document count to Matrix Stats aggregation response #24776
- Adds an implementation of LogLogBeta for the cardinality aggregation #22323 (issue: #22230)
- Allocation
- Analysis
- CAT API
- Cluster
- Core
-
- Add max file size bootstrap check #25974
- Add compatibility versions to main action response #25799
- Index ids in binary form. #25352 (issues: #18154, #24615)
- Explicitly reject duplicate data paths #25178
- Use SPI in High Level Rest Client to load XContent parsers #25097
- Upgrade to lucene-7.0.0-snapshot-a0aef2f #24775
- Speed up PK lookups at index time. #19856
- Engine
- Exceptions
-
- IllegalStateException: Only duplicated jar instead of classpath #24953
- Highlighting
- Index APIs
- Ingest
-
- Add Ingest-Processor specific Rest Endpoints & Add Grok endpoint #25059 (issue: #24725)
- Port support for commercial GeoIP2 databases from Logstash. #24889
-
add
exclude_keys
option to KeyValueProcessor #24876 (issue: #23856) - Allow removing multiple fields in ingest processor #24750 (issue: #24622)
- Add target_field parameter to ingest processors #24133 (issues: #23228, #23682)
- Inner Hits
- Internal
-
- Cleanup IndexFieldData visibility #25900
- Bump the min compat version to 5.6.0 #25805
- "shard started" should show index and shard ID #25157
- Break out clear scroll logic from TransportClearScrollAction #25125 (issue: #25094)
- Add helper methods to TransportActionProxy to identify proxy actions and requests #25124
- Add remote cluster infrastructure to fetch discovery nodes. #25123 (issue: #25094)
- Add the ability to set eager_global_ordinals in the new parent-join field #25019
- Disallow multiple parent-join fields per mapping #25002
- Remove the need for _UNRELEASED suffix in versions #24798 (issue: #24768)
- Optimize the order of bytes in uuids for better compression. #24615 (issue: #18209)
- Java API
- Java High Level REST Client
-
- [DOCS] restructure java clients docs pages #25517
- Use SPI in High Level Rest Client to load XContent parsers #25098 (issues: #25024, #25097)
- Add support for clear scroll to high level REST client #25038
- Add search scroll method to high level REST client #24938 (issue: #23331)
- Add search method to high level REST client #24796 (issues: #24794, #24795)
- Java REST Client
- Logging
- Mapping
- Network
- Packaging
-
- Remove memlock suggestion from systemd service #25979
- Set address space limit in systemd service file #25975
- Version option should display if snapshot #25970
- Ignore JVM options before checking Java version #25969
- Also skip JAVA_TOOL_OPTIONS on Windows #25968
- Introduce elasticsearch-env for Windows #25958
- Introduce elasticsearch-env #25815 (issue: #20286)
- Stop exporting HOSTNAME from scripts #25807
- Parent/Child
- Plugin Lang Painless
-
- Allow Custom Whitelists in Painless #25557
- Update Painless to Allow Augmentation from Any Class #25360
- Add Needs Methods to Painless Script Context Factories #25267
- Support Script Context Stateful Factory in Painless #25233
- Generate Painless Factory for Creating Script Instances #25120
- Update Painless to Use New Script Contexts #25015
- Optimize instance creation in LambdaBootstrap #24618
- Plugin Repository GCS
-
- GCS Repository: Add secure storage of credentials #24697
- Plugin Repository S3
-
- S3 Repository: Add back repository level credentials #24609
- Plugins
-
- Move tribe to a module #25778
- Plugins can register pre-configured char filters #25000 (issue: #23658)
- Add purge option to remove plugin CLI #24981
- Allow plugins to register pre-configured tokenizers #24751 (issues: #24223, #24572)
- Move ReindexAction class to core #24684 (issue: #24578)
- Make PreConfiguredTokenFilter harder to misuse #24572 (issue: #23658)
- Query DSL
- REST
- Recovery
- Scripting
- Search
-
- Rewrite search requests on the coordinating nodes #25814 (issue: #25791)
- Ensure query resources are fetched asynchronously during rewrite #25791
- Introduce a new Rewriteable interface to streamline rewriting #25788
-
Reduce the scope of
QueryRewriteContext
#25787 - Reduce the overhead of timeouts and low-level search cancellation. #25776
- Reduce profiling overhead. #25772 (issue: #24799)
-
Prevent
can_match
requests from sending to incompatible nodes #25705 (issue: #25704) - Add a shard filter search phase to pre-filter shards based on query rewriting #25658
-
Ensure we rewrite common queries to
match_none
if possible #25650 - Limit the number of concurrent shard requests per search request #25632
- Add cluster name validation to RemoteClusterConnection #25568
- Speed up sorted scroll when the index sort matches the search sort #25138 (issue: #6720)
- Leverage scorerSupplier when applicable. #25109
- Add Cross Cluster Search support for scroll searches #25094
- Track EWMA[1] of task execution time in search threadpool executor #24989 (issue: #24915)
- Query range fields by doc values when they are expected to be more efficient than points #24823 (issue: #24314)
- Search: Fairer balancing when routing searches by session ID #24671 (issue: #24642)
- Sequence IDs
-
- Move primary term from ReplicationRequest to ConcreteShardRequest #25822
- Add reason to global checkpoint updates on replica #25612 (issue: #10708)
- Introduce primary/replica mode for GlobalCheckPointTracker #25468
- Throw back replica local checkpoint on new primary #25452 (issues: #10708, #25355)
- Update global checkpoint when increasing primary term on replica #25422 (issues: #10708, #25355)
- Enable a long translog retention policy by default #25294 (issues: #10708, #25147)
- Introduce primary context #25122 (issues: #10708, #25355)
- Block older operations on primary term transition #24779 (issue: #10708)
- Settings
-
- Add disk threshold settings validation #25600 (issue: #25560)
- Enable cross-setting validation #25560 (issue: #25541)
-
Validate
transport.profiles.*
settings #25508 - Cleanup network / transport related settings #25489
- Emit settings deprecation logging at most once #25457
- IndexMetaData: Introduce internal format index setting #25292
- Snapshot/Restore
-
- Improves snapshot logging and snapshot deletion error handling #25264
- Stats
- Translog
Bug Fixes
edit- Aggregations
- Aliases
- Analysis
- Circuit Breakers
- Core
- Discovery
- Engine
- Geo
-
- Fix typo in GeoUtils#isValidLongitude #25121
- Highlighting
- Index APIs
- Ingest
- Inner Hits
- Internal
-
- Fix BytesReferenceStreamInput#skip with offset #25634
- Fix race condition in RemoteClusterConnection node supplier #25432
- Initialise empty lists in BaseTaskResponse constructor #25290
- Extract a common base class for scroll executions #24979 (issue: #16555)
- Obey lock order if working with store to get metadata snapshots #24787 (issue: #24481)
- Fix Version based BWC and set correct minCompatVersion #24732
-
Fix
_field_caps
serialization in order to support cross cluster search #24722 - Avoid race when shutting down controller processes #24579
- Mapping
-
- Fix parsing of ip range queries. #25768 (issue: #25636)
- Disable date field mapping changing #25285 (issue: #25271)
- Correctly enable _all for older 5.x indices #25087 (issue: #25068)
- token_count datatype should handle null value #25046 (issue: #24928)
- keep _parent field while updating child type mapping #24407 (issue: #23381)
- More Like This
- Nested Docs
-
- In case of a single type the _id field should be added to the nested document instead of _uid field #25149
- Network
- Packaging
-
- Exit Windows scripts promptly on failure #25959
- Pass config path as a system property #25943
- ES_HOME needs to be made absolute before attempt at traversal #25865
- Fix elasticsearch-keystore handling of path.conf #25811
- Stop disabling explicit GC #25759
- Avoid failing install if system-sysctl is masked #25657 (issue: #24234)
- Get short path name for native controllers #25344
- When stopping via systemd only kill the JVM, not its control group #25195
- remove remaining references to scripts directory #24771
- Handle parentheses in batch file path #24731 (issue: #24712)
- Parent/Child
- Percolator
- Plugin Lang Painless
-
- Painless: allow doubles to be casted to longs. #25936
- Plugin Repository Azure
- Plugin Repository GCS
- Plugin Repository HDFS
- Plugin Repository S3
- Plugins
-
- X-Pack plugin download fails on Windows desktop #24570
- Query DSL
- REST
- Recovery
- Reindex API
- Search
-
- Caching a MinDocQuery can lead to wrong results. #25909
- Fix random score generation when no seed is provided. #25908
- Merge FunctionScoreQuery and FiltersFunctionScoreQuery #25889 (issues: #15709, #23628)
-
Respect cluster alias in
_index
aggs and queries #25885 (issue: #25606) - First increment shard stats before notifying and potentially sending response #25818
- Remove assertion about deviation when casting to a float. #25806 (issue: #25330)
- Prevent skipping shards if a suggest builder is present #25739 (issue: #25658)
- Ensure remote cluster alias is preserved in inner hits aggs #25627 (issue: #25606)
- Do not search locally if remote index pattern resolves to no indices #25436 (issue: #25426)
- Adds check for negative search request size #25397 (issue: #22530)
- Make sure range queries are correctly profiled. #25108
- Fix RangeFieldMapper rangeQuery to properly handle relations #24808 (issue: #24744)
- Fix ExpandSearchPhase when response contains no hits #24688 (issue: #24672)
- Sequence IDs
-
- Fix pre-6.0 response to unknown replication actions #25744 (issue: #10708)
- Track local checkpoint on primary immediately #25434 (issues: #10708, #25355, #25415)
- Initialize max unsafe auto ID timestamp on shrink #25356 (issues: #10708, #25355)
- Use correct primary term for replicating NOOPs #25128
- Handle already closed while filling gaps #25021 (issue: #24925)
- Settings
- Snapshot/Restore
-
- Snapshot/Restore: Ensure that shard failure reasons are correctly stored in CS #25941 (issue: #25878)
- Output all empty snapshot info fields if in verbose mode #25455 (issue: #24477)
- Remove redundant and broken MD5 checksum from repository-s3 #25270 (issue: #25269)
- Consolidates the logic for cleaning up snapshots on master election #24894 (issue: #24605)
- Removes completed snapshot from cluster state on master change #24605 (issue: #24452)
- Stats
- Suggesters
Regressions
edit- Highlighting
- Search
Upgrades
editOn this page
Was this helpful?
Thank you for your feedback.