- Elasticsearch Guide: other versions:
- Elasticsearch basics
- Quick starts
- Set up Elasticsearch
- Run Elasticsearch locally
- Installing Elasticsearch
- Configuring Elasticsearch
- Important Elasticsearch configuration
- Secure settings
- Auditing settings
- Circuit breaker settings
- Cluster-level shard allocation and routing settings
- Miscellaneous cluster settings
- Cross-cluster replication settings
- Discovery and cluster formation settings
- Field data cache settings
- Health Diagnostic settings
- Index lifecycle management settings
- Data stream lifecycle settings
- Index management settings
- Index recovery settings
- Indexing buffer settings
- License settings
- Local gateway settings
- Logging
- Machine learning settings
- Inference settings
- Monitoring settings
- Nodes
- Networking
- Node query cache settings
- Search settings
- Security settings
- Shard allocation, relocation, and recovery
- Shard request cache settings
- Snapshot and restore settings
- Transforms settings
- Thread pools
- Watcher settings
- Advanced configuration
- Important system configuration
- Bootstrap Checks
- Heap size check
- File descriptor check
- Memory lock check
- Maximum number of threads check
- Max file size check
- Maximum size virtual memory check
- Maximum map count check
- Client JVM check
- Use serial collector check
- System call filter check
- OnError and OnOutOfMemoryError checks
- Early-access check
- All permission check
- Discovery configuration check
- Bootstrap Checks for X-Pack
- Starting Elasticsearch
- Stopping Elasticsearch
- Discovery and cluster formation
- Add and remove nodes in your cluster
- Full-cluster restart and rolling restart
- Remote clusters
- Plugins
- Search your data
- Re-ranking
- Index modules
- Index templates
- Aliases
- Mapping
- Dynamic mapping
- Explicit mapping
- Runtime fields
- Field data types
- Aggregate metric
- Alias
- Arrays
- Binary
- Boolean
- Completion
- Date
- Date nanoseconds
- Dense vector
- Flattened
- Geopoint
- Geoshape
- Histogram
- IP
- Join
- Keyword
- Nested
- Numeric
- Object
- Pass-through object
- Percolator
- Point
- Range
- Rank feature
- Rank features
- Search-as-you-type
- Semantic text
- Shape
- Sparse vector
- Text
- Token count
- Unsigned long
- Version
- Metadata fields
- Mapping parameters
analyzer
coerce
copy_to
doc_values
dynamic
eager_global_ordinals
enabled
format
ignore_above
index.mapping.ignore_above
ignore_malformed
index
index_options
index_phrases
index_prefixes
meta
fields
normalizer
norms
null_value
position_increment_gap
properties
search_analyzer
similarity
store
subobjects
term_vector
- Mapping limit settings
- Removal of mapping types
- Text analysis
- Overview
- Concepts
- Configure text analysis
- Built-in analyzer reference
- Tokenizer reference
- Token filter reference
- Apostrophe
- ASCII folding
- CJK bigram
- CJK width
- Classic
- Common grams
- Conditional
- Decimal digit
- Delimited payload
- Dictionary decompounder
- Edge n-gram
- Elision
- Fingerprint
- Flatten graph
- Hunspell
- Hyphenation decompounder
- Keep types
- Keep words
- Keyword marker
- Keyword repeat
- KStem
- Length
- Limit token count
- Lowercase
- MinHash
- Multiplexer
- N-gram
- Normalization
- Pattern capture
- Pattern replace
- Phonetic
- Porter stem
- Predicate script
- Remove duplicates
- Reverse
- Shingle
- Snowball
- Stemmer
- Stemmer override
- Stop
- Synonym
- Synonym graph
- Trim
- Truncate
- Unique
- Uppercase
- Word delimiter
- Word delimiter graph
- Character filters reference
- Normalizers
- Ingest pipelines
- Example: Parse logs
- Enrich your data
- Processor reference
- Append
- Attachment
- Bytes
- Circle
- Community ID
- Convert
- CSV
- Date
- Date index name
- Dissect
- Dot expander
- Drop
- Enrich
- Fail
- Fingerprint
- Foreach
- Geo-grid
- GeoIP
- Grok
- Gsub
- HTML strip
- Inference
- IP Location
- Join
- JSON
- KV
- Lowercase
- Network direction
- Pipeline
- Redact
- Registered domain
- Remove
- Rename
- Reroute
- Script
- Set
- Set security user
- Sort
- Split
- Terminate
- Trim
- Uppercase
- URL decode
- URI parts
- User agent
- Ingest pipelines in Search
- Connectors
- Data streams
- Data management
- ILM: Manage the index lifecycle
- Tutorial: Customize built-in policies
- Tutorial: Automate rollover
- Index management in Kibana
- Overview
- Concepts
- Index lifecycle actions
- Configure a lifecycle policy
- Migrate index allocation filters to node roles
- Troubleshooting index lifecycle management errors
- Start and stop index lifecycle management
- Manage existing indices
- Skip rollover
- Restore a managed data stream or index
- Data tiers
- Roll up or transform your data
- Query DSL
- EQL
- ES|QL
- SQL
- Overview
- Getting Started with SQL
- Conventions and Terminology
- Security
- SQL REST API
- SQL Translate API
- SQL CLI
- SQL JDBC
- SQL ODBC
- SQL Client Applications
- SQL Language
- Functions and Operators
- Comparison Operators
- Logical Operators
- Math Operators
- Cast Operators
- LIKE and RLIKE Operators
- Aggregate Functions
- Grouping Functions
- Date/Time and Interval Functions and Operators
- Full-Text Search Functions
- Mathematical Functions
- String Functions
- Type Conversion Functions
- Geo Functions
- Conditional Functions And Expressions
- System Functions
- Reserved keywords
- SQL Limitations
- Scripting
- Aggregations
- Bucket aggregations
- Adjacency matrix
- Auto-interval date histogram
- Categorize text
- Children
- Composite
- Date histogram
- Date range
- Diversified sampler
- Filter
- Filters
- Frequent item sets
- Geo-distance
- Geohash grid
- Geohex grid
- Geotile grid
- Global
- Histogram
- IP prefix
- IP range
- Missing
- Multi Terms
- Nested
- Parent
- Random sampler
- Range
- Rare terms
- Reverse nested
- Sampler
- Significant terms
- Significant text
- Terms
- Time series
- Variable width histogram
- Subtleties of bucketing range fields
- Metrics aggregations
- Pipeline aggregations
- Average bucket
- Bucket script
- Bucket count K-S test
- Bucket correlation
- Bucket selector
- Bucket sort
- Change point
- Cumulative cardinality
- Cumulative sum
- Derivative
- Extended stats bucket
- Inference bucket
- Max bucket
- Min bucket
- Moving function
- Moving percentiles
- Normalize
- Percentiles bucket
- Serial differencing
- Stats bucket
- Sum bucket
- Bucket aggregations
- Geospatial analysis
- Watcher
- Monitor a cluster
- Secure the Elastic Stack
- Elasticsearch security principles
- Start the Elastic Stack with security enabled automatically
- Manually configure security
- Updating node security certificates
- User authentication
- Built-in users
- Service accounts
- Internal users
- Token-based authentication services
- User profiles
- Realms
- Realm chains
- Security domains
- Active Directory user authentication
- File-based user authentication
- LDAP user authentication
- Native user authentication
- OpenID Connect authentication
- PKI user authentication
- SAML authentication
- Kerberos authentication
- JWT authentication
- Integrating with other authentication systems
- Enabling anonymous access
- Looking up users without authentication
- Controlling the user cache
- Configuring SAML single-sign-on on the Elastic Stack
- Configuring single sign-on to the Elastic Stack using OpenID Connect
- User authorization
- Built-in roles
- Defining roles
- Role restriction
- Security privileges
- Document level security
- Field level security
- Granting privileges for data streams and aliases
- Mapping users and groups to roles
- Setting up field and document level security
- Submitting requests on behalf of other users
- Configuring authorization delegation
- Customizing roles and authorization
- Enable audit logging
- Restricting connections with IP filtering
- Securing clients and integrations
- Operator privileges
- Troubleshooting
- Some settings are not returned via the nodes settings API
- Authorization exceptions
- Users command fails due to extra arguments
- Users are frequently locked out of Active Directory
- Certificate verification fails for curl on Mac
- SSLHandshakeException causes connections to fail
- Common SSL/TLS exceptions
- Common Kerberos exceptions
- Common SAML issues
- Internal Server Error in Kibana
- Setup-passwords command fails due to connection failure
- Failures due to relocation of the configuration files
- Limitations
- Set up a cluster for high availability
- Optimizations
- Autoscaling
- Snapshot and restore
- REST APIs
- API conventions
- Common options
- REST API compatibility
- Autoscaling APIs
- Behavioral Analytics APIs
- Compact and aligned text (CAT) APIs
- cat aliases
- cat allocation
- cat anomaly detectors
- cat component templates
- cat count
- cat data frame analytics
- cat datafeeds
- cat fielddata
- cat health
- cat indices
- cat master
- cat nodeattrs
- cat nodes
- cat pending tasks
- cat plugins
- cat recovery
- cat repositories
- cat segments
- cat shards
- cat snapshots
- cat task management
- cat templates
- cat thread pool
- cat trained model
- cat transforms
- Cluster APIs
- Cluster allocation explain
- Cluster get settings
- Cluster health
- Health
- Cluster reroute
- Cluster state
- Cluster stats
- Cluster update settings
- Nodes feature usage
- Nodes hot threads
- Nodes info
- Prevalidate node removal
- Nodes reload secure settings
- Nodes stats
- Cluster Info
- Pending cluster tasks
- Remote cluster info
- Task management
- Voting configuration exclusions
- Create or update desired nodes
- Get desired nodes
- Delete desired nodes
- Get desired balance
- Reset desired balance
- Cross-cluster replication APIs
- Connector APIs
- Create connector
- Delete connector
- Get connector
- List connectors
- Update connector API key id
- Update connector configuration
- Update connector index name
- Update connector features
- Update connector filtering
- Update connector name and description
- Update connector pipeline
- Update connector scheduling
- Update connector service type
- Create connector sync job
- Cancel connector sync job
- Delete connector sync job
- Get connector sync job
- List connector sync jobs
- Check in a connector
- Update connector error
- Update connector last sync stats
- Update connector status
- Check in connector sync job
- Claim connector sync job
- Set connector sync job error
- Set connector sync job stats
- Data stream APIs
- Document APIs
- Enrich APIs
- EQL APIs
- ES|QL APIs
- Features APIs
- Fleet APIs
- Graph explore API
- Index APIs
- Alias exists
- Aliases
- Analyze
- Analyze index disk usage
- Clear cache
- Clone index
- Close index
- Create index
- Create or update alias
- Create or update component template
- Create or update index template
- Create or update index template (legacy)
- Delete component template
- Delete dangling index
- Delete alias
- Delete index
- Delete index template
- Delete index template (legacy)
- Exists
- Field usage stats
- Flush
- Force merge
- Get alias
- Get component template
- Get field mapping
- Get index
- Get index settings
- Get index template
- Get index template (legacy)
- Get mapping
- Import dangling index
- Index recovery
- Index segments
- Index shard stores
- Index stats
- Index template exists (legacy)
- List dangling indices
- Open index
- Refresh
- Resolve index
- Resolve cluster
- Rollover
- Shrink index
- Simulate index
- Simulate template
- Split index
- Unfreeze index
- Update index settings
- Update mapping
- Index lifecycle management APIs
- Create or update lifecycle policy
- Get policy
- Delete policy
- Move to step
- Remove policy
- Retry policy
- Get index lifecycle management status
- Explain lifecycle
- Start index lifecycle management
- Stop index lifecycle management
- Migrate indices, ILM policies, and legacy, composable and component templates to data tiers routing
- Inference APIs
- Delete inference API
- Get inference API
- Perform inference API
- Create inference API
- Stream inference API
- Update inference API
- AlibabaCloud AI Search inference service
- Amazon Bedrock inference service
- Anthropic inference service
- Azure AI studio inference service
- Azure OpenAI inference service
- Cohere inference service
- Elasticsearch inference service
- ELSER inference service
- Google AI Studio inference service
- Google Vertex AI inference service
- HuggingFace inference service
- Mistral inference service
- OpenAI inference service
- Watsonx inference service
- Info API
- Ingest APIs
- Licensing APIs
- Logstash APIs
- Machine learning APIs
- Machine learning anomaly detection APIs
- Add events to calendar
- Add jobs to calendar
- Close jobs
- Create jobs
- Create calendars
- Create datafeeds
- Create filters
- Delete calendars
- Delete datafeeds
- Delete events from calendar
- Delete filters
- Delete forecasts
- Delete jobs
- Delete jobs from calendar
- Delete model snapshots
- Delete expired data
- Estimate model memory
- Flush jobs
- Forecast jobs
- Get buckets
- Get calendars
- Get categories
- Get datafeeds
- Get datafeed statistics
- Get influencers
- Get jobs
- Get job statistics
- Get model snapshots
- Get model snapshot upgrade statistics
- Get overall buckets
- Get scheduled events
- Get filters
- Get records
- Open jobs
- Post data to jobs
- Preview datafeeds
- Reset jobs
- Revert model snapshots
- Start datafeeds
- Stop datafeeds
- Update datafeeds
- Update filters
- Update jobs
- Update model snapshots
- Upgrade model snapshots
- Machine learning data frame analytics APIs
- Create data frame analytics jobs
- Delete data frame analytics jobs
- Evaluate data frame analytics
- Explain data frame analytics
- Get data frame analytics jobs
- Get data frame analytics jobs stats
- Preview data frame analytics
- Start data frame analytics jobs
- Stop data frame analytics jobs
- Update data frame analytics jobs
- Machine learning trained model APIs
- Clear trained model deployment cache
- Create or update trained model aliases
- Create part of a trained model
- Create trained models
- Create trained model vocabulary
- Delete trained model aliases
- Delete trained models
- Get trained models
- Get trained models stats
- Infer trained model
- Start trained model deployment
- Stop trained model deployment
- Update trained model deployment
- Migration APIs
- Node lifecycle APIs
- Query rules APIs
- Reload search analyzers API
- Repositories metering APIs
- Rollup APIs
- Root API
- Script APIs
- Search APIs
- Search Application APIs
- Searchable snapshots APIs
- Security APIs
- Authenticate
- Change passwords
- Clear cache
- Clear roles cache
- Clear privileges cache
- Clear API key cache
- Clear service account token caches
- Create API keys
- Create or update application privileges
- Create or update role mappings
- Create or update roles
- Bulk create or update roles API
- Bulk delete roles API
- Create or update users
- Create service account tokens
- Delegate PKI authentication
- Delete application privileges
- Delete role mappings
- Delete roles
- Delete service account token
- Delete users
- Disable users
- Enable users
- Enroll Kibana
- Enroll node
- Get API key information
- Get application privileges
- Get builtin privileges
- Get role mappings
- Get roles
- Query Role
- Get service accounts
- Get service account credentials
- Get Security settings
- Get token
- Get user privileges
- Get users
- Grant API keys
- Has privileges
- Invalidate API key
- Invalidate token
- OpenID Connect prepare authentication
- OpenID Connect authenticate
- OpenID Connect logout
- Query API key information
- Query User
- Update API key
- Update Security settings
- Bulk update API keys
- SAML prepare authentication
- SAML authenticate
- SAML logout
- SAML invalidate
- SAML complete logout
- SAML service provider metadata
- SSL certificate
- Activate user profile
- Disable user profile
- Enable user profile
- Get user profiles
- Suggest user profile
- Update user profile data
- Has privileges user profile
- Create Cross-Cluster API key
- Update Cross-Cluster API key
- Snapshot and restore APIs
- Snapshot lifecycle management APIs
- SQL APIs
- Synonyms APIs
- Text structure APIs
- Transform APIs
- Usage API
- Watcher APIs
- Definitions
- Command line tools
- elasticsearch-certgen
- elasticsearch-certutil
- elasticsearch-create-enrollment-token
- elasticsearch-croneval
- elasticsearch-keystore
- elasticsearch-node
- elasticsearch-reconfigure-node
- elasticsearch-reset-password
- elasticsearch-saml-metadata
- elasticsearch-service-tokens
- elasticsearch-setup-passwords
- elasticsearch-shard
- elasticsearch-syskeygen
- elasticsearch-users
- Troubleshooting
- Fix common cluster issues
- Diagnose unassigned shards
- Add a missing tier to the system
- Allow Elasticsearch to allocate the data in the system
- Allow Elasticsearch to allocate the index
- Indices mix index allocation filters with data tiers node roles to move through data tiers
- Not enough nodes to allocate all shard replicas
- Total number of shards for an index on a single node exceeded
- Total number of shards per node has been reached
- Troubleshooting corruption
- Fix data nodes out of disk
- Fix master nodes out of disk
- Fix other role nodes out of disk
- Start index lifecycle management
- Start Snapshot Lifecycle Management
- Restore from snapshot
- Troubleshooting broken repositories
- Addressing repeated snapshot policy failures
- Troubleshooting an unstable cluster
- Troubleshooting discovery
- Troubleshooting monitoring
- Troubleshooting transforms
- Troubleshooting Watcher
- Troubleshooting searches
- Troubleshooting shards capacity health issues
- Troubleshooting an unbalanced cluster
- Capture diagnostics
- Upgrade Elasticsearch
- Migration guide
- Release notes
- Dependencies and versions
Basic full-text search and filtering in Elasticsearch
editBasic full-text search and filtering in Elasticsearch
editThis is a hands-on introduction to the basics of full-text search with Elasticsearch, also known as lexical search, using the _search
API and Query DSL.
You’ll also learn how to filter data, to narrow down search results based on exact criteria.
In this scenario, we’re implementing a search function for a cooking blog. The blog contains recipes with various attributes including textual content, categorical data, and numerical ratings.
The goal is to create search queries that enable users to:
- Find recipes based on ingredients they want to use or avoid
- Discover dishes suitable for their dietary needs
- Find highly-rated recipes in specific categories
- Find recent recipes from their favorite authors
To achieve these goals we’ll use different Elasticsearch queries to perform full-text search, apply filters, and combine multiple search criteria.
Requirements
editYou’ll need a running Elasticsearch cluster, together with Kibana to use the Dev Tools API Console. Run the following command in your terminal to set up a single-node local cluster in Docker:
curl -fsSL https://elastic.co/start-local | sh
Step 1: Create an index
editCreate the cooking_blog
index to get started:
resp = client.indices.create( index="cooking_blog", ) print(resp)
const response = await client.indices.create({ index: "cooking_blog", }); console.log(response);
PUT /cooking_blog
Now define the mappings for the index:
resp = client.indices.put_mapping( index="cooking_blog", properties={ "title": { "type": "text", "analyzer": "standard", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, "description": { "type": "text", "fields": { "keyword": { "type": "keyword" } } }, "author": { "type": "text", "fields": { "keyword": { "type": "keyword" } } }, "date": { "type": "date", "format": "yyyy-MM-dd" }, "category": { "type": "text", "fields": { "keyword": { "type": "keyword" } } }, "tags": { "type": "text", "fields": { "keyword": { "type": "keyword" } } }, "rating": { "type": "float" } }, ) print(resp)
const response = await client.indices.putMapping({ index: "cooking_blog", properties: { title: { type: "text", analyzer: "standard", fields: { keyword: { type: "keyword", ignore_above: 256, }, }, }, description: { type: "text", fields: { keyword: { type: "keyword", }, }, }, author: { type: "text", fields: { keyword: { type: "keyword", }, }, }, date: { type: "date", format: "yyyy-MM-dd", }, category: { type: "text", fields: { keyword: { type: "keyword", }, }, }, tags: { type: "text", fields: { keyword: { type: "keyword", }, }, }, rating: { type: "float", }, }, }); console.log(response);
PUT /cooking_blog/_mapping { "properties": { "title": { "type": "text", "analyzer": "standard", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, "description": { "type": "text", "fields": { "keyword": { "type": "keyword" } } }, "author": { "type": "text", "fields": { "keyword": { "type": "keyword" } } }, "date": { "type": "date", "format": "yyyy-MM-dd" }, "category": { "type": "text", "fields": { "keyword": { "type": "keyword" } } }, "tags": { "type": "text", "fields": { "keyword": { "type": "keyword" } } }, "rating": { "type": "float" } } }
The |
|
Multi-fields are used here to index |
|
The |
Full-text search is powered by text analysis. Text analysis normalizes and standardizes text data so it can be efficiently stored in an inverted index and searched in near real-time. Analysis happens at both index and search time. This tutorial won’t cover analysis in detail, but it’s important to understand how text is processed to create effective search queries.
Step 2: Add sample blog posts to your index
editNow you’ll need to index some example blog posts using the Bulk API.
Note that text
fields are analyzed and multi-fields are generated at index time.
resp = client.bulk( index="cooking_blog", refresh="wait_for", operations=[ { "index": { "_id": "1" } }, { "title": "Perfect Pancakes: A Fluffy Breakfast Delight", "description": "Learn the secrets to making the fluffiest pancakes, so amazing you won't believe your tastebuds. This recipe uses buttermilk and a special folding technique to create light, airy pancakes that are perfect for lazy Sunday mornings.", "author": "Maria Rodriguez", "date": "2023-05-01", "category": "Breakfast", "tags": [ "pancakes", "breakfast", "easy recipes" ], "rating": 4.8 }, { "index": { "_id": "2" } }, { "title": "Spicy Thai Green Curry: A Vegetarian Adventure", "description": "Dive into the flavors of Thailand with this vibrant green curry. Packed with vegetables and aromatic herbs, this dish is both healthy and satisfying. Don't worry about the heat - you can easily adjust the spice level to your liking.", "author": "Liam Chen", "date": "2023-05-05", "category": "Main Course", "tags": [ "thai", "vegetarian", "curry", "spicy" ], "rating": 4.6 }, { "index": { "_id": "3" } }, { "title": "Classic Beef Stroganoff: A Creamy Comfort Food", "description": "Indulge in this rich and creamy beef stroganoff. Tender strips of beef in a savory mushroom sauce, served over a bed of egg noodles. It's the ultimate comfort food for chilly evenings.", "author": "Emma Watson", "date": "2023-05-10", "category": "Main Course", "tags": [ "beef", "pasta", "comfort food" ], "rating": 4.7 }, { "index": { "_id": "4" } }, { "title": "Vegan Chocolate Avocado Mousse", "description": "Discover the magic of avocado in this rich, vegan chocolate mousse. Creamy, indulgent, and secretly healthy, it's the perfect guilt-free dessert for chocolate lovers.", "author": "Alex Green", "date": "2023-05-15", "category": "Dessert", "tags": [ "vegan", "chocolate", "avocado", "healthy dessert" ], "rating": 4.5 }, { "index": { "_id": "5" } }, { "title": "Crispy Oven-Fried Chicken", "description": "Get that perfect crunch without the deep fryer! This oven-fried chicken recipe delivers crispy, juicy results every time. A healthier take on the classic comfort food.", "author": "Maria Rodriguez", "date": "2023-05-20", "category": "Main Course", "tags": [ "chicken", "oven-fried", "healthy" ], "rating": 4.9 } ], ) print(resp)
const response = await client.bulk({ index: "cooking_blog", refresh: "wait_for", operations: [ { index: { _id: "1", }, }, { title: "Perfect Pancakes: A Fluffy Breakfast Delight", description: "Learn the secrets to making the fluffiest pancakes, so amazing you won't believe your tastebuds. This recipe uses buttermilk and a special folding technique to create light, airy pancakes that are perfect for lazy Sunday mornings.", author: "Maria Rodriguez", date: "2023-05-01", category: "Breakfast", tags: ["pancakes", "breakfast", "easy recipes"], rating: 4.8, }, { index: { _id: "2", }, }, { title: "Spicy Thai Green Curry: A Vegetarian Adventure", description: "Dive into the flavors of Thailand with this vibrant green curry. Packed with vegetables and aromatic herbs, this dish is both healthy and satisfying. Don't worry about the heat - you can easily adjust the spice level to your liking.", author: "Liam Chen", date: "2023-05-05", category: "Main Course", tags: ["thai", "vegetarian", "curry", "spicy"], rating: 4.6, }, { index: { _id: "3", }, }, { title: "Classic Beef Stroganoff: A Creamy Comfort Food", description: "Indulge in this rich and creamy beef stroganoff. Tender strips of beef in a savory mushroom sauce, served over a bed of egg noodles. It's the ultimate comfort food for chilly evenings.", author: "Emma Watson", date: "2023-05-10", category: "Main Course", tags: ["beef", "pasta", "comfort food"], rating: 4.7, }, { index: { _id: "4", }, }, { title: "Vegan Chocolate Avocado Mousse", description: "Discover the magic of avocado in this rich, vegan chocolate mousse. Creamy, indulgent, and secretly healthy, it's the perfect guilt-free dessert for chocolate lovers.", author: "Alex Green", date: "2023-05-15", category: "Dessert", tags: ["vegan", "chocolate", "avocado", "healthy dessert"], rating: 4.5, }, { index: { _id: "5", }, }, { title: "Crispy Oven-Fried Chicken", description: "Get that perfect crunch without the deep fryer! This oven-fried chicken recipe delivers crispy, juicy results every time. A healthier take on the classic comfort food.", author: "Maria Rodriguez", date: "2023-05-20", category: "Main Course", tags: ["chicken", "oven-fried", "healthy"], rating: 4.9, }, ], }); console.log(response);
POST /cooking_blog/_bulk?refresh=wait_for {"index":{"_id":"1"}} {"title":"Perfect Pancakes: A Fluffy Breakfast Delight","description":"Learn the secrets to making the fluffiest pancakes, so amazing you won't believe your tastebuds. This recipe uses buttermilk and a special folding technique to create light, airy pancakes that are perfect for lazy Sunday mornings.","author":"Maria Rodriguez","date":"2023-05-01","category":"Breakfast","tags":["pancakes","breakfast","easy recipes"],"rating":4.8} {"index":{"_id":"2"}} {"title":"Spicy Thai Green Curry: A Vegetarian Adventure","description":"Dive into the flavors of Thailand with this vibrant green curry. Packed with vegetables and aromatic herbs, this dish is both healthy and satisfying. Don't worry about the heat - you can easily adjust the spice level to your liking.","author":"Liam Chen","date":"2023-05-05","category":"Main Course","tags":["thai","vegetarian","curry","spicy"],"rating":4.6} {"index":{"_id":"3"}} {"title":"Classic Beef Stroganoff: A Creamy Comfort Food","description":"Indulge in this rich and creamy beef stroganoff. Tender strips of beef in a savory mushroom sauce, served over a bed of egg noodles. It's the ultimate comfort food for chilly evenings.","author":"Emma Watson","date":"2023-05-10","category":"Main Course","tags":["beef","pasta","comfort food"],"rating":4.7} {"index":{"_id":"4"}} {"title":"Vegan Chocolate Avocado Mousse","description":"Discover the magic of avocado in this rich, vegan chocolate mousse. Creamy, indulgent, and secretly healthy, it's the perfect guilt-free dessert for chocolate lovers.","author":"Alex Green","date":"2023-05-15","category":"Dessert","tags":["vegan","chocolate","avocado","healthy dessert"],"rating":4.5} {"index":{"_id":"5"}} {"title":"Crispy Oven-Fried Chicken","description":"Get that perfect crunch without the deep fryer! This oven-fried chicken recipe delivers crispy, juicy results every time. A healthier take on the classic comfort food.","author":"Maria Rodriguez","date":"2023-05-20","category":"Main Course","tags":["chicken","oven-fried","healthy"],"rating":4.9}
Step 3: Perform basic full-text searches
editFull-text search involves executing text-based queries across one or more document fields. These queries calculate a relevance score for each matching document, based on how closely the document’s content aligns with the search terms. Elasticsearch offers various query types, each with its own method for matching text and relevance scoring.
match
query
editThe match
query is the standard query for full-text, or "lexical", search.
The query text will be analyzed according to the analyzer configuration specified on each field (or at query time).
First, search the description
field for "fluffy pancakes":
resp = client.search( index="cooking_blog", query={ "match": { "description": { "query": "fluffy pancakes" } } }, ) print(resp)
const response = await client.search({ index: "cooking_blog", query: { match: { description: { query: "fluffy pancakes", }, }, }, }); console.log(response);
GET /cooking_blog/_search { "query": { "match": { "description": { "query": "fluffy pancakes" } } } }
By default, the |
At search time, Elasticsearch defaults to the analyzer defined in the field mapping. In this example, we’re using the standard
analyzer. Using a different analyzer at search time is an advanced use case.
Example response
{ "took": 0, "timed_out": false, "_shards": { "total": 1, "successful": 1, "skipped": 0, "failed": 0 }, "hits": { "total": { "value": 1, "relation": "eq" }, "max_score": 1.8378843, "hits": [ { "_index": "cooking_blog", "_id": "1", "_score": 1.8378843, "_source": { "title": "Perfect Pancakes: A Fluffy Breakfast Delight", "description": "Learn the secrets to making the fluffiest pancakes, so amazing you won't believe your tastebuds. This recipe uses buttermilk and a special folding technique to create light, airy pancakes that are perfect for lazy Sunday mornings.", "author": "Maria Rodriguez", "date": "2023-05-01", "category": "Breakfast", "tags": [ "pancakes", "breakfast", "easy recipes" ], "rating": 4.8 } } ] } }
The |
|
|
|
|
|
The title contains both "Fluffy" and "Pancakes", matching our search terms exactly. |
|
The description includes "fluffiest" and "pancakes", further contributing to the document’s relevance due to the analysis process. |
Require all terms in a match query
editSpecify the and
operator to require both terms in the description
field.
This stricter search returns zero hits on our sample data, as no document contains both "fluffy" and "pancakes" in the description.
resp = client.search( index="cooking_blog", query={ "match": { "description": { "query": "fluffy pancakes", "operator": "and" } } }, ) print(resp)
const response = await client.search({ index: "cooking_blog", query: { match: { description: { query: "fluffy pancakes", operator: "and", }, }, }, }); console.log(response);
GET /cooking_blog/_search { "query": { "match": { "description": { "query": "fluffy pancakes", "operator": "and" } } } }
Example response
Specify a minimum number of terms to match
editUse the minimum_should_match
parameter to specify the minimum number of terms a document should have to be included in the search results.
Search the title field to match at least 2 of the 3 terms: "fluffy", "pancakes", or "breakfast". This is useful for improving relevance while allowing some flexibility.
resp = client.search( index="cooking_blog", query={ "match": { "title": { "query": "fluffy pancakes breakfast", "minimum_should_match": 2 } } }, ) print(resp)
const response = await client.search({ index: "cooking_blog", query: { match: { title: { query: "fluffy pancakes breakfast", minimum_should_match: 2, }, }, }, }); console.log(response);
GET /cooking_blog/_search { "query": { "match": { "title": { "query": "fluffy pancakes breakfast", "minimum_should_match": 2 } } } }
Step 4: Search across multiple fields at once
editWhen users enter a search query, they often don’t know (or care) whether their search terms appear in a specific field.
A multi_match
query allows searching across multiple fields simultaneously.
Let’s start with a basic multi_match
query:
resp = client.search( index="cooking_blog", query={ "multi_match": { "query": "vegetarian curry", "fields": [ "title", "description", "tags" ] } }, ) print(resp)
const response = await client.search({ index: "cooking_blog", query: { multi_match: { query: "vegetarian curry", fields: ["title", "description", "tags"], }, }, }); console.log(response);
GET /cooking_blog/_search { "query": { "multi_match": { "query": "vegetarian curry", "fields": ["title", "description", "tags"] } } }
This query searches for "vegetarian curry" across the title, description, and tags fields. Each field is treated with equal importance.
However, in many cases, matches in certain fields (like the title) might be more relevant than others. We can adjust the importance of each field using field boosting:
resp = client.search( index="cooking_blog", query={ "multi_match": { "query": "vegetarian curry", "fields": [ "title^3", "description^2", "tags" ] } }, ) print(resp)
const response = await client.search({ index: "cooking_blog", query: { multi_match: { query: "vegetarian curry", fields: ["title^3", "description^2", "tags"], }, }, }); console.log(response);
GET /cooking_blog/_search { "query": { "multi_match": { "query": "vegetarian curry", "fields": ["title^3", "description^2", "tags"] } } }
The
|
Learn more about fields and per-field boosting in the multi_match
query reference.
Example response
{ "took": 0, "timed_out": false, "_shards": { "total": 1, "successful": 1, "skipped": 0, "failed": 0 }, "hits": { "total": { "value": 1, "relation": "eq" }, "max_score": 7.546015, "hits": [ { "_index": "cooking_blog", "_id": "2", "_score": 7.546015, "_source": { "title": "Spicy Thai Green Curry: A Vegetarian Adventure", "description": "Dive into the flavors of Thailand with this vibrant green curry. Packed with vegetables and aromatic herbs, this dish is both healthy and satisfying. Don't worry about the heat - you can easily adjust the spice level to your liking.", "author": "Liam Chen", "date": "2023-05-05", "category": "Main Course", "tags": [ "thai", "vegetarian", "curry", "spicy" ], "rating": 4.6 } } ] } }
The title contains "Vegetarian" and "Curry", which matches our search terms. The title field has the highest boost (^3), contributing significantly to this document’s relevance score. |
|
The description contains "curry" and related terms like "vegetables", further increasing the document’s relevance. |
|
The tags include both "vegetarian" and "curry", providing an exact match for our search terms, albeit with no boost. |
This result demonstrates how the multi_match
query with field boosts helps users find relevant recipes across multiple fields.
Even though the exact phrase "vegetarian curry" doesn’t appear in any single field, the combination of matches across fields produces a highly relevant result.
The multi_match
query is often recommended over a single match
query for most text search use cases, as it provides more flexibility and better matches user expectations.
Step 5: Filter and find exact matches
editFiltering allows you to narrow down your search results based on exact criteria. Unlike full-text searches, filters are binary (yes/no) and do not affect the relevance score. Filters execute faster than queries because excluded results don’t need to be scored.
This bool
query will return only blog posts in the "Breakfast" category.
resp = client.search( index="cooking_blog", query={ "bool": { "filter": [ { "term": { "category.keyword": "Breakfast" } } ] } }, ) print(resp)
const response = await client.search({ index: "cooking_blog", query: { bool: { filter: [ { term: { "category.keyword": "Breakfast", }, }, ], }, }, }); console.log(response);
GET /cooking_blog/_search { "query": { "bool": { "filter": [ { "term": { "category.keyword": "Breakfast" } } ] } } }
Note the use of |
The .keyword
suffix accesses the unanalyzed version of a field, enabling exact, case-sensitive matching. This works in two scenarios:
-
When using dynamic mapping for text fields. Elasticsearch automatically creates a
.keyword
sub-field. -
When text fields are explicitly mapped with a
.keyword
sub-field. For example, we explicitly mapped thecategory
field in Step 1 of this tutorial.
Search for posts within a date range
editOften users want to find content published within a specific time frame.
A range
query finds documents that fall within numeric or date ranges.
resp = client.search( index="cooking_blog", query={ "range": { "date": { "gte": "2023-05-01", "lte": "2023-05-31" } } }, ) print(resp)
const response = await client.search({ index: "cooking_blog", query: { range: { date: { gte: "2023-05-01", lte: "2023-05-31", }, }, }, }); console.log(response);
GET /cooking_blog/_search { "query": { "range": { "date": { "gte": "2023-05-01", "lte": "2023-05-31" } } } }
Find exact matches
editSometimes users want to search for exact terms to eliminate ambiguity in their search results.
A term
query searches for an exact term in a field without analyzing it.
Exact, case-sensitive matches on specific terms are often referred to as "keyword" searches.
Here you’ll search for the author "Maria Rodriguez" in the author.keyword
field.
resp = client.search( index="cooking_blog", query={ "term": { "author.keyword": "Maria Rodriguez" } }, ) print(resp)
const response = await client.search({ index: "cooking_blog", query: { term: { "author.keyword": "Maria Rodriguez", }, }, }); console.log(response);
The |
Avoid using the term
query for text
fields because they are transformed by the analysis process.
Step 6: Combine multiple search criteria
editA bool
query allows you to combine multiple query clauses to create sophisticated searches.
In this tutorial scenario it’s useful for when users have complex requirements for finding recipes.
Let’s create a query that addresses the following user needs:
- Must be a vegetarian recipe
- Should contain "curry" or "spicy" in the title or description
- Should be a main course
- Must not be a dessert
- Must have a rating of at least 4.5
- Should prefer recipes published in the last month
resp = client.search( index="cooking_blog", query={ "bool": { "must": [ { "term": { "tags": "vegetarian" } }, { "range": { "rating": { "gte": 4.5 } } } ], "should": [ { "term": { "category": "Main Course" } }, { "multi_match": { "query": "curry spicy", "fields": [ "title^2", "description" ] } }, { "range": { "date": { "gte": "now-1M/d" } } } ], "must_not": [ { "term": { "category.keyword": "Dessert" } } ] } }, ) print(resp)
const response = await client.search({ index: "cooking_blog", query: { bool: { must: [ { term: { tags: "vegetarian", }, }, { range: { rating: { gte: 4.5, }, }, }, ], should: [ { term: { category: "Main Course", }, }, { multi_match: { query: "curry spicy", fields: ["title^2", "description"], }, }, { range: { date: { gte: "now-1M/d", }, }, }, ], must_not: [ { term: { "category.keyword": "Dessert", }, }, ], }, }, }); console.log(response);
GET /cooking_blog/_search { "query": { "bool": { "must": [ { "term": { "tags": "vegetarian" } }, { "range": { "rating": { "gte": 4.5 } } } ], "should": [ { "term": { "category": "Main Course" } }, { "multi_match": { "query": "curry spicy", "fields": [ "title^2", "description" ] } }, { "range": { "date": { "gte": "now-1M/d" } } } ], "must_not": [ { "term": { "category.keyword": "Dessert" } } ] } } }
The |
Example response
{ "took": 1, "timed_out": false, "_shards": { "total": 1, "successful": 1, "skipped": 0, "failed": 0 }, "hits": { "total": { "value": 1, "relation": "eq" }, "max_score": 7.444513, "hits": [ { "_index": "cooking_blog", "_id": "2", "_score": 7.444513, "_source": { "title": "Spicy Thai Green Curry: A Vegetarian Adventure", "description": "Dive into the flavors of Thailand with this vibrant green curry. Packed with vegetables and aromatic herbs, this dish is both healthy and satisfying. Don't worry about the heat - you can easily adjust the spice level to your liking.", "author": "Liam Chen", "date": "2023-05-05", "category": "Main Course", "tags": [ "thai", "vegetarian", "curry", "spicy" ], "rating": 4.6 } } ] } }
The title contains "Spicy" and "Curry", matching our should condition. With the default best_fields behavior, this field contributes most to the relevance score. |
|
While the description also contains matching terms, only the best matching field’s score is used by default. |
|
The recipe was published within the last month, satisfying our recency preference. |
|
The "Main Course" category satisfies another |
|
The "vegetarian" tag satisfies a |
|
The rating of 4.6 meets our minimum rating requirement of 4.5. |
Learn more
editThis tutorial introduced the basics of full-text search and filtering in Elasticsearch. Building a real-world search experience requires understanding many more advanced concepts and techniques. Here are some resources once you’re ready to dive deeper:
- Elasticsearch basics — Search and analyze data: Understand all your options for searching and analyzing data in Elasticsearch.
- Text analysis: Understand how text is processed for full-text search.
-
Search your data: Learn about more advanced search techniques using the
_search
API, including semantic search.
On this page
- Requirements
- Step 1: Create an index
- Step 2: Add sample blog posts to your index
- Step 3: Perform basic full-text searches
match
query- Require all terms in a match query
- Specify a minimum number of terms to match
- Step 4: Search across multiple fields at once
- Step 5: Filter and find exact matches
- Search for posts within a date range
- Find exact matches
- Step 6: Combine multiple search criteria
- Learn more
ElasticON events are back!
Learn about the Elastic Search AI Platform from the experts at our live events.
Register now