Engines and content sources
editEngines and content sources
editTool compatibility introduces the concepts of indices, engines, and content sources. Engines and content sources are abstractions introduced by Enterprise Search. Many Enterprise Search tools are designed to operate on these abstractions rather than Elasticsearch indices.
You can create engines from indices, and you can access the index that corresponds to an engine or content source. You can use a meta engine to search across multiple engines.
This document provides the following solutions for working with engines and content sources:
- Create an App Search engine from an Elasticsearch index
- Create an App Search engine from a Workplace Search content source
- Create a meta engine from multiple App Search engines
- Access an Elasticsearch index for an App Search engine
- Access the Elasticsearch index for a Workplace Search content source
Create an App Search engine from an Elasticsearch index
editSee the App Search documentation to create an App Search engine containing documents from an existing Elasticsearch index. Use the features of an engine and still have access to the index.
Example use case: You have ingested documents into an Elasticsearch index directly and now you want to build a search experience for it.
Example use case: You have a Workplace Search content source and want to use the App Search search API to query the content.
Learn about the specific features and field types supported by Elasticsearch index engines.
Create an App Search engine from a Workplace Search content source
editTo create an App Search engine from a Workplace Search content source:
-
Find the underlying Elasticsearch index for your Workplace Search content source.
For example,
.ent-search-engine-documents-custom-62fe4494b1720490773b75d7
. -
Use the Elasticsearch aliases API to create a
search-
prefixed alias for this Workplace Search content source index. For example:PUT /.ent-search-engine-documents-custom-62fe4494b1720490773b75d7/_alias/search-my-content
- In App Search, create an Elasticsearch index engine from this alias.
Example use case: You’ve ingested content using a Workplace Search connector. You want to build a search experience for this content with relevance tuning features in App Search.
Example use case: You want to build a search experience that searches content in Workplace Search content sources and App Search engines. Create an App Search engine and then combine multiple engines using a meta engine for a unified search experience.
Users should be aware of some differences between Workplace Search content sources and App Search engines:
- Workplace Search synonyms are applied globally across all content sources. App Search requires specifying synonyms for a specific engine or meta engine.
- App Search engines do not provide document-level permissions and access control.
- The App Search Search API does not provide automatic query refinement and query expansion phrases.
Create a meta engine from multiple App Search engines
editMeta engines allow you to combine multiple engines into one searchable engine. Users can leverage all App Search features for these engines. See the App Search documentation to learn all about meta engines.
Example use case: Your organization might have multiple content sources, each with different search settings and read/write permissions, but you also need a company-wide search experience with distinct curations or relevance tunings.
Example use case: You’ve ingested content into Elasticsearch indices using the Elastic web crawler. You’ve also ingested content into Workplace Search content sources using various Workplace Search connectors. You can create a unified search experience across all this content by creating engines and a meta engine.
See the App Search documentation to learn how to create meta engines in the Kibana UI.
Access an Elasticsearch index for an App Search engine
editYou can access the underlying Elasticsearch index for an App Search engine. To find the index follow these steps in the Kibana UI:
- Navigate to Enterprise Search > Content > Elasticsearch indices.
- Use the Show hidden indices toggle to ensure the index is listed under Available indices.
- Search for the engine name in the search box.
-
The engine’s underlying index is prefixed with
.ent-search-engine-documents-
. For example, if your engine is namedmy-engine
, the underlying Elasticsearch index is.ent-search-engine-documents-my-engine
.
Use the tools of your choice to manage and search the index’s documents.
Example use case: You created an App Search engine from an Elasticsearch index, but you still want to be able to access the index directly.
Example use case: You are an App Search user, but your requirements have outgrown the out-of-the-box capabilities of App Search. You want to leverage the full power of Elasticsearch to build more customized search experiences.
Access the Elasticsearch index for a Workplace Search content source
editYou can access the underlying Elasticsearch index for a Workplace Search content source. To find the index follow these steps in the Kibana UI:
- Navigate to Enterprise Search > Content > Elasticsearch indices.
- Use the Show hidden indices toggle to ensure the index is listed under Available indices.
- Search for the content source in the search box.
-
Each content source stores documents in a hidden Elasticsearch index prefixed with
.ent-search-engine-documents-
followed by the type of the source and the unique ID of the source. For example, a custom source might have an index named.ent-search-engine-documents-custom-62fe4494b1720490773b75d7
. A GitHub custom source might have an index named.ent-search-engine-documents-github-62fe460fb1720464b93b75e3
.
Use the tools of your choice to manage and search the index’s documents.
Example use case: You want to create an engine from a content source. Accessing the underlying Elasticsearch index is a required intermediate step.
Example use case: You want to use Elastic index tools to work with the documents in a content source.
See Search across indices, engines, and content sources for a detailed guide to searching across multiple Elastic data stores in a single UI.