Creating meta engines in App Search to scale your search experiences

By
David James

We introduced meta engines for Elastic App Search on Elastic Cloud and self-managed versions in the 7.6 release and have been thrilled to see the response to the new feature. If you missed the 7.6 announcement, check out the enhancements we’ve made to the entire Enterprise Search solution.

But back to meta engines. 

Meta engines provide the ability to search across multiple existing or new engines. Think of adding a new search box to a page that then goes off and searches the documents in the sub-engines of your choosing. This allows large companies — or really any size company — to unify and scale their search function while still allowing search/website admins complete control over the behavior of each individual sub-engine.

Creating a new meta engine

This blog dives into why meta engines have become so useful as your search needs grow and uncovers scenarios where meta engines make sense — as well as times when a regular engine would be a better fit. 

TL;DR 

Rather watch the movie than read the book? Check out our meta engines webinar for step-by-step instructions and best practices for setting up meta engines in your organization. 

You say you want scale?

Raise your hand if this sounds familiar: Your organization started out with a single search engine for your primary website with its own rules, boosts, and weights. 

And then along comes customer support requesting a new, separate engine for FAQs, videos, tech pubs, and how-to articles. Impressed by the speed and relevance of the first two engines’ performance, the marketing team quickly follows with a request to set up another search box on their blog homepage. 

Waiting in line: the Documentation team with thousands of tutorials and articles waiting to be indexed and searched.

So how do you create a unified, universal search experience for users who don’t know whether their question can best be answered by a blog, support issue, or how-to doc? Hunting for the answer across multiple, siloed engines creates a fragmented customer experience — one that creates friction between users and the business. Elastic App Search meta engines eliminates this drag and smooths the user experience. 

Now, with meta engines, companies can unify and scale their search function by setting up a new “uber engine” to search across all of these sub-engines in marketing, customer support, documentation departments (and any others, too). 

This new meta engine is “documentless,” meaning it relies on the documents of the sub-engines for content; however, this new meta engine does contain its own independent rules for tuning, synonyms, relevance, and curations. And the new meta engine supersedes existing rules set up in sub-engines so admins can create new customizations and rules for that particular meta engine.

One of these (sub) engines is not like the other

There’s no argument that growing the search functionality across your site drives a better overall customer experience. But on the surface this can appear to complicate the day-to-day job of search/site admins. Therein lies the inherent beauty of meta-engines.

Specifying role mapping for engines

Each individual sub-engine remains independent of other sub-engines  — working autonomously of each other with their own tuning, relevance rules, boosting, and curations.  

Search admins retain complete autonomy of their engine. So if, for example, the documentation team wanted to curate search results to promote the latest software release, this new rule would have no impact on the results used for the meta-engine. Meta engine rules exist separately from the sub-engines’ rules and customizations.

Managing users and their access

And as your business grows, the need for user management and access rights becomes a bigger day-to-day concern. With the App Search 7.6 release, role-based access controls are granular down to the sub-engine level as well as meta-engine level. So users can protect who has access to which engine at all times.

Important reminders for using meta engines

Hopefully by now you’re envisioning  all the flexibility, autonomy, and value meta engines could bring to your org’s search experiences.  

But before you create your first meta engine, here’s some helpful information:

  • Fields — Currently, each engine (sub or meta) is limited to 64 fields.
  • Ingestion — Ensure your schemas across engines match.  
  • Read-only files — Documents indexed in the meta engine are “read-only” files. Edits to JSON data must be made at the sub-engine index.

Let’s get started

App Search meta engines are available on Elastic Cloud and self-managed versions. This feature is available for Platinum and Enterprise subscription levels.

New Elastic Cloud customers can start a free 14-day trial of App Search on Elastic Cloud. Or download the self-managed version of App Search. Existing Elastic Cloud customers can access App Search directly from the Elastic Cloud console today.  

There is no additional fee for meta engines. With Elastic’s new, simplified pricing model, you only pay for the resources you consume, without worrying about artificial constraints around the number of users, documents, engines, queries, or operations made. It’s a whole new approach to pricing search that’s transparent and fair to our customers. 

Additional resources

Interested in learning more about Elastic App Search? Get a walkthrough of all things meta engine in the webinar Creating meta engines to scale your search experiences. Or check out our blog post to learn more about Elastic App Search availability directly in Elastic Cloud. See our docs to get more technical detail about meta engine setup and operation.