Personalizing content with signed search keys in Elastic App Search

By
Orhan Toy

Signed search keys in Elastic App Search give you more control over a user's search experience. You can tailor the experience to show results you know are more relevant to the specific user while also controlling what data the user can see and search over.

API keys in App Search

Elastic App Search has the concept of search keys and private keys:

  • A search key is prefixed with search- and can only be used to search over your engines.
  • A private key is prefixed with private- and can create, update, and delete documents if the write-flag is enabled. It can also perform searches and reads if the read-flag is enabled.

With both search keys and private keys, you can grant access to all engines or limit to specific ones.

Elastic App Search also has the concept of signed search keys, which, as the name implies, you can only use to search. A signed search key is a JSON Web Token and is signed with an API key, ideally a read-only private key, using the HMAC with SHA-256 (HS256) algorithm.

If you have Node.js on your back end, this is how simple it is to generate a signed search key using the Elastic App Search Node.js client: 

const AppSearchClient = require('@elastic/app-search-node');
const apiKey = 'private-xxxxxxxxxxxxxxxxxxxxxxxx';
const apiKeyName = 'key-for-signed-search';
const enforcedOptions = {
  result_fields: { title: { raw: {} } },
  filters: { world_heritage_site: 'true' }
};
const signedSearchKey = AppSearchClient.createSignedSearchKey(
  apiKey,
  apiKeyName,
  enforcedOptions
);
console.log(signedSearchKey);

Run the above code and you will generate a signed search key that looks something like this: 

eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJyZXN1bHRfZmllbGRzIjp7InRpdGxlIjp7InJhdyI6e319fSwiZmlsdGVycyI6eyJ3b3JsZF9oZXJpdGFnZV9zaXRlIjoidHJ1ZSJ9LCJhcGlfa2V5X25hbWUiOiJrZXktZm9yLXNpZ25lZC1zZWFyY2giLCJpYXQiOjE2MDc1MzczOTB9.b_stzPl5uUfr_c5eAFDKSY-Lm5oH3M7lDDFGsnofL4k

Clients can use this signed search key just like they would any App Search search key.

What makes the signed search key special and quite powerful is that you can enforce search options and perform searches against the App Search API directly from your front end. The alternative is to have each search request go through your own back end, so you can enforce certain options yourself.

We can actually inspect the generated signed search key with the neat debugger on jwt.io. We can see that the example search filter we used, world_heritage_site: true, is embedded directly in this search key: 

{ 
  "result_fields": { 
    "title": { 
      "raw": {} 
    } 
  }, 
  "filters": { 
    "world_heritage_site": "true" 
  }, 
  "api_key_name": "key-for-signed-search", 
  "iat": 1607537390 
}

Marketplace example

To explain and highlight specific use cases for signed search keys, we'll look at a marketplace where users can put items up for sale and purchase things from other sellers.

User-scoped search

Items in the marketplace can be in draft mode, meaning the user is not ready to make the item public just yet. So when the user performs a search, we can exclude the draft items by enforcing a filter: 

{ 
  "filters": { "is_draft": "false" } 
}

Apply this filter and regenerate a signed search key with it. We can inspect the generated signed search key with jwt.io again and see that the new filter is embedded within it. 

You might wonder why we're indexing draft items into the App Search engine and then filtering the draft items out. By doing so, we’re able to create a "My Items" page. On that page, we want to show the authenticated user’s

  • draft items
  • current items
  • sold items

For that occasion, we would generate a separate signed search key and enforce a filter on the user ID: 

{ 
  "filters": { "user_id": user.id } 
}

As long as the item belongs to the current user, we don't need to filter out the draft items.

Again, by embedding filters into the user’s signed search key, we’re restricting the user’s searches to always apply those filters.

User-relevant search

If we know that the currently authenticated user is interested in specific categories, we could boost items from those categories. You could, for instance, derive this data from the user's purchase history. In addition to the draft filter, we can now enforce boosts in the generated signed search key: 

{ 
  "filters": { "is_draft": "false" }, 
  "boosts": { 
    "categories": [ 
      { 
        "type": "value", 
        "value": user.categories_of_interest, 
        "operation": "multiply", 
        "factor": 3 
      } 
    ] 
  } 
}

Anonymous search

We should still enforce the draft filter for unauthenticated users, but we would also like to exclude specific fields from the results. The reasoning is that we include both first_name and full_name in the App Search document but only authenticated users should see full_name. The same applies to the email field.

The enforced options would, therefore, look like this: 

{ 
  "filters": { is_draft: "false" }, 
  "result_fields": { 
    "title": { raw: { } }, 
    "description": { raw: { } }, 
    "categories": { raw: { } }, 
    "image_url": { raw: { } }, 
    "published_at": { raw: { } }, 
    "country_code": { raw: { } }, 
    "country_name": { raw: { } }, 
    "first_name": { raw: { } } 
  } 
}

Demo

You can check out a CodeSandbox demo with the marketplace example.

Please note that the demo has no back end to generate the signed search keys, but rather a Mirage JS mock server.

Next steps

We’ve seen how we can embed filters and boosts into signed search keys, and even exclude specific fields from the results. When a client searches using a generated signed search key, we can effectively create a personalized search experience.

If you’d like, check out Making it personal: Tailoring content with signed search keys, a presentation from our recent ElasticON Global virtual event (along with dozens of other presentations across all Elastic solutions). And, as always, you can try out Elastic App Search, and all of the Elastic Stack, with a free 14-day trial of Elastic Cloud (or a free download).