This documentation contains work-in-progress information for future Elastic Stack and Cloud releases. Use the version selector to view supported release docs. It also contains some Elastic Cloud serverless information. Check out our serverless docs for more details.
« Retrievers Learning To Rank »
Elastic Docs ›Elasticsearch Guide [master] ›Search your data ›Retrievers and reranking

Semantic reranking

edit

Semantic reranking

edit

This functionality is in technical preview and may be changed or removed in a future release. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features.

This overview focuses more on the high-level concepts and use cases for semantic reranking. For full implementation details on how to set up and use semantic reranking in Elasticsearch, see the reference documentation in the Search API docs.

Rerankers improve the relevance of results from earlier-stage retrieval mechanisms. Semantic rerankers use machine learning models to reorder search results based on their semantic similarity to a query.

First-stage retrievers and rankers must be very fast and efficient because they process either the entire corpus, or all matching documents. In a multi-stage pipeline, you can progressively use more computationally intensive ranking functions and techniques, as they will operate on smaller result sets at each step. This helps avoid query latency degradation and keeps costs manageable.

Semantic reranking requires relatively large and complex machine learning models and operates in real-time in response to queries. This technique makes sense on a small top-k result set, as one the of the final steps in a pipeline. This is a powerful technique for improving search relevance that works equally well with keyword, semantic, or hybrid retrieval algorithms.

The next sections provide more details on the benefits, use cases, and model types used for semantic reranking. The final sections include a practical, high-level overview of how to implement semantic reranking in Elasticsearch and links to the full reference documentation.

Use cases

edit

Semantic reranking enables a variety of use cases:

  • Lexical (BM25) retrieval results reranking

    • Out-of-the-box semantic search by adding a simple API call to any lexical/BM25 retrieval pipeline.
    • Adds semantic search capabilities on top of existing indices without reindexing, perfect for quick improvements.
    • Ideal for environments with complex existing indices.

  • Semantic retrieval results reranking

    • Improves results from semantic retrievers using ELSER sparse vector embeddings or dense vector embeddings by using more powerful models.
    • Adds a refinement layer on top of hybrid retrieval with reciprocal rank fusion (RRF).

  • General applications

    • Supports automatic and transparent chunking, eliminating the need for pre-chunking at index time.
    • Provides explicit control over document relevance in retrieval-augmented generation (RAG) uses cases or other scenarios involving language model (LLM) inputs.

Now that we’ve outlined the value of semantic reranking, we’ll explore the specific models that power this process and how they differ.

Cross-encoder and bi-encoder models

edit

At a high level, two model types are used for semantic reranking: cross-encoders and bi-encoders.

In this version, Elasticsearch only supports cross-encoders for semantic reranking.

  • A cross-encoder model can be thought of as a more powerful, all-in-one solution, because it generates query-aware document representations. It takes the query and document texts as a single, concatenated input.

  • A bi-encoder model takes as input either document or query text. Documents and query embeddings are computed separately, so they aren’t aware of each other.

    • To compute a ranking score, an external operation is required. This typically involves computing dot-product or cosine similarity between the query and document embeddings.

In brief, cross-encoders provide high accuracy but are more resource-intensive. Bi-encoders are faster and more cost-effective but less precise.

In future versions, Elasticsearch will also support bi-encoders. If you’re interested in a more detailed analysis of the practical differences between cross-encoders and bi-encoders, untoggle the next section.

Comparisons between cross-encoder and bi-encoder

The following is a non-exhaustive list of considerations when choosing between cross-encoders and bi-encoders for semantic reranking:

  • Because a cross-encoder model simultaneously processes both query and document texts, it can better infer their relevance, making it more effective as a reranker than a bi-encoder.
  • Cross-encoder models are generally larger and more computationally intensive, resulting in higher latencies and increased computational costs.
  • There are significantly fewer open-source cross-encoders, while bi-encoders offer a wide variety of sizes, languages, and other trade-offs.
  • The effectiveness of cross-encoders can also improve the relevance of semantic retrievers. For example, their ability to take word order into account can improve on dense or sparse embedding retrieval.
  • When trained in tandem with specific retrievers (like lexical/BM25), cross-encoders can “correct” typical errors made by those retrievers.
  • Cross-encoders output scores that are consistent across queries. This enables you to maintain high relevance in result sets, by setting a minimum score threshold for all queries. For example, this is important when using results in a RAG workflow or if you’re otherwise feeding results to LLMs. Note that similarity scores from bi-encoders/embedding similarities are query-dependent, meaning you cannot set universal cut-offs.
  • Bi-encoders rerank using embeddings. You can improve your reranking latency by creating embeddings at ingest-time. These embeddings can be stored for reranking without being indexed for retrieval, reducing your memory footprint.

Semantic reranking in Elasticsearch

edit

In Elasticsearch, semantic rerankers are implemented using the Elasticsearch Inference API and a retriever.

To use semantic reranking in Elasticsearch, you need to:

  1. Choose a reranking model. Currently you can:

  2. Create a rerank task using the Elasticsearch Inference API. The Inference API creates an inference endpoint and configures your chosen machine learning model to perform the reranking task.
  3. Define a text_similarity_reranker retriever in your search request. The retriever syntax makes it simple to configure both the retrieval and reranking of search results in a single API call.
Example search request with semantic reranker

The following example shows a search request that uses a semantic reranker to reorder the top-k documents based on their semantic similarity to the query.

resp = client.search(
    retriever={
        "text_similarity_reranker": {
            "retriever": {
                "standard": {
                    "query": {
                        "match": {
                            "text": "How often does the moon hide the sun?"
                        }
                    }
                }
            },
            "field": "text",
            "inference_id": "my-cohere-rerank-model",
            "inference_text": "How often does the moon hide the sun?",
            "rank_window_size": 100,
            "min_score": 0.5
        }
    },
)
print(resp)
const response = await client.search({
  retriever: {
    text_similarity_reranker: {
      retriever: {
        standard: {
          query: {
            match: {
              text: "How often does the moon hide the sun?",
            },
          },
        },
      },
      field: "text",
      inference_id: "my-cohere-rerank-model",
      inference_text: "How often does the moon hide the sun?",
      rank_window_size: 100,
      min_score: 0.5,
    },
  },
});
console.log(response);
POST _search
{
  "retriever": {
    "text_similarity_reranker": {
      "retriever": {
        "standard": {
          "query": {
            "match": {
              "text": "How often does the moon hide the sun?"
            }
          }
        }
      },
      "field": "text",
      "inference_id": "my-cohere-rerank-model",
      "inference_text": "How often does the moon hide the sun?",
      "rank_window_size": 100,
      "min_score": 0.5
    }
  }
}

Learn more

edit
« Retrievers Learning To Rank »