Building multilingual RAG with Elastic and Mistral

Mixtral 8x22B is the most performant open model, and one of its most powerful features is fluency in many languages; including English, Spanish, French, Italian, and German.

Imagine a multinational company with support tickets and solutions in different languages and wants to take advantage of that knowledge across divisions. Currently, knowledge is limited to the language the agent speaks. Let's fix that!

In this article, I’m going to show you how to test Mixtral’s language capabilities, by creating a multilingual RAG system.

You can follow the notebook to reproduce this article's example here

Steps

1. Creating embeddings endpoint
2. Creating mappings
3. Indexing data
4. Asking questions

Creating embeddings endpoint

Our support tickets for this example will come in English, Spanish, and German. The Mistral embeddings model is not multilingual, but we can generate multilingual embeddings using the e5 model, so we can index text on different languages and manage it as a single source, giving us a much richer context.

To create e5 multilingual embeddings you can use Kibana:

Or the _inference API:

PUT _inference/text_embedding/multilingual-embeddings
 {
    "service": "elasticsearch",
    "service_settings": {
        "model_id": ".multilingual-e5-small",
        "num_allocations": 1 ,
        "num_threads": 1
    }
}

Creating Mappings

For the mappings we will use semantic_text mapping type, which is one of my favorite features. It handles the process of chunking the data, generating embeddings, and querying embeddings for you!

PUT multilingual-mistral
{
  "mappings": {
    "properties": {
      "super_body": {
        "type": "semantic_text",
        "inference_id": "multilingual-embeddings"
      }
    }
  }
}

We call the text field super_body because with a single mapping type it will handle chunks and embeddings.

Indexing data

We will index a couple of support tickets with problems and solutions in two languages, and then ask a question about problems within many documents in a third.

The following documents will be added to the index:

This is how a document will look like inside Elasticsearch:

{
    "took": 9,
    "timed_out": false,
    "_shards": {
        "total": 1,
        "successful": 1,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": {
            "value": 2,
            "relation": "eq"
        },
        "max_score": 0.9155389,
        "hits": [
            {
                "_index": "multilingual-mistral",
                "_id": "1",
                "_score": 0.9155389,
                "_source": {
                    "super_body": {
                        "text": "\n        _Support Ticket #EN1234_\n        **Subject**: Calendar sync not working with Google Calendar\n\n        **Description**:\n        I'm having trouble syncing my project deadlines with Google Calendar. Whenever I try to sync, I get an error message saying \"Unable to connect to external calendar service.\"\n\n        **Resolution**:\n        The issue was resolved by following these steps:\n        1. Go to Settings > Integrations\n        2. Disconnect the Google Calendar integration\n        3. Clear browser cache and cookies\n        4. Reconnect the Google Calendar integration\n        5. Authorize the app again in Google's security settings\n\n        The sync should now work correctly. If problems persist, ensure that third-party cookies are enabled in your browser settings.\n    ",
                        "inference": {
                            "inference_id": "multilingual-embeddings",
                            "model_settings": {
                                "task_type": "text_embedding",
                                "dimensions": 384,
                                "similarity": "cosine",
                                "element_type": "float"
                            },
                            "chunks": [
                                {
                                    "text": "passage: \n        _Support Ticket #EN1234_\n        **Subject**: Calendar sync not working with Google Calendar\n\n        **Description**:\n        I'm having trouble syncing my project deadlines with Google Calendar. Whenever I try to sync, I get an error message saying \"Unable to connect to external calendar service.\"\n\n        **Resolution**:\n        The issue was resolved by following these steps:\n        1. Go to Settings > Integrations\n        2. Disconnect the Google Calendar integration\n        3. Clear browser cache and cookies\n        4. Reconnect the Google Calendar integration\n        5. Authorize the app again in Google's security settings\n\n        The sync should now work correctly. If problems persist, ensure that third-party cookies are enabled in your browser settings.",
                                    "embeddings": [
                                        0.0059651174,
                                        0.0016363655,
                                        -0.064753555,
                                        0.0093298275,
                                        0.05689768,
                                        -0.049640983,
                                        0.02504726,
                                        0.0048340675,
                                        0.08093895,
                                        ...
                                    ]
                                }
                            ]
                        }
                    }
                }
            }
        ]
    }
}

Asking questions

Now, we are going to ask a question in Spanish:

Hola, estoy teniendo problemas para ocupar su aplicación, estoy teniendo problemas para sincronizar mi calendario, y encima al intentar subir un archivo me da error.

The expectation is retrieving documents #1 and #2, then sending them to the LLM as additional context, and finally, getting an answer in Spanish.

Retrieving documents

To retrieve the relevant documents, we can use this nice and short query that will run a search on the embeddings, and return the support tickets most relevant to the question.

GET multilingual-mistral/_search
{
   "size": 2,
   "_source": {
    "excludes": ["*embeddings", "*chunks"]
   },
  "query": {
    "semantic": {
      "field": "super_body",
      "query": "Hola, estoy teniendo problemas para ocupar su aplicación, estoy teniendo problemas para sincronizar mi calendario, y encima al intentar subir un archivo me da error."
    }
  }
}

Notes about the parameters set: size: 2 Because we know we want the top 2 documents. excludes For clarity in the response. Documents are short so each one will be one chunk long.

Answering the question

Now we can call the Mistral completion API using the Python library to answer the question.

from mistralai.client import MistralClient
from mistralai.models.chat_completion import ChatMessage

api_key = os.environ["MISTRAL_API_KEY"]
model = "open-mixtral-8x22b"

client = MistralClient(api_key=api_key)

system_message = "You are a helpful multilingual agent that helps users with their problems. You have access to a knowledge base of different languages and you must answer in the same language the question was asked."

user_message = """
## Question:

Hola, estoy teniendo problemas para ocupar su aplicación, estoy teniendo problemas para sincronizar mi calendario, y encima al intentar subir un archivo me da error. 

## Related knowledge:

Support Ticket #EN1234 Subject: Calendar sync not working with Google Calendar...
(the rest of the content of the document)

\n

Support-Ticket #DE5678 Betreff: Datei-Upload funktioniert nicht... 
(the rest of the content of the document)

ANSWER:

"""

messages = [
    ChatMessage(role="system", content=system_message),
    ChatMessage(role="user", content=user_message)
]

chat_response = client.chat(
    model=model,
    messages=messages,
)

print(chat_response.choices[0].message.content)

The answer is in perfect Spanish and on point!

Conclusion

Mixtral 8x22B is a powerful model that enables us to leverage data sources in different languages, being able to answer, understand, and translate in many languages. This ability– together with multilingual embeddings– allows you to have multilingual support both in the data retrieval and the answer generation stages, removing language barriers entirely.

If you are interested on reproducing the examples of this article, you can find the Python Notebook with the requests here