Generation Phase

The prompt for the LLM is now ready, so what remains is to send it over and receive a response. To connect to LLMs, the application uses Langchain's streaming support, which nicely fits with the event streaming used in this application:

answer = ''
for chunk in get_llm().stream(qa_prompt):
    yield f'data: {chunk.content}\n\n'
    answer += chunk.content

The get_llm() function is defined in api/llm_integrations.py. Its purpose is to return the correct LLM integration from Langchain according to the configuration. Assuming you configured OpenAI, the LLM returned is going to be an instance of the ChatOpenAI class from Langchain.

Ready to build state of the art search experiences?

Sufficiently advanced search isn’t achieved with the efforts of one. Elasticsearch is powered by data scientists, ML ops, engineers, and many more who are just as passionate about search as your are. Let’s connect and work together to build the magical search experience that will get you the results you want.

Try it yourself