Accessing machine learning models in Elastic

Elastic® lets you apply the machine learning (ML) that’s appropriate for your use case and level of ML expertise. You have multiple options:

1. Leverage the models that come built-in. Aside from models targeting specific security threats and types of system issues in our observability and security solution, you can use our proprietary Elastic Learned Sparse Encoder model out of the box, as well as a language identification — useful if you’re working with non-English text data.
2. Access third-party PyTorch models from anywhere including the HuggingFace model hub.
3. Load a model you trained yourself — primarily NLP transformers at this point.

Using built-in models gets you value out of the box, without requiring any ML expertise from you, yet you have the flexibility to try out different models and determine what performs best on your data

We designed our model management to be scalable across multiple nodes in a cluster, while also ensuring good inference performance for both high throughput and low latency workloads. That’s in part by empowering ingest pipelines to run inference and by using dedicated nodes for the computationally demanding model inference — during the ingestion phase, as well as data analysis and search.

Read on to learn more about the Eland library that lets you load models into Elastic and how that plays out for the various types of machine learning you might use within Elasticsearch® — from the latest transformer and natural language processing (NLP) models to boosted tree models for regression.

Further below we'll walk through each of the arguments of eland_import_hub_model. And you can issue the same command from a Docker container.

Once uploaded, Kibana's ML Model Management user interface lets you manage the models on an Elasticsearch cluster, including increasing allocations for additional throughput, and stop/resume models while (re)configuring your system.

1. Specify the Elastic Cloud identifier. Alternatively, use --url.
2. Provide authentication details to access your cluster. You can look up available authentication methods. 
3. Specify the identifier for the model in the Hugging Face model hub.
4. Specify the type of NLP task. Supported values are fill_mask, ner, text_classification, text_embedding, and zero_shot_classification.

Once you've loaded the model, next you need to deploy it. You accomplish this on the Model Management screen of the Machine Learning tab in Kibana. Then you'd typically test the model to ensure it's working properly.

Now you're ready to use the deployed model for inference. For example to extract named entities, you call the _infer endpoint on the loaded NER model:

The model identifies two entities: the person "Josh" and the location "Berlin."

For additional steps, like using this model in an inference pipeline and tuning the deployment, read the blog that describes this example.

Want to see how to apply semantic search — for example, how to create embeddings for text and then apply vector search to find related documents? This blog lays that out step-by-step, including validating model performance.

Don't know which type of task for which model? This table should help you get started.

Elastic also supports comparing how similar two pieces of text are to each other as text_similarity task-type — this is useful for ranking document text when comparing it to another provided text input, and it's sometimes referred to as cross-encoding. 

Check these resources for more details

• Support for PyTorch transformers, including design considerations for Eland
• Steps for loading transformers into Elastic and using them in inference
• Blog describing how to query your proprietary data using ChatGPT
• Adapt a pre trained transformer to a text classification task, and load the custom model into Elastic
• Built-in language identification that lets you identify non-English text before passing into models that support only English

Elastic, Elasticsearch and associated marks are trademarks, logos or registered trademarks of Elasticsearch N.V. in the United States and other countries. All other company and product names are trademarks, logos or registered trademarks of their respective owners.

The release and timing of any features or functionality described in this post remain at Elastic's sole discretion. Any features or functionality not currently available may not be delivered on time or at all.