Implementing image search: vector search via image processing in Elasticsearch

Case study: Finding your puppy with image search

Have you ever been in a situation where you found a lost puppy on the street and didn’t know if it had an owner? Using vector search through image processing in Elasticsearch, this task can be as simple as reading a comic strip.

Imagine this scene: On a tumultuous afternoon, Luigi, a small and lively puppy, found himself wandering alone through busy streets after accidentally slipping out of his leash during a walk around Elastic. His desperate owner was searching for him at every corner, calling his name with a voice full of hope and anxiety. Meanwhile, somewhere in the city, an attentive person noticed the puppy with a lost expression and decided to help. Quickly, they took a photo of Luigi and, using the vector image search technology of the company they worked for, began a search in the database hoping to find some clue about the owner of the little runaway.

If you want to follow and execute the code while reading, access the file Python code running on a Jupyter Notebook (Google Collab).

The architecture

We'll solve this problem using a Jupyter Notebook. First, we download the images of the puppies to be registered, and then we install the necessary packages.

*Note: To implement this sample, we will need to create an index in Elasticsearch before populating our vector database with our image data.

Begin by deploying Elasticsearch (we have a 14-day free trial for you).
During the process, remember to store the credentials (username, password) to be used in our Python code.
For simplicity, we will use Python code running on a Jupyter Notebook (Google Colab).

Download the code zip file and install the necessary packages

Let's create 4 classes to assist us in this task, and they are:

Util class: responsible for handling preliminary tasks and Elasticsearch index maintenance.
Dog class: responsible for storing the attributes of our little dogs.
DogRepository class: responsible for data persistence tasks.
DogService class: it will be our service layer.

Util class

The Util class provides utility methods for managing the Elasticsearch index, such as creating and deleting the index.

Methods:

create_index(): Creates a new index in Elasticsearch.
delete_index(): Deletes an existing index from Elasticsearch.

Dog class

The Dog class represents a dog and its attributes, such as ID, image path, breed, owner name, and image embeddings.

Attributes

dog_id: The dog's ID.
image_path: The path to the dog's image.
breed: The dog's breed.
owner_name: The dog's owner's name.
image_embedding: The dog's image embedding.

Methods

__init__(): Initializes a new Dog object.
generate_embedding(): Generates the dog's image embedding.
to_dict(): Converts the Dog object to a dictionary.

DogRepository Class

The DogRepository class provides methods for persisting and retrieving dog data from Elasticsearch.

Methods

insert(): Inserts a new dog into Elasticsearch.
bulk_insert(): Inserts multiple dogs into Elasticsearch in bulk.
search_by_image(): Searches for similar dogs by image.

DogService Class

The DogService class provides business logic for managing dog data, such as inserting and searching for dogs.

Methods

insert_dog(): Inserts a new dog into Elasticsearch.
search_dogs_by_image(): Searches for similar dogs by image.

The classes presented above provide a solid foundation for building a dog data management system. The Util class provides utility methods for managing the Elasticsearch index. The Dog class represents the attributes of a dog. The DogRepository class offers methods for persisting and retrieving dog data from Elasticsearch. The DogService class provides the business logic for efficient dog data management.

The main code

We'll basically have 2 main flows or phases in our code:

Register the Dogs with basic information and image.
Perform a search using a new image to find the Dog in the vector database.

Phase 01: Registering the Puppy

To store the information about Luigi and the other company's little dogs, we'll use the Dog class.

For this purpose, let's code the sequence:

Start registering the puppies

Output

Registering Luigi

Registering all the others puppies

Visualizing the new dogs

Output

Phase 02: Finding the lost dog

Now that we have all the little dogs registered, let's perform a search. Our developer took this picture of the lost puppy.

Output

Let's see if we find the owner of this cute little puppy?

Get the results

Let's see what we found...

Output

Voilà!! We found it!!!

But who will be the owner and their name?

Output

Luigi
Jack Russel/Rat Terrier
Ully

Happy end

We found Luigi !!! Let's notify Ully.

Output

In no time, Ully and Luigi were reunited. The little puppy wagged his tail with pure delight, and Ully hugged him close, promising to never let him out of her sight again. They had been through a whirlwind of emotions, but they were together now, and that was all that mattered. And so, with hearts full of love and joy, Ully and Luigi lived happily ever after.

Conclusion

In this blog post, we have explored how to use vector search to find a lost puppy using Elasticsearch. We have demonstrated how to generate image embeddings for dogs, index them in Elasticsearch, and then search for similar dogs using a query image. This technique can be used to find lost pets, as well as to identify other objects of interest in images.

Vector search is a powerful tool that can be used for a variety of applications. It is particularly well-suited for tasks that require searching for similar objects based on their appearance, such as image retrieval and object recognition.

We hope that this blog post has been informative and that you will find the techniques we have discussed to be useful in your own projects.

Resources

Try out vector search for yourself using this self-paced hands-on learning for Search AI. You can start a free cloud trial or try Elastic on your local machine now.

Report an issue