The Challenge
The Solution
By using the power of Elasticsearch to respond to 180 requests per second at 100 milliseconds per query
Case Study Highlights
Handle high traffic
- Serve 25 million unique users per day
- Process up to 180 queries per second
- Deliver results in 100 ms
Meet user expectations
- Eliminate downtime
- Guarantee all queries generate results
- Increase conversions by 8%
Serving content to 25 million users every day
Globo.com is the web portal of the Brazil-based Globo Organization, the largest media group in Latin America. The website is one of the largest web portals in Brazil, delivering news, entertainment and sports content to 25 million unique users per day.
Search is a key driver behind the site, impacting every user's experience. Prior to Elasticsearch, however, Globo.com used a well-known search server which caused some serious challenges.
"We decided to move forward and give Elasticsearch a try." recalls Luiz Guilherme Pais dos Santos, Software Engineer at Globo.com. "It is a big challenge to administer our 100GB database and Elasticsearch claimed to handle large amounts of data very well. It is critical that our users don't experience downtime."
"As Elasticsearch plays very well with multiples indexes compared to other platforms, it became the natural choice for us," Pais dos Santos continues. "Using alias management gives us the ability to deal with complex data flows without any downtime."
Elasticsearch makes searching across multiple indices painless. That allows us to aggregate different products to our main search with a single click.
Conducting 2 million daily searches via Elasticsearch
"Elasticsearch fits like a glove," Luiz says. "It is cloud-ready and really easy to scale. Furthermore, Elasticsearch's REST interface is quite powerful and the search response is very fast."
Today, all Globo.com content is delivered through Elasticsearch. The site's 25 million daily end-users conduct 20 queries per second – almost 2 million searches per day – through millions of videos and other content items.
Currently, Globo.com gets an average of 20 queries per second routed through Elasticsearch, however queries peak on Brazil's election day when Globo.com receives as many as 120 queries per second. To ensure support for its massive user base, Globo.com leverages Elasticsearch to handle up to 180 requests per second.
Elasticsearch also provides continuous real-time updating, eliminating the regularly disruptive downtime Globo.com experienced with the previous solution.
"It would not be possible for us to deliver our content without a search solution like Elasticsearch," says Luiz. "Elasticsearch is a robust implementation delivering great experience and instantaneous results to millions of users searching through millions of documents."
"Globo.com searches a 100 GB database via Elasticsearch with instant functionality, which was an issue on our old platform," he adds.
Creating an amazing user experience
"Without Elasticsearch, we couldn't deliver a good search experience to our users," Luis admits. "We had a very poor search experience but now Elasticsearch allows many types of queries, and the speed of the search makes our solution very good for the end-user."
"With Elasticsearch, our users get results instantly as they type," he continues. "We need response time below 200 ms. If the response time is longer than 200 ms it would be very bad for the end-user. It would be considered a slow search. For the initial search, we are below 100 ms with Elasticsearch."
Globo.com has configured Elasticsearch to automatically make a second – "fuzzy" – query if there are no results from the first query. The fuzzy query is designed to address typos by finding related content that is spelled similar. Today, Globo.com delivers almost no queries without results.
"The ability of Elasticsearch to provide fuzzy queries translates directly into increased conversion rates," Luiz adds. "This increased our conversion rate about 8% – and that is a lot for us. When equating with advertising dollars that is a lot of money."
Flexibility deriving from the schema-less architecture and the speed of Elasticsearch paves the way to create a richer search experience.
Empowering development teams with creative freedom
"Elasticsearch is easy to develop and scale," Luiz explains. "Elasticsearch is schema-free, which makes it really easy to deal with heterogeneous data. Our developers also take advantage of Elasticsearch's powerful and flexible APIs for easy data integration."
Multiple development teams at Globo.com modify the search functionality regularly. They create many new fields and types, which became very difficult to administer prior to Elasticsearch, because every day the schema had to be changed. Because Elasticsearch is schema-free, Luiz says it is much easier to administer because all the developers can create the fields and types they want with no problems.
"Since we are schema-free with Elasticsearch, we have a lot of flexibility," says Luiz. "In development, we have a news team, video team, photo team and more. We have 20 teams using Elasticsearch, and the types and fields that each team uses are all different. Every team can create their own index in Elasticsearch. This is very good for us."
"Our team that takes care of Elasticsearch does not have to care about all the solutions for everybody," he adds. "Without this flexibility, it would be very difficult. Our team would spend a lot of time guessing about and administering the many types and fields our developers want to use.
"Elasticsearch allows us to abstract the complexity of real-time, high-performance search and as a consequence we can freely use creativity to build applications that our users will really need," concludes Luiz.
