Yatego: Bringing the local shopping mall experience online

Yatego is a German-language online shopping mall, hosting 10,000 merchants selling 5 million products and 16 million product variants. The site serves 3.4 million unique users per month from Germany, Austria and Switzerland.

"Search is very important to our business," explains Thorsten Rinne, Engineering Team Lead for Yatego. "We have 3 million search pages in the Google index, which are quite high rank in Google Germany. So if a user searches in Google for a product sold by one of our merchants, they get a result in Google, a URL for Yatego, containing only the search term. So for us everything is a search."

Prior to deploying Elasticsearch, Yatego used MySQL text search, but they experienced slow query response, slow indexing and unreliable search results. The company looked for a more powerful search solution – and found Elasticsearch.

Today, Yatego uses Elasticsearch as the main site search on the front-end for customers to find products across thousands of online merchants. Search query speed is now significantly faster, which is a vital advantage in the online shopping industry.

"Before we switched to Elasticsearch, our old search was a bit outdated," Rinne explains. "If users searched for the term 'black' in German, for example, it took about 45 seconds. Now, we are at 15 milliseconds for a search for 'black'. In this case, Elasticsearch is 3,000 times faster than our previous solution."

"This is critical because the customer is not going to wait 45 seconds," he continues. "They are going to go to another shopping site. Customers usually wait less than 3 seconds for the search. Now we are much, much faster than 3 seconds."

Rinne explains that fast search speed provides Yatego with a competitive advantage: "Most online shops in Germany are PHP-based with MySQL on the back end, so they are a bit slow. Since we have a very fast shop, due to Elasticsearch, it is a unique selling point."

The speed and the advanced search capabilities of Elasticsearch also improve the search results for Yatego users, another critical advantage for online shopping.

"Fulltext search in our previous solution was suboptimal," Rinne confirms. "The whole infrastructure was complicated and we had to transform a lot of data in strange ways. Elasticsearch improved the results of our search queries in a tremendous way."

In addition, Elasticsearch's performance has improved Yatego's search referrer traffic from major search engines like Google Germany. "Since Yatego has 3 million pages indexed by search engines, our site gets a lot of traffic from web crawlers," said Rinne. "Our previous backend could not keep up with the load. With Elasticsearch, our performance is so much faster now, so search engines can crawl more of our pages much more quickly, meaning customers searching for products on Google, etc. find better, fresher results on Yatego."

"Our merchants also make a lot of updates to our data," Rinne explains. "We have 10,000 merchants, and they are doing 5 to 9 million product and stock updates per day." Timely updating of product data is very important so that customers can have access to any new products immediately. Real-time updating of stock is also essential, so the system reflects up-to-date inventory. Otherwise customers could be ordering products that are out-of-stock, causing serious customer service issues.

"Sometimes it would take us four hours to update data on the front-end," Rinne notes. "Now, with Elasticsearch it takes two minutes. Elasticsearch is very fast with indexing new data, which is very important for us."

"Our plan is to scale up in 2014, and Elasticsearch helps us to scale," says Rinne. "With the old infrastructure, we had 12 servers. With Elasticsearch – which is many times faster than the old system – we only need three servers."

"One of the main benefits of Elasticsearch is it reduces the number of servers we have to use to provide our products to customers. And with the Elasticsearch infrastructure we can always add more servers if we get more merchants and more users."

Yatego's product management and development departments also utilize Elasticsearch internally for quality assurance, to improve and ensure the quality of the product data.

Yatego specializes in hosting small- to medium-sized merchants, and many of them provide data to Yatego manually, so the data quality can vary widely depending on the merchant. While some merchants provide very good quality data, others input data that is not always accurate. For example, delivery times can be incorrect, or there may be typo errors in the product descriptions. Elasticsearch Facets enable Yatego to easily search the data, check the consistency, and identify and correct any typos or other mistakes in the merchant data.

"Before Elasticsearch, we had to check everything manually because the search was too slow to get all of the information back," Rinnes recalls. "We did not have enough time to find all of the inaccurate data because the backend engines were too slow, and the queries were too complicated. With Elasticsearch, it is much much easier to find such bad data. Our merchants do as many as 9 million updates per day on Elasticsearch and we can check the data integrity automatically."