Searching and Alerting for application logs with Elasticsearch at Naver (Japanese subtitles)

Jae Ik Lee

Naver

Seung Jin Lee

Naver

Naver Corporation is the biggest Internet content service provider in South Korea. The company is famous for developing and distributing such mobile and server applications like Naver search portal, Line and Band messengers. Our developers often need to check application logs to resolve customer issues. However finding the root reason of distributed or mobile application's fault is not easy. Logs are stored on hundreds of servers and/or only on customers' devices. NELO2 is an in-house logging system that provides log collection, storage, full-text search, alert and analytics features. Developers can easily send their logs to NELO2 collector servers using various SDKs from many platforms. Collectors filter logs, throttle, and store them to Apache Kafka which is a distributed messaging queue. Logs in Kafka are consumed and indexed into Elasticsearch. Through NELO2 Web interface developers can search their logs, aggregate them, register alerts with lucene query, subscribe for daily reports, etc. Elasticsearch is used in NELO2 as a main log storage, search and alert. NELO2 is indexing and percolating 1.5 billion logs daily with Elasticsearch. 126 TB of logs are stored in 7 Elasticsearch clusters. In this session, first, we will talk about the architecture of NELO2 and how we use and maintain Elasticsearch clusters. Then, we will explain the detailed implementation design of the new real time notification system in NELO2.