Optimizing Storage Efficiency in Elasticsearch

Christian Dahlqvist

Solutions Architect

Elastic

Alan Woodward

Elastic

Organizations are using the Elastic Stack for a variety of use cases. Many of these use cases, such as centralized logging and security analytics, deal with indexing, storing, and analyzing large volumes of logs and metrics into Elasticsearch. At these large volumes, cluster architectures and data tiering strategies are important to ensure that your access and speed needs are balanced with the storage costs. 

In this webinar, we will look at how hot-warm-cold cluster architectures allow you to achieve that balance, and the various considerations that go into deciding the data tiering policy. We will also discuss the factors that influence how much data can be put on each node, and how you can think about optimizing that. 

Highlights include:

• Overview of hot-warm-cold architecture and node characteristics
• Impact of use-case characteristics on node storage density
• How to optimize your data for storage efficiency
• How different lucene data structures can affect storage density

Additional Resources (Related content)

• Blogs:
  • Efficient duplicate prevention for event-based data in Elasticsearch 
  • Sizing hot-warm architectures on Elastic Cloud
  • Overview of Elasticsearch hot-warm architecture
  • 7 tips for better Elasticsearch benchmarks
  • How many shards should I have in my Elasticsearch cluster?
• Videos:
  • Quantitative Cluster Sizing
  • Using Rally to get your Elasticsearch cluster size right
• Docs:
  • Tune for disk usage