- Elastic Ingest Reference Architectures: other versions:
- Ingest architectures
- Elastic Agent to Elasticsearch
- Elastic Agent to Logstash to Elasticsearch
- Elastic Agent to Logstash (for enrichment) to Elasticsearch
- Elastic Agent to Logstash to Elasticsearch: Logstash Persistent Queue (PQ) for buffering
- Elastic Agent to Logstash to Elasticsearch: Logstash as a proxy
- Elastic Agent to Logstash for routing to multiple Elasticsearch clusters and additional destinations
- Elastic Agent to proxy to Elasticsearch
- Elastic Agent to Elasticsearch with Kafka as middleware message queue
- Logstash to Elasticsearch
- Elastic air-gapped architectures
Elastic Agent to Logstash to Kafka to Logstash to Elasticsearch: Kafka as middleware message queue
editElastic Agent to Logstash to Kafka to Logstash to Elasticsearch: Kafka as middleware message queue
edit
- Ingest model
-
Control path: Elastic Agent to Fleet to Elasticsearch
Data path: Elastic Agent to Logstash to Kafka to Logstash to Elasticsearch: Kafka as middleware message queue.Logstash reads data from Kafka and routes it to Elasticsearch clusters (and/or other destinations)
- Use when
- You are standardizing on Kafka as middleware message queue between Elastic Agent and Elasticsearch
- Notes
- The transformation from raw data to Elastic Common Schema (ECS) and any other enrichment can be handled by Logstash as described in Elastic Agent to Logstash (for enrichment) to Elasticsearch.
Resources
editInfo on Elastic Agent and agent integrations:
Info on Logstash and Logstash Kafka plugins:
Info on Elasticsearch:
On this page
Was this helpful?
Thank you for your feedback.