This documentation contains work-in-progress information for future Elastic Stack and Cloud releases. Use the version selector to view supported release docs. It also contains some Elastic Cloud serverless information. Check out our serverless docs for more details.
Elastic Agent to Logstash to Kafka to Kafka ES Sink to Elasticsearch: Kafka as middleware message queue
editElastic Agent to Logstash to Kafka to Kafka ES Sink to Elasticsearch: Kafka as middleware message queue
edit
- Ingest model
-
Control path: Elastic Agent to Fleet to Elasticsearch
Data path: Elastic Agent to Logstash to Kafka to Kafka ES Sink to Elasticsearch: Kafka as middleware message queue.Kafka ES Sink connector reads from Kafka and writes to Elasticsearch.
- Use when
- You are standardizing on Kafka as middleware message queue between Elastic Agent and Elasticsearch
- Notes
- The transformation from raw data to Elastic Common Schema (ECS) and any other enrichment can be handled by Logstash as described in Elastic Agent to Logstash (for enrichment) to Elasticsearch.
Resources
editInfo on Elastic Agent and agent integrations:
Info on Logstash and Logstash plugins:
Info on Elasticsearch:
- Elasticsearch Guide
- ES sink [ToDo: Add link]