Logstash Introduction
editLogstash Introduction
editLogstash is an open source data collection engine with real-time pipelining capabilities. Logstash can dynamically unify data from disparate sources and normalize the data into destinations of your choice. Cleanse and democratize all your data for diverse advanced downstream analytics and visualization use cases.
While Logstash originally drove innovation in log collection, its capabilities extend well beyond that use case. Any type of event can be enriched and transformed with a broad array of input, filter, and output plugins, with many native codecs further simplifying the ingestion process. Logstash accelerates your insights by harnessing a greater volume and variety of data.
The Power of Logstash
editThe ingestion workhorse for Elasticsearch and more
Horizontally scalable data processing pipeline with strong Elasticsearch and Kibana synergy
Pluggable pipeline architecture
Mix, match, and orchestrate different inputs, filters, and outputs to play in pipeline harmony
Community-extensible and developer-friendly plugin ecosystem
Over 200 plugins available, plus the flexibility of creating and contributing your own
Logstash Loves Data
editCollect more, so you can know more. Logstash welcomes data of all shapes and sizes.
Logs and Metrics
editWhere it all started.
-
Handle all types of logging data
- Easily ingest a multitude of web logs like Apache, and application logs like log4j for Java
- Capture many other log formats like syslog, Windows event logs, networking and firewall logs, and more
- Enjoy complementary secure log forwarding capabilities with Filebeat
- Collect metrics from Ganglia, collectd, NetFlow, JMX, and many other infrastructure and application platforms over TCP and UDP
The Web
editUnlock the World Wide Web.
-
Transform HTTP requests into events (blog)
-
Create events by polling HTTP endpoints on demand (blog)
- Universally capture health, performance, metrics, and other types of data from web application interfaces
- Perfect for scenarios where the control of polling is preferred over receiving
Data Stores and Streams
editDiscover more value from the data you already own.
Sensors and IoT
editExplore an expansive breadth of other data.
- In this age of technological advancement, the massive IoT world unleashes endless use cases through capturing and harnessing data from connected sensors.
- Logstash is the common event collection backbone for ingestion of data shipped from mobile devices to intelligent homes, connected vehicles, healthcare sensors, and many other industry specific applications.
- Watch as Logstash, in conjunction with the broader ELK stack, centralizes and enriches sensor data to gain deeper knowledge regarding a residential home.
Easily Enrich Everything
editThe better the data, the better the knowledge. Clean and transform your data during ingestion to gain near real-time insights immediately at index or output time. Logstash comes out-of-box with many aggregations and mutations along with pattern matching, geo mapping, and dynamic lookup capabilities.
- Grok is the bread and butter of Logstash filters and is used ubiquitously to derive structure out of unstructured data. Enjoy a wealth of integrated patterns aimed to help quickly resolve web, systems, networking, and other types of event formats.
- Expand your horizons by deciphering geo coordinates from IP addresses, normalizing date complexity, simplifying key-value pairs and CSV data, anonymizing sensitive information, and further enriching your data with local lookups or Elasticsearch queries.
- Codecs are often used to ease the processing of common event structures like JSON and multiline events.
Choose Your Stash
editRoute your data where it matters most. Unlock various downstream analytical and operational use cases by storing, analyzing, and taking action on your data.