Processing Details

Understanding how Logstash works and how components interrelate can help you make better decisions when you are setting up or adjusting your Logstash environment. This section is designed to elevate concepts to assist with that level of knowledge.

By design and by default, Logstash does not guarantee event order. Reordering can occur in two places:

When maintaining event order is important, use a single worker and set pipeline.ordered ⇒ true. This approach ensures that batches are computed one-after-the-other, and that events maintain their order within the batch.

The pipeline.ordered setting in logstash.yml gives you more control over event ordering for single worker pipelines.

auto automatically enables ordering if the pipeline.workers setting is also set to 1. true will enforce ordering on the pipeline and prevent logstash from starting if there are multiple workers. false will disable the processing required to preserve order. Ordering will not be guaranteed, but you save the processing cost required to preserve order.

The Java pipeline initialization time appears in the startup logs at INFO level. Initialization time is the time it takes to compile the pipeline config and instantiate the compiled execution for all workers.

Some fields in Logstash events are reserved, or are required to adhere to a certain shape. Using these fields can cause runtime exceptions when the event API or plugins encounter incompatible values.