« Ingest pipelines Machine learning »
Elastic Docs ›Serverless ›Project and management settings ›Management settings

Logstash pipelines

edit

Logstash pipelines

edit

[preview] This functionality is in technical preview and may be changed or removed in a future release. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features.

This content applies to: Elasticsearch Observability Security

In Project settings → Management → Logstash Pipelines, you can control multiple Logstash instances and pipeline configurations.

Logstash Pipelines"

On the Logstash side, you must enable configuration management and register Logstash to use the centrally managed pipeline configurations.

After you configure Logstash to use centralized pipeline management, you can no longer specify local pipeline configurations. The pipelines.yml file and settings such as path.config and config.string are inactive when centralized pipeline management is enabled.

Manage pipelines
edit
  1. Configure centralized pipeline management.

  2. To add a new pipeline, go to Project settings → Management → Logstash Pipelines and click Create pipeline. Provide the following details, then click Create and deploy.

    Pipeline ID
    A name that uniquely identifies the pipeline. This is the ID that you used when you configured centralized pipeline management and specified a list of pipeline IDs in the xpack.management.pipeline.id setting.
    Description
    A description of the pipeline configuration. This information is for your use.
    Pipeline
    The pipeline configuration. You can treat the editor like any other editor. You don’t have to worry about whitespace or indentation.
    Pipeline workers
    The number of parallel workers used to run the filter and output stages of the pipeline.
    Pipeline batch size
    The maximum number of events an individual worker thread collects before executing filters and outputs.
    Pipeline batch delay
    Time in milliseconds to wait for each event before sending an undersized batch to pipeline workers.
    Queue type
    The internal queueing model for event buffering. Options are memory for in-memory queueing and persisted for disk-based acknowledged queueing.
    Queue max bytes
    The total capacity of the queue when persistent queues are enabled.
    Queue checkpoint writes
    The maximum number of events written before a checkpoint is forced when persistent queues are enabled.

To delete one or more pipelines, select their checkboxes then click Delete.

For more information about pipeline behavior, go to Centralized Pipeline Management.

« Ingest pipelines Machine learning »