This documentation contains work-in-progress information for future Elastic Stack and Cloud releases. Use the version selector to view supported release docs. It also contains some Elastic Cloud serverless information. Check out our serverless docs for more details.

« Elastic Agent input plugin Elasticsearch input plugin »

› ›

Elastic Serverless Forwarder input plugin

edit

Elastic Serverless Forwarder input plugin

edit

Plugin version: v0.1.5
Released on: 2024-09-12
Changelog

For other versions, see the Versioned plugin docs.

Getting help

edit

For questions about the plugin, open a topic in the Discuss forums. For bugs or feature requests, open an issue in Github. For the list of Elastic supported plugins, please consult the Elastic Support Matrix.

Description

edit

Using this input you can receive events from Elastic Serverless Forwarder over http(s) connections to the configured port.

Minimum Configuration

edit

SSL Enabled	SSL Disabled
input { elastic_serverless_forwarder { port => 8080 ssl_certificate => "/path/to/logstash.crt" ssl_key => "/path/to/logstash.key" } }	input { elastic_serverless_forwarder { port => 8080 ssl_enabled => false } }

SSL Enabled

SSL Disabled

input {
  elastic_serverless_forwarder {
    port => 8080
    ssl_certificate => "/path/to/logstash.crt"
    ssl_key => "/path/to/logstash.key"
  }
}

input {
  elastic_serverless_forwarder {
    port => 8080
    ssl_enabled => false
  }
}

Enrichment

edit

This input provides minimal enrichment on events, and avoids including information about itself, the client from which it received the data, or about the original event as-decoded from the request.

Senders are advised to use care with respect to fields that are reserved in Logstash. ESF sends the Logstash-required @timestamp field by default, but if this value is missing it will be populated with the current time.

Security

edit

This plugin has SSL on-by-default.

At a minimum, you will need to either configure the plugin to present its identity, or disable SSL.

Additionally, you may wish to authenticate clients using SSL client authentication, and/or authenticate requests using HTTP Basic authentication as described below.

SSL Identity

edit

In order to establish SSL with a client, this input plugin will need to present an SSL certificate that the client trusts, and have access to the associated key. These are configurable with ssl_certificate, ssl_key, and optionally ssl_key_passphrase.

SSL Client Authentication

edit

By default, this plugin does not request certificates from clients during SSL negotiation.

It can be configured to either request or require client certificates using ssl_client_authentication, which often also requires configuring it with a list of ssl_certificate_authorities to trust. When validating a certificate that is presented, ssl_verification_mode controls how certificates are verified.

ESF does not currently support presenting client certificates, so requesting or requiring clients to present identity is only useful when combined with an SSL-terminating proxy.

SSL Advanced Configuration

edit

This plugin exposes several advanced SSL configurations:

HTTP Basic Authentication

edit

You can configure this plugin to authenticate requests using HTTP Basic authentication by configuring auth_basic_username and auth_basic_password.

Basic Authentication is not a substitute for SSL, as it provides neither secrecy nor security on its own. When used with SSL disabled, HTTP Basic credentials are transmitted in effectively clear-text and can be easily recovered by an adversary.

Using Elastic Serverless Forwarder with the Elasticsearch output

edit

Here are some tips for configuring the Elastic Serverless Forwarder input to work with the elasticsearch output:

Set the document_id in the output configuration when you use the Elastic Serverless Forwarder input with an Elasticsearch output plugin.
```
output {
  elasticsearch {
      ...
      document_id => "%{[@metadata][_id]}"
      ...
  }
}
```
Starting from version 1.10.0 of Elastic Serverless Forwarder, configuring document_id as shown in the example above is sufficient (the _id field is no longer available, and instead, Logstash now receives the @metadata._id field).

For Elastic Serverless Forwarder v1.9.0 and earlier, rename the field _id to @metadata._id with a filter:

filter {
    # support ESF < 1.10
    if [_id] and ![@metadata][_id] {
      mutate { rename => { "_id" => "[@metadata][_id]" } }
    }
}

Elastic Serverless Forwarder Input Configuration Options

edit

This plugin supports the following configuration options plus the Common options described later.

Setting	Input type	Required
`auth_basic_username`	string	No
`auth_basic_password`	password	No
`host`	string	No
`port`	number	No
`ssl`	boolean	Deprecated
`ssl_certificate`	a valid filesystem path	No
`ssl_certificate_authorities`	array	No
`ssl_client_authentication`	string, one of `["none", "optional", "required"]`	No
`ssl_cipher_suites`	array	No
`ssl_enabled`	boolean	No
`ssl_handshake_timeout`	number	No
`ssl_key`	a valid filesystem path	No
`ssl_key_passphrase`	password	No
`ssl_supported_protocols`	array	No
`ssl_verification_mode`	string, one of `["certificate"]`	No

Also see Common options for a list of options supported by all input plugins.

`auth_basic_password`

edit

Value type is password
There is no default value for this setting.

Password for HTTP basic authorization. Requires auth_basic_username.

`auth_basic_username`

edit

Value type is string
There is no default value for this setting.

Username for basic authorization. Requires auth_basic_password.

`host`

edit

Value type is string
Default value is "0.0.0.0" (all available interfaces)

The host or ip to bind

`port`

edit

Value type is number
Default value is 8080

The TCP port to bind to

`ssl`

edit

Deprecated in 0.1.3.

Replaced by ssl_enabled

Value type is boolean
Default value is true

Events are by default sent over SSL, which requires configuring this plugin to present an identity certificate using ssl_certificate and key using ssl_key.

You can disable SSL with +ssl => false+.

`ssl_certificate`

edit

Value type is path
There is no default value for this setting.

SSL certificate to use. This certificate MUST be PEM-formatted, and MAY contain a chain of certificates starting with the certificate that identifies itself, followed by zero or more ordered intermediates optionally ending with the root signing authority. Providing a complete chain allows clients to trust our certificate if their configuration allows them to trust one of our intermediates.

`ssl_certificate_authorities`

edit

Value type is array
Default value is []

Validate client certificates against these authorities. You can define multiple files or paths. All the certificates will be read and added to the trust store.

If you wish to perform client authentication, you need to set ssl_client_authentication to optional or required.

`ssl_cipher_suites`

edit

Value type is array
Default value is ['TLS_AES_256_GCM_SHA384', 'TLS_AES_128_GCM_SHA256', 'TLS_CHACHA20_POLY1305_SHA256', 'TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384', 'TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384', 'TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256', 'TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256', 'TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256', 'TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256', 'TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384', 'TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384', 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256', 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256']

The list of cipher suites to use, listed by priorities.

This is an advanced SSL configuration.

This default list applies for OpenJDK 11.0.14 and higher. For older JDK versions, the default list includes only suites supported by that version. For example, the ChaCha20 family of ciphers is not supported in older versions.

`ssl_client_authentication`

edit

Value can be any of:
- none: do not request client’s certificate, or validate certificates that are presented
- optional: request client’s certificate, and validate it against our trust authorities if-and-only-if it is presented
- required: require a valid certificate from the client that is signed by a trusted certificate authority
Default value is "none"

By default the server doesn’t do any client authentication. This means that connections from clients are private when SSL is enabled, but that this input will allow SSL connections from any client. If you wish to configure this plugin to reject connections from untrusted hosts, you will need to configure this plugin to authenticate clients, and may also need to configure it with a list of ssl_certificate_authorities.

`ssl_enabled`

edit

Value type is boolean
Default value is true

Events are, by default, sent over SSL, which requires configuring this plugin to present an identity certificate using ssl_certificate and key using ssl_key.

You can disable SSL with +ssl_enabled => false+.

`ssl_handshake_timeout`

edit

Value type is number
Default value is 10000

Time in milliseconds for an incomplete ssl handshake to timeout

This is an advanced SSL configuration.

`ssl_key`

edit

Value type is path
There is no default value for this setting.

SSL key to use.

This key need to be in the PKCS8 format, you can convert it with OpenSSL for more information.

`ssl_key_passphrase`

edit

Value type is password
There is no default value for this setting.

SSL key passphrase to use.

`ssl_supported_protocols`

edit

Value type is array
Allowed values are: 'TLSv1.1', 'TLSv1.2', 'TLSv1.3'
Default depends on the JDK being used. With up-to-date Logstash, the default is ['TLSv1.2', 'TLSv1.3']. 'TLSv1.1' is not considered secure and is only provided for legacy applications.

List of allowed SSL/TLS versions to use when establishing a connection to the HTTP endpoint.

This is an advanced SSL configuration.

For Java 8 'TLSv1.3' is supported only since 8u262 (AdoptOpenJDK), but requires that you set the LS_JAVA_OPTS="-Djdk.tls.client.protocols=TLSv1.3" system property in Logstash.

If you configure the plugin to use 'TLSv1.1' on any recent JVM, such as the one packaged with Logstash, the protocol is disabled by default and needs to be enabled manually by changing jdk.tls.disabledAlgorithms in the $JDK_HOME/conf/security/java.security configuration file. That is, TLSv1.1 needs to be removed from the list.

`ssl_verification_mode`

edit

Value type is string
There is only one currently-supported mode:
- certificate: verifies that a certificate provided by the client is signed by a trusted authority (CA), is within its valid date range, and that the client has possession of the associated key, but does not perform hostname validation.

The default value is certificate.

When ssl_client_authentication causes a client to present a certificate, this setting controls how that certificate is verified.

Client identity is not typically validated using SSL because the receiving server only has access to the client’s outbound-ip, which is not always constant and is frequently not represented in the certificate’s subject or subjectAltNames extensions. For more information, see RFC2818 § 3.2 (HTTP over TLS — Client Identity)

Common options

edit

These configuration options are supported by all input plugins:

Setting	Input type	Required
`add_field`	hash	No
`enable_metric`	boolean	No
`id`	string	No
`tags`	array	No
`type`	string	No

`add_field`

edit

Value type is hash
Default value is {}

Add a field to an event

`enable_metric`

edit

Value type is boolean
Default value is true

Disable or enable metric logging for this specific plugin instance by default we record all the metrics we can, but you can disable metrics collection for a specific plugin.

`id`

edit

Value type is string
There is no default value for this setting.

Add a unique ID to the plugin configuration. If no ID is specified, Logstash will generate one. It is strongly recommended to set this ID in your configuration. This is particularly useful when you have two or more plugins of the same type, for example, if you have 2 elastic_serverless_forwarder inputs. Adding a named ID in this case will help in monitoring Logstash when using the monitoring APIs.

input {
  elastic_serverless_forwarder {
    id => "my_plugin_id"
  }
}

Variable substitution in the id field only supports environment variables and does not support the use of values from the secret store.

`tags`

edit

Value type is array
There is no default value for this setting.

Add any number of arbitrary tags to your event.

This can help with processing later.

`type`

edit

Value type is string
There is no default value for this setting.

Add a type field to all events handled by this input.

Types are used mainly for filter activation.

The type is stored as part of the event itself, so you can also use the type to search for it in Kibana.

If you try to set a type on an event that already has one (for example when you send an event from a shipper to an indexer) then a new input will not override the existing type. A type set at the shipper stays with that event for its life even when sent to another Logstash server.

« Elastic Agent input plugin Elasticsearch input plugin »