Tcp input plugin

Log4j2 can send JSON over a socket, and we can use that combined with our tcp input to accept the logs.

First, we need to configure your application to send logs in JSON over a socket. The following log4j2.xml accomplishes this task.

Note, you will want to change the host and port settings in this configuration to match your needs.

<Configuration>
  <Appenders>
     <Socket name="Socket" host="localhost" port="12345">
       <JsonLayout compact="true" eventEol="true" />
    </Socket>
  </Appenders>
  <Loggers>
    <Root level="info">
      <AppenderRef ref="Socket"/>
    </Root>
  </Loggers>
</Configuration>

To accept this in Logstash, you will want tcp input and a date filter:

input {
  tcp {
    port => 12345
    codec => json
  }
}

and add a date filter to take log4j2’s timeMillis field and use it as the event timestamp

filter {
  date {
    match => [ "timeMillis", "UNIX_MS" ]
  }
}

It is possible to avoid DNS reverse-lookups by disabling this setting. If disabled, the address metadata that is added to events will contain the source address as-specified at the TCP layer and IPs will not be resolved to hostnames.

Controls this plugin’s compatibility with the Elastic Common Schema (ECS). The value of this setting affects the placement of a TCP connection’s metadata on events.

When mode is server, the address to listen on. When mode is client, the address to connect to.

Mode to operate in. server listens for client connections, client connects to a server.

When mode is server, the port to listen on. When mode is client, the port to connect to.

Proxy protocol support, only v1 is supported at this time http://www.haproxy.org/download/1.5/doc/proxy-protocol.txt

Path to certificate in PEM format. This certificate will be presented to the connecting clients.

Path to certificate in PEM format. This certificate will be presented to the other part of the TLS connection.

Validate client certificate or certificate chain against these authorities. You can define multiple files or paths. All the certificates will be read and added to the trust store.

Supported cipher suites vary depending on Java version used, and entries look like TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384. For more information, see Oracle’s JDK SunJSSE provider documentation and the table of supported Java cipher suite names.

Controls the server’s behavior in regard to requesting a certificate from client connections: none disables the client authentication. required forces a client to present a certificate, while optional requests a client certificate but the client is not required to present one.

When mutual TLS is enabled (optional or required), the certificate presented by the client must be signed by trusted ssl_certificate_authorities (CAs). Please note that the server does not validate the client certificate CN (Common Name) or SAN (Subject Alternative Name).

Enable SSL (must be set for other ssl_ options to take effect).

Enable SSL (must be set for other ssl_ options to take effect).

An Array of paths to extra X509 certificates. These are used together with the certificate to construct the certificate chain presented to the client.

The path to the private key corresponding to the specified certificate (PEM format).

SSL key passphrase for the private key.

List of allowed SSL/TLS versions to use when establishing a secure connection.

Defines how to verify the certificates presented by another party in the TLS connection:

full validates that the server certificate has an issue date that’s within the not_before and not_after dates; chains to a trusted Certificate Authority (CA), and has a hostname or IP address that matches the names within the certificate.

none performs no certificate validation.

This setting can be used only if mode is client.

Verify the identity of the other end of the SSL connection against the CA. For input, sets the field sslsubject to that of the client certificate.

Instruct the socket to use TCP keep alive. If it’s true then the underlying socket will use the OS defaults settings for keep alive. If it’s false it doesn’t configure any keep alive setting for the underlying socket.

Add a field to an event

The codec used for input data. Input codecs are a convenient method for decoding your data before it enters the input, without needing a separate filter in your Logstash pipeline.

Disable or enable metric logging for this specific plugin instance by default we record all the metrics we can, but you can disable metrics collection for a specific plugin.

Add a unique ID to the plugin configuration. If no ID is specified, Logstash will generate one. It is strongly recommended to set this ID in your configuration. This is particularly useful when you have two or more plugins of the same type, for example, if you have 2 tcp inputs. Adding a named ID in this case will help in monitoring Logstash when using the monitoring APIs.

input {
  tcp {
    id => "my_plugin_id"
  }
}

Add any number of arbitrary tags to your event.

This can help with processing later.

Add a type field to all events handled by this input.

Types are used mainly for filter activation.

The type is stored as part of the event itself, so you can also use the type to search for it in Kibana.

If you try to set a type on an event that already has one (for example when you send an event from a shipper to an indexer) then a new input will not override the existing type. A type set at the shipper stays with that event for its life even when sent to another Logstash server.