Transaction Protocols Configuration

The enabled setting is a boolean setting to enable or disable protocols without having to comment out configuration sections. If set to false, the protocol is disabled.

The default value is true.

Exception: For ICMP the option enabled has to be used instead.

The ports where Packetbeat will look to capture traffic for specific protocols. Packetbeat installs a BPF filter based on the ports specified in this section. If a packet doesn’t match the filter, very little CPU is required to discard the packet. Packetbeat also uses the ports specified here to determine which parser to use for each packet.

If this option is enabled, the raw message of the request (request field) is sent to Elasticsearch. The default is false. This option is useful when you want to index the whole request. Note that for HTTP, the body is not included by default, only the HTTP headers.

If this option is enabled, the raw message of the response (response field) is sent to Elasticsearch. The default is false. This option is useful when you want to index the whole response. Note that for HTTP, the body is not included by default, only the HTTP headers.

The per protocol transaction timeout. Expired transactions will no longer be correlated to incoming responses, but sent to Elasticsearch immediately.

The ICMP protocol can be enabled/disabled via this option. The default is true.

If enabled Packetbeat will generate the following BPF filter: "icmp or icmp6".

If this option is enabled, dns.authority fields (authority resource records) are added to DNS events. The default is false.

If this option is enabled, dns.additionals fields (additional resource records) are added to DNS events. The default is false.

A list of query parameters that Packetbeat will automatically censor in the transactions that it saves. The values associated with these parameters are replaced by 'xxxxx'. By default, no changes are made to the HTTP messages.

Packetbeat has this option because, unlike SQL traffic, which typically only contains the hashes of the passwords, HTTP traffic may contain sensitive data. To reduce security risks, you can configure this option to avoid sending the contents of certain HTTP POST parameters.

When this option is enabled, Packetbeat obscures the value of Authorization and Proxy-Authorization HTTP headers, and censors those strings in the response.

You should set this option to true for transactions that use Basic Authentication because they may contain the base64 unencrypted username and password.

A list of header names to capture and send to Elasticsearch. These headers are placed under the headers dictionary in the resulting JSON.

Instead of sending a white list of headers to Elasticsearch, you can send all headers by setting this option to true. The default is false.

The list of content types for which Packetbeat exports the full HTTP payload. The HTTP body is available under http.request.body and http.response.body for these Content-Types.

In addition, if send_response option is enabled, then the HTTP body is exported together with the HTTP headers under response and if send_request enabled, then request contains the entire HTTP message including the body.

In the following example, the HTML attachments of the HTTP responses are exported under the response field and under http.request.body or http.response.body:

packetbeat.protocols.http:
  ports: [80, 8080]
  send_response: true
  include_body_for: ["text/html"]

If the Cookie or Set-Cookie headers are sent, this option controls whether they are split into individual values. For example, with this option set, an HTTP response might result in the following JSON:

"response": {
  "code": 200,
  "headers": {
    "connection": "close",
    "content-language": "en",
    "content-type": "text/html; charset=utf-8",
    "date": "Fri, 21 Nov 2014 17:07:34 GMT",
    "server": "gunicorn/19.1.1",
    "set-cookie": { 
      "csrftoken": "S9ZuJF8mvIMT5CL4T1Xqn32wkA6ZSeyf",
      "expires": "Fri, 20-Nov-2015 17:07:34 GMT",
      "max-age": "31449600",
      "path": "/"
    },
    "vary": "Cookie, Accept-Language"
  },
  "phrase": "OK"
}

The default is false.

The header field to extract the real IP from. This setting is useful when you want to capture traffic behind a reverse proxy, but you want to get the geo-location information. If this header is present and contains a valid IP addresses, the information is used for the real_ip field.

If an individual HTTP message is larger than this setting (in bytes), it will be trimmed to this size. Unless this value is very small (<1.5K), Packetbeat is able to still correctly follow the transaction and create an event for it. The default is 10485760 (10 MB).

The maximum size in bytes of the message displayed in the request or response fields. Messages that are bigger than the specified size are truncated. Use this option to avoid publishing huge messages when send_request or send_request is enabled. The default is 1000 bytes.

If set to true, Packetbeat parses the additional arguments specified in the headers field of a message. Those arguments are key-value pairs that specify information such as the content type of the message or the message priority. The default is true.

If set to true, Packetbeat parses the additional arguments specified in AMQP methods. Those arguments are key-value pairs specified by the user and can be of any length. The default is true.

If set to false, the connection layer methods of the protocol are also displayed, such as the opening and closing of connections and channels by clients, or the quality of service negotiation. The default is true.

If this option is enabled, the raw message of the response (cassandra_request.request_headers field) is sent to Elasticsearch. The default is true. enable send_request first before enable this option.

If this option is enabled, the raw message of the response (cassandra_response.response_headers field) is included in published events. The default is true. enable send_response first before enable this option.

This option indicates which Operator/Operators captured will be ignored. currently support: ERROR ,STARTUP ,READY ,AUTHENTICATE ,OPTIONS ,SUPPORTED , QUERY ,RESULT ,PREPARE ,EXECUTE ,REGISTER ,EVENT , BATCH ,AUTH_CHALLENGE,AUTH_RESPONSE ,AUTH_SUCCESS .

Configures the default compression algorithm being used to uncompress compressed frames by name. Currently only snappy is can be configured. By default no compressor is configured.

When this option is enabled, it forces the memcache text protocol parser to accept unknown commands.

The maximum number of values to store in the message (multi-get). All values will be base64 encoded.

The possible settings for this option are:

The maximum number of bytes to be copied for each value element.

The transaction timeout in milliseconds. The defaults is 10000 milliseconds.

The maximum number of rows from the SQL message to publish to Elasticsearch. The default is 10 rows.

The maximum length in bytes of a row from the SQL message to publish to Elasticsearch. The default is 1024 bytes.

The Thrift transport type. Currently this option accepts the values socket for TSocket, which is the default Thrift transport, and framed for the TFramed Thrift transport. The default is socket.

The Thrift protocol type. Currently the only accepted value is binary for the TBinary protocol, which is the default Thrift protocol.

The Thrift interface description language (IDL) files for the service that Packetbeat is monitoring. Providing the IDL files is optional, because the Thrift messages contain enough information to decode them without having the IDL files. However, providing the IDL enables Packetbeat to include parameter and exception names.

The maximum length for strings in parameters or return values. If a string is longer than this value, the string is automatically truncated to this length. Packetbeat adds dots at the end of the string to mark that it was truncated. The default is 200.

The maximum number of elements in a Thrift list, set, map, or structure. If a collection has more elements than this value, Packetbeat captures only the specified number of elements. Packetbeat adds a fictive last element ... to the end of the collection to mark that it was truncated. The default is 15.

If this option is set to false, Packetbeat decodes the method name from the reply and simply skips the rest of the response message. This setting can be useful for performance, disk usage, or data retention reasons. The default is true.

If this option is set to true, Packetbeat replaces all strings found in method parameters, return codes, or exception structures with the "*" string.

The maximum number of fields that a structure can have before Packetbeat ignores the whole transaction. This is a memory protection mechanism (so that Packetbeat’s memory doesn’t grow indefinitely), so you would typically set this to a relatively high value. The default is 500.

The maximum number of documents from the response to index in the response field. The default is 10. You can set this to 0 to index an unlimited number of documents.

Packetbeat adds a [...] line at the end to signify that there were additional documents that weren’t saved because of this setting.

The maximum number of characters in a single document indexed in the response field. The default is 5000. You can set this to 0 to index an unlimited number of characters per document.

If the document is trimmed because of this setting, Packetbeat adds the string ... at the end of the document.

Note that limiting documents in this way means that they are no longer correctly formatted JSON objects.