- Filebeat Reference: other versions:
- Overview
- Getting Started With Filebeat
- Setting up and running Filebeat
- Upgrading Filebeat
- How Filebeat works
- Configuring Filebeat
- Specify which modules to run
- Configure inputs
- Manage multiline messages
- Specify general settings
- Load external configuration files
- Configure the internal queue
- Configure the output
- Configure index lifecycle management
- Load balance the output hosts
- Specify SSL settings
- Filter and enhance the exported data
- Define processors
- Add cloud metadata
- Add fields
- Add labels
- Add the local time zone
- Add tags
- Decode CEF
- Decode CSV fields
- Decode JSON fields
- Decode Base64 fields
- Decompress gzip fields
- Community ID Network Flow Hash
- Convert
- Drop events
- Drop fields from events
- Extract array
- Keep fields from events
- Registered Domain
- Rename fields from events
- Add Kubernetes metadata
- Add Docker metadata
- Add Host metadata
- Add Observer metadata
- Dissect strings
- DNS Reverse Lookup
- Add process metadata
- Script Processor
- Timestamp
- Parse data by using ingest node
- Enrich events with geoIP information
- Configure project paths
- Configure the Kibana endpoint
- Load the Kibana dashboards
- Load the Elasticsearch index template
- Configure logging
- Use environment variables in the configuration
- Autodiscover
- YAML tips and gotchas
- Regular expression support
- HTTP Endpoint
- filebeat.reference.yml
- Beats central management
- Modules
- Modules overview
- Apache module
- Auditd module
- AWS module
- CEF module
- Cisco module
- Coredns Module
- Elasticsearch module
- Envoyproxy Module
- Google Cloud module
- haproxy module
- IBM MQ module
- Icinga module
- IIS module
- Iptables module
- Kafka module
- Kibana module
- Logstash module
- MongoDB module
- MSSQL module
- MySQL module
- nats module
- NetFlow module
- Nginx module
- Osquery module
- Palo Alto Networks module
- PostgreSQL module
- RabbitMQ module
- Redis module
- Santa module
- Suricata module
- System module
- Traefik module
- Zeek (Bro) Module
- Exported fields
- Apache fields
- Auditd fields
- AWS fields
- Beat fields
- Decode CEF processor fields fields
- CEF fields
- Cisco fields
- Cloud provider metadata fields
- Coredns fields
- Docker fields
- ECS fields
- elasticsearch fields
- Envoyproxy fields
- Google Cloud fields
- haproxy fields
- Host fields
- ibmmq fields
- Icinga fields
- IIS fields
- iptables fields
- Jolokia Discovery autodiscover provider fields
- Kafka fields
- kibana fields
- Kubernetes fields
- Log file content fields
- logstash fields
- mongodb fields
- mssql fields
- MySQL fields
- nats fields
- NetFlow fields
- NetFlow fields
- Nginx fields
- Osquery fields
- panw fields
- PostgreSQL fields
- Process fields
- RabbitMQ fields
- Redis fields
- s3 fields
- Google Santa fields
- Suricata fields
- System fields
- Traefik fields
- Zeek fields
- Monitoring Filebeat
- Securing Filebeat
- Troubleshooting
- Get help
- Debug
- Common problems
- Can’t read log files from network volumes
- Filebeat isn’t collecting lines from a file
- Too many open file handlers
- Registry file is too large
- Inode reuse causes Filebeat to skip lines
- Log rotation results in lost or duplicate events
- Open file handlers cause issues with Windows file rotation
- Filebeat is using too much CPU
- Dashboard in Kibana is breaking up data fields incorrectly
- Fields are not indexed or usable in Kibana visualizations
- Filebeat isn’t shipping the last line of a file
- Filebeat keeps open file handlers of deleted files for a long time
- Filebeat uses too much bandwidth
- Error loading config file
- Found unexpected or unknown characters
- Logstash connection doesn’t work
- @metadata is missing in Logstash
- Not sure whether to use Logstash or Beats
- SSL client fails to connect to Logstash
- Monitoring UI shows fewer Beats than expected
- Contributing to Beats
AWS fields
editAWS fields
editModule for handling logs from AWS.
aws
editFields from AWS logs.
s3access
editFields for AWS S3 server access logs.
-
aws.s3access.bucket_owner -
The canonical user ID of the owner of the source bucket.
type: keyword
-
aws.s3access.bucket -
The name of the bucket that the request was processed against.
type: keyword
-
aws.s3access.remote_ip -
The apparent internet address of the requester.
type: ip
-
aws.s3access.requester -
The canonical user ID of the requester, or a - for unauthenticated requests.
type: keyword
-
aws.s3access.request_id -
A string generated by Amazon S3 to uniquely identify each request.
type: keyword
-
aws.s3access.operation -
The operation listed here is declared as SOAP.operation, REST.HTTP_method.resource_type, WEBSITE.HTTP_method.resource_type, or BATCH.DELETE.OBJECT.
type: keyword
-
aws.s3access.key -
The "key" part of the request, URL encoded, or "-" if the operation does not take a key parameter.
type: keyword
-
aws.s3access.request_uri -
The Request-URI part of the HTTP request message.
type: keyword
-
aws.s3access.http_status -
The numeric HTTP status code of the response.
type: long
-
aws.s3access.error_code -
The Amazon S3 Error Code, or "-" if no error occurred.
type: keyword
-
aws.s3access.bytes_sent -
The number of response bytes sent, excluding HTTP protocol overhead, or "-" if zero.
type: long
-
aws.s3access.object_size -
The total size of the object in question.
type: long
-
aws.s3access.total_time -
The number of milliseconds the request was in flight from the server’s perspective.
type: long
-
aws.s3access.turn_around_time -
The number of milliseconds that Amazon S3 spent processing your request.
type: long
-
aws.s3access.referrer -
The value of the HTTP Referrer header, if present.
type: keyword
-
aws.s3access.user_agent -
The value of the HTTP User-Agent header.
type: keyword
-
aws.s3access.version_id -
The version ID in the request, or "-" if the operation does not take a versionId parameter.
type: keyword
-
aws.s3access.host_id -
The x-amz-id-2 or Amazon S3 extended request ID.
type: keyword
-
aws.s3access.signature_version -
The signature version, SigV2 or SigV4, that was used to authenticate the request or a - for unauthenticated requests.
type: keyword
-
aws.s3access.cipher_suite -
The Secure Sockets Layer (SSL) cipher that was negotiated for HTTPS request or a - for HTTP.
type: keyword
-
aws.s3access.authentication_type -
The type of request authentication used, AuthHeader for authentication headers, QueryString for query string (pre-signed URL) or a - for unauthenticated requests.
type: keyword
-
aws.s3access.host_header -
The endpoint used to connect to Amazon S3.
type: keyword
-
aws.s3access.tls_version -
The Transport Layer Security (TLS) version negotiated by the client.
type: keyword