- Filebeat Reference: other versions:
- Filebeat overview
- Quick start: installation and configuration
- Set up and run
- Upgrade
- How Filebeat works
- Configure
- Inputs
- Modules
- General settings
- Project paths
- Config file loading
- Output
- Kerberos
- SSL
- Index lifecycle management (ILM)
- Elasticsearch index template
- Kibana endpoint
- Kibana dashboards
- Processors
- Define processors
- add_cloud_metadata
- add_cloudfoundry_metadata
- add_docker_metadata
- add_fields
- add_host_metadata
- add_id
- add_kubernetes_metadata
- add_labels
- add_locale
- add_network_direction
- add_observer_metadata
- add_process_metadata
- add_tags
- community_id
- convert
- copy_fields
- decode_base64_field
- decode_cef
- decode_csv_fields
- decode_json_fields
- decompress_gzip_field
- detect_mime_type
- dissect
- dns
- drop_event
- drop_fields
- extract_array
- fingerprint
- include_fields
- rate_limit
- registered_domain
- rename
- script
- timestamp
- translate_sid
- truncate_fields
- urldecode
- Autodiscover
- Internal queue
- Load balancing
- Logging
- HTTP endpoint
- Regular expression support
- Instrumentation
- filebeat.reference.yml
- How to guides
- Override configuration settings
- Load the Elasticsearch index template
- Change the index name
- Load Kibana dashboards
- Load ingest pipelines
- Enrich events with geoIP information
- Deduplicate data
- Parse data by using ingest node
- Use environment variables in the configuration
- Avoid YAML formatting problems
- Beats central management
- Modules
- Modules overview
- ActiveMQ module
- Apache module
- Auditd module
- AWS module
- Azure module
- Barracuda module
- Bluecoat module
- CEF module
- Check Point module
- Cisco module
- CoreDNS module
- Crowdstrike module
- Cyberark module
- Cylance module
- Elasticsearch module
- Envoyproxy Module
- F5 module
- Fortinet module
- Google Cloud module
- Google Workspace module
- GSuite module
- haproxy module
- IBM MQ module
- Icinga module
- IIS module
- Imperva module
- Infoblox module
- Iptables module
- Juniper module
- Kafka module
- Kibana module
- Logstash module
- Microsoft module
- MISP module
- MongoDB module
- MSSQL module
- MySQL module
- MySQL Enterprise module
- nats module
- NetFlow module
- Netscout module
- Nginx module
- Office 365 module
- Okta module
- Oracle module
- Osquery module
- Palo Alto Networks module
- PostgreSQL module
- Proofpoint module
- RabbitMQ module
- Radware module
- Redis module
- Santa module
- Snort module
- Snyk module
- Sonicwall module
- Sophos module
- Squid module
- Suricata module
- System module
- Tomcat module
- Traefik module
- Zeek (Bro) Module
- Zoom module
- Zscaler module
- Exported fields
- ActiveMQ fields
- Apache fields
- Auditd fields
- AWS fields
- aws-cloudwatch fields
- Azure fields
- Barracuda Web Application Firewall fields
- Beat fields
- Blue Coat Director fields
- Decode CEF processor fields fields
- CEF fields
- Checkpoint fields
- Cisco fields
- Cloud provider metadata fields
- Coredns fields
- Crowdstrike fields
- Cyber-Ark fields
- CylanceProtect fields
- Docker fields
- ECS fields
- Elasticsearch fields
- Envoyproxy fields
- Big-IP Access Policy Manager fields
- Fortinet fields
- Google Cloud Platform (GCP) fields
- google_workspace fields
- gsuite fields
- HAProxy fields
- Host fields
- ibmmq fields
- Icinga fields
- IIS fields
- Imperva SecureSphere fields
- Infoblox NIOS fields
- iptables fields
- Jolokia Discovery autodiscover provider fields
- Juniper JUNOS fields
- Kafka fields
- kibana fields
- Kubernetes fields
- Log file content fields
- logstash fields
- Microsoft fields
- MISP fields
- mongodb fields
- mssql fields
- MySQL fields
- MySQL Enterprise fields
- NATS fields
- NetFlow fields
- Arbor Peakflow SP fields
- Nginx fields
- Office 365 fields
- Okta fields
- Oracle fields
- Osquery fields
- panw fields
- PostgreSQL fields
- Process fields
- Proofpoint Email Security fields
- RabbitMQ fields
- Radware DefensePro fields
- Redis fields
- s3 fields
- Google Santa fields
- Snort/Sourcefire fields
- Snyk fields
- Sonicwall-FW fields
- sophos fields
- Squid fields
- Suricata fields
- System fields
- Apache Tomcat fields
- Traefik fields
- Zeek fields
- Zoom fields
- Zscaler NSS fields
- Monitor
- Secure
- Troubleshoot
- Get help
- Debug
- Common problems
- Can’t read log files from network volumes
- Filebeat isn’t collecting lines from a file
- Too many open file handlers
- Registry file is too large
- Inode reuse causes Filebeat to skip lines
- Log rotation results in lost or duplicate events
- Open file handlers cause issues with Windows file rotation
- Filebeat is using too much CPU
- Dashboard in Kibana is breaking up data fields incorrectly
- Fields are not indexed or usable in Kibana visualizations
- Filebeat isn’t shipping the last line of a file
- Filebeat keeps open file handlers of deleted files for a long time
- Filebeat uses too much bandwidth
- Error loading config file
- Found unexpected or unknown characters
- Logstash connection doesn’t work
- Publishing to Logstash fails with "connection reset by peer" message
- @metadata is missing in Logstash
- Not sure whether to use Logstash or Beats
- SSL client fails to connect to Logstash
- Monitoring UI shows fewer Beats than expected
- Dashboard could not locate the index-pattern
- Contribute to Beats
MySQL fields
editMySQL fields
editModule for parsing the MySQL log files.
mysql
editFields from the MySQL log files.
-
mysql.thread_id
-
The connection or thread ID for the query.
type: long
error
editContains fields from the MySQL error logs.
-
mysql.error.thread_id
-
type: alias
alias to: mysql.thread_id
-
mysql.error.level
-
type: alias
alias to: log.level
-
mysql.error.message
-
type: alias
alias to: message
slowlog
editContains fields from the MySQL slow logs.
-
mysql.slowlog.lock_time.sec
-
The amount of time the query waited for the lock to be available. The value is in seconds, as a floating point number.
type: float
-
mysql.slowlog.rows_sent
-
The number of rows returned by the query.
type: long
-
mysql.slowlog.rows_examined
-
The number of rows scanned by the query.
type: long
-
mysql.slowlog.rows_affected
-
The number of rows modified by the query.
type: long
-
mysql.slowlog.bytes_sent
-
The number of bytes sent to client.
type: long
format: bytes
-
mysql.slowlog.bytes_received
-
The number of bytes received from client.
type: long
format: bytes
-
mysql.slowlog.query
-
The slow query.
-
mysql.slowlog.id
-
type: alias
alias to: mysql.thread_id
-
mysql.slowlog.schema
-
The schema where the slow query was executed.
type: keyword
-
mysql.slowlog.current_user
-
Current authenticated user, used to determine access privileges. Can differ from the value for user.
type: keyword
-
mysql.slowlog.last_errno
-
Last SQL error seen.
type: keyword
-
mysql.slowlog.killed
-
Code of the reason if the query was killed.
type: keyword
-
mysql.slowlog.query_cache_hit
-
Whether the query cache was hit.
type: boolean
-
mysql.slowlog.tmp_table
-
Whether a temporary table was used to resolve the query.
type: boolean
-
mysql.slowlog.tmp_table_on_disk
-
Whether the query needed temporary tables on disk.
type: boolean
-
mysql.slowlog.tmp_tables
-
Number of temporary tables created for this query
type: long
-
mysql.slowlog.tmp_disk_tables
-
Number of temporary tables created on disk for this query.
type: long
-
mysql.slowlog.tmp_table_sizes
-
Size of temporary tables created for this query.
type: long
format: bytes
-
mysql.slowlog.filesort
-
Whether filesort optimization was used.
type: boolean
-
mysql.slowlog.filesort_on_disk
-
Whether filesort optimization was used and it needed temporary tables on disk.
type: boolean
-
mysql.slowlog.priority_queue
-
Whether a priority queue was used for filesort.
type: boolean
-
mysql.slowlog.full_scan
-
Whether a full table scan was needed for the slow query.
type: boolean
-
mysql.slowlog.full_join
-
Whether a full join was needed for the slow query (no indexes were used for joins).
type: boolean
-
mysql.slowlog.merge_passes
-
Number of merge passes executed for the query.
type: long
-
mysql.slowlog.sort_merge_passes
-
Number of merge passes that the sort algorithm has had to do.
type: long
-
mysql.slowlog.sort_range_count
-
Number of sorts that were done using ranges.
type: long
-
mysql.slowlog.sort_rows
-
Number of sorted rows.
type: long
-
mysql.slowlog.sort_scan_count
-
Number of sorts that were done by scanning the table.
type: long
-
mysql.slowlog.log_slow_rate_type
-
Type of slow log rate limit, it can be
session
if the rate limit is applied per session, orquery
if it applies per query.type: keyword
-
mysql.slowlog.log_slow_rate_limit
-
Slow log rate limit, a value of 100 means that one in a hundred queries or sessions are being logged.
type: keyword
-
mysql.slowlog.read_first
-
The number of times the first entry in an index was read.
type: long
-
mysql.slowlog.read_last
-
The number of times the last key in an index was read.
type: long
-
mysql.slowlog.read_key
-
The number of requests to read a row based on a key.
type: long
-
mysql.slowlog.read_next
-
The number of requests to read the next row in key order.
type: long
-
mysql.slowlog.read_prev
-
The number of requests to read the previous row in key order.
type: long
-
mysql.slowlog.read_rnd
-
The number of requests to read a row based on a fixed position.
type: long
-
mysql.slowlog.read_rnd_next
-
The number of requests to read the next row in the data file.
type: long
innodb
editContains fields relative to InnoDB engine
-
mysql.slowlog.innodb.trx_id
-
Transaction ID
type: keyword
-
mysql.slowlog.innodb.io_r_ops
-
Number of page read operations.
type: long
-
mysql.slowlog.innodb.io_r_bytes
-
Bytes read during page read operations.
type: long
format: bytes
-
mysql.slowlog.innodb.io_r_wait.sec
-
How long it took to read all needed data from storage.
type: long
-
mysql.slowlog.innodb.rec_lock_wait.sec
-
How long the query waited for locks.
type: long
-
mysql.slowlog.innodb.queue_wait.sec
-
How long the query waited to enter the InnoDB queue and to be executed once in the queue.
type: long
-
mysql.slowlog.innodb.pages_distinct
-
Approximated count of pages accessed to execute the query.
type: long
-
mysql.slowlog.user
-
type: alias
alias to: user.name
-
mysql.slowlog.host
-
type: alias
alias to: source.domain
-
mysql.slowlog.ip
-
type: alias
alias to: source.ip