- Filebeat Reference: other versions:
- Filebeat overview
- Quick start: installation and configuration
- Set up and run
- Upgrade
- How Filebeat works
- Configure
- Inputs
- Modules
- General settings
- Project paths
- Config file loading
- Output
- Kerberos
- SSL
- Index lifecycle management (ILM)
- Elasticsearch index template
- Kibana endpoint
- Kibana dashboards
- Processors
- Define processors
- add_cloud_metadata
- add_cloudfoundry_metadata
- add_docker_metadata
- add_fields
- add_host_metadata
- add_id
- add_kubernetes_metadata
- add_labels
- add_locale
- add_network_direction
- add_observer_metadata
- add_process_metadata
- add_tags
- community_id
- convert
- copy_fields
- decode_base64_field
- decode_cef
- decode_csv_fields
- decode_json_fields
- decompress_gzip_field
- detect_mime_type
- dissect
- dns
- drop_event
- drop_fields
- extract_array
- fingerprint
- include_fields
- rate_limit
- registered_domain
- rename
- script
- timestamp
- translate_sid
- truncate_fields
- urldecode
- Autodiscover
- Internal queue
- Load balancing
- Logging
- HTTP endpoint
- Regular expression support
- Instrumentation
- filebeat.reference.yml
- How to guides
- Override configuration settings
- Load the Elasticsearch index template
- Change the index name
- Load Kibana dashboards
- Load ingest pipelines
- Enrich events with geoIP information
- Deduplicate data
- Parse data by using ingest node
- Use environment variables in the configuration
- Avoid YAML formatting problems
- Beats central management
- Modules
- Modules overview
- ActiveMQ module
- Apache module
- Auditd module
- AWS module
- Azure module
- Barracuda module
- Bluecoat module
- CEF module
- Check Point module
- Cisco module
- CoreDNS module
- Crowdstrike module
- Cyberark module
- Cylance module
- Elasticsearch module
- Envoyproxy Module
- F5 module
- Fortinet module
- Google Cloud module
- Google Workspace module
- GSuite module
- haproxy module
- IBM MQ module
- Icinga module
- IIS module
- Imperva module
- Infoblox module
- Iptables module
- Juniper module
- Kafka module
- Kibana module
- Logstash module
- Microsoft module
- MISP module
- MongoDB module
- MSSQL module
- MySQL module
- MySQL Enterprise module
- nats module
- NetFlow module
- Netscout module
- Nginx module
- Office 365 module
- Okta module
- Oracle module
- Osquery module
- Palo Alto Networks module
- PostgreSQL module
- Proofpoint module
- RabbitMQ module
- Radware module
- Redis module
- Santa module
- Snort module
- Snyk module
- Sonicwall module
- Sophos module
- Squid module
- Suricata module
- System module
- Tomcat module
- Traefik module
- Zeek (Bro) Module
- Zoom module
- Zscaler module
- Exported fields
- ActiveMQ fields
- Apache fields
- Auditd fields
- AWS fields
- aws-cloudwatch fields
- Azure fields
- Barracuda Web Application Firewall fields
- Beat fields
- Blue Coat Director fields
- Decode CEF processor fields fields
- CEF fields
- Checkpoint fields
- Cisco fields
- Cloud provider metadata fields
- Coredns fields
- Crowdstrike fields
- Cyber-Ark fields
- CylanceProtect fields
- Docker fields
- ECS fields
- Elasticsearch fields
- Envoyproxy fields
- Big-IP Access Policy Manager fields
- Fortinet fields
- Google Cloud Platform (GCP) fields
- google_workspace fields
- gsuite fields
- HAProxy fields
- Host fields
- ibmmq fields
- Icinga fields
- IIS fields
- Imperva SecureSphere fields
- Infoblox NIOS fields
- iptables fields
- Jolokia Discovery autodiscover provider fields
- Juniper JUNOS fields
- Kafka fields
- kibana fields
- Kubernetes fields
- Log file content fields
- logstash fields
- Microsoft fields
- MISP fields
- mongodb fields
- mssql fields
- MySQL fields
- MySQL Enterprise fields
- NATS fields
- NetFlow fields
- Arbor Peakflow SP fields
- Nginx fields
- Office 365 fields
- Okta fields
- Oracle fields
- Osquery fields
- panw fields
- PostgreSQL fields
- Process fields
- Proofpoint Email Security fields
- RabbitMQ fields
- Radware DefensePro fields
- Redis fields
- s3 fields
- Google Santa fields
- Snort/Sourcefire fields
- Snyk fields
- Sonicwall-FW fields
- sophos fields
- Squid fields
- Suricata fields
- System fields
- Apache Tomcat fields
- Traefik fields
- Zeek fields
- Zoom fields
- Zscaler NSS fields
- Monitor
- Secure
- Troubleshoot
- Get help
- Debug
- Common problems
- Can’t read log files from network volumes
- Filebeat isn’t collecting lines from a file
- Too many open file handlers
- Registry file is too large
- Inode reuse causes Filebeat to skip lines
- Log rotation results in lost or duplicate events
- Open file handlers cause issues with Windows file rotation
- Filebeat is using too much CPU
- Dashboard in Kibana is breaking up data fields incorrectly
- Fields are not indexed or usable in Kibana visualizations
- Filebeat isn’t shipping the last line of a file
- Filebeat keeps open file handlers of deleted files for a long time
- Filebeat uses too much bandwidth
- Error loading config file
- Found unexpected or unknown characters
- Logstash connection doesn’t work
- Publishing to Logstash fails with "connection reset by peer" message
- @metadata is missing in Logstash
- Not sure whether to use Logstash or Beats
- SSL client fails to connect to Logstash
- Monitoring UI shows fewer Beats than expected
- Dashboard could not locate the index-pattern
- Contribute to Beats
Elasticsearch fields
editElasticsearch fields
editelasticsearch Module
elasticsearch
edit-
elasticsearch.component
-
Elasticsearch component from where the log event originated
type: keyword
example: o.e.c.m.MetaDataCreateIndexService
-
elasticsearch.cluster.uuid
-
UUID of the cluster
type: keyword
example: GmvrbHlNTiSVYiPf8kxg9g
-
elasticsearch.cluster.name
-
Name of the cluster
type: keyword
example: docker-cluster
-
elasticsearch.node.id
-
ID of the node
type: keyword
example: DSiWcTyeThWtUXLB9J0BMw
-
elasticsearch.node.name
-
Name of the node
type: keyword
example: vWNJsZ3
-
elasticsearch.index.name
-
Index name
type: keyword
example: filebeat-test-input
-
elasticsearch.index.id
-
Index id
type: keyword
example: aOGgDwbURfCV57AScqbCgw
-
elasticsearch.shard.id
-
Id of the shard
type: keyword
example: 0
audit
edit-
elasticsearch.audit.layer
-
The layer from which this event originated: rest, transport or ip_filter
type: keyword
example: rest
-
elasticsearch.audit.event_type
-
The type of event that occurred: anonymous_access_denied, authentication_failed, access_denied, access_granted, connection_granted, connection_denied, tampered_request, run_as_granted, run_as_denied
type: keyword
example: access_granted
-
elasticsearch.audit.origin.type
-
Where the request originated: rest (request originated from a REST API request), transport (request was received on the transport channel), local_node (the local node issued the request)
type: keyword
example: local_node
-
elasticsearch.audit.realm
-
The authentication realm the authentication was validated against
type: keyword
-
elasticsearch.audit.user.realm
-
The user’s authentication realm, if authenticated
type: keyword
-
elasticsearch.audit.user.roles
-
Roles to which the principal belongs
type: keyword
example: [kibana_admin, beats_admin]
-
elasticsearch.audit.action
-
The name of the action that was executed
type: keyword
example: cluster:monitor/main
-
elasticsearch.audit.url.params
-
REST URI parameters
example: {username=jacknich2}
-
elasticsearch.audit.indices
-
Indices accessed by action
type: keyword
example: [foo-2019.01.04, foo-2019.01.03, foo-2019.01.06]
-
elasticsearch.audit.request.id
-
Unique ID of request
type: keyword
example: WzL_kb6VSvOhAq0twPvHOQ
-
elasticsearch.audit.request.name
-
The type of request that was executed
type: keyword
example: ClearScrollRequest
-
elasticsearch.audit.request_body
-
type: alias
alias to: http.request.body.content
-
elasticsearch.audit.origin_address
-
type: alias
alias to: source.ip
-
elasticsearch.audit.uri
-
type: alias
alias to: url.original
-
elasticsearch.audit.principal
-
type: alias
alias to: user.name
-
elasticsearch.audit.message
-
type: text
deprecation
editgc
editGC fileset fields.
phase
editFields specific to GC phase.
-
elasticsearch.gc.phase.name
-
Name of the GC collection phase.
type: keyword
-
elasticsearch.gc.phase.duration_sec
-
Collection phase duration according to the Java virtual machine.
type: float
-
elasticsearch.gc.phase.scrub_symbol_table_time_sec
-
Pause time in seconds cleaning up symbol tables.
type: float
-
elasticsearch.gc.phase.scrub_string_table_time_sec
-
Pause time in seconds cleaning up string tables.
type: float
-
elasticsearch.gc.phase.weak_refs_processing_time_sec
-
Time spent processing weak references in seconds.
type: float
-
elasticsearch.gc.phase.parallel_rescan_time_sec
-
Time spent in seconds marking live objects while application is stopped.
type: float
-
elasticsearch.gc.phase.class_unload_time_sec
-
Time spent unloading unused classes in seconds.
type: float
cpu_time
editProcess CPU time spent performing collections.
-
elasticsearch.gc.phase.cpu_time.user_sec
-
CPU time spent outside the kernel.
type: float
-
elasticsearch.gc.phase.cpu_time.sys_sec
-
CPU time spent inside the kernel.
type: float
-
elasticsearch.gc.phase.cpu_time.real_sec
-
Total elapsed CPU time spent to complete the collection from start to finish.
type: float
-
elasticsearch.gc.jvm_runtime_sec
-
The time from JVM start up in seconds, as a floating point number.
type: float
-
elasticsearch.gc.threads_total_stop_time_sec
-
Garbage collection threads total stop time seconds.
type: float
-
elasticsearch.gc.stopping_threads_time_sec
-
Time took to stop threads seconds.
type: float
-
elasticsearch.gc.tags
-
GC logging tags.
type: keyword
heap
editHeap allocation and total size.
-
elasticsearch.gc.heap.size_kb
-
Total heap size in kilobytes.
type: integer
-
elasticsearch.gc.heap.used_kb
-
Used heap in kilobytes.
type: integer
old_gen
editOld generation occupancy and total size.
-
elasticsearch.gc.old_gen.size_kb
-
Total size of old generation in kilobytes.
type: integer
-
elasticsearch.gc.old_gen.used_kb
-
Old generation occupancy in kilobytes.
type: integer
young_gen
editYoung generation occupancy and total size.
-
elasticsearch.gc.young_gen.size_kb
-
Total size of young generation in kilobytes.
type: integer
-
elasticsearch.gc.young_gen.used_kb
-
Young generation occupancy in kilobytes.
type: integer
server
editServer log file
-
elasticsearch.server.stacktrace
-
Field is not indexed.
gc
editGC log
young
editYoung GC
-
elasticsearch.server.gc.young.one
-
type: long
example:
-
elasticsearch.server.gc.young.two
-
type: long
example:
-
elasticsearch.server.gc.overhead_seq
-
Sequence number
type: long
example: 3449992
-
elasticsearch.server.gc.collection_duration.ms
-
Time spent in GC, in milliseconds
type: float
example: 1600
-
elasticsearch.server.gc.observation_duration.ms
-
Total time over which collection was observed, in milliseconds
type: float
example: 1800
slowlog
editSlowlog events from Elasticsearch
-
elasticsearch.slowlog.logger
-
Logger name
type: keyword
example: index.search.slowlog.fetch
-
elasticsearch.slowlog.took
-
Time it took to execute the query
type: keyword
example: 300ms
-
elasticsearch.slowlog.types
-
Types
type: keyword
example:
-
elasticsearch.slowlog.stats
-
Stats groups
type: keyword
example: group1
-
elasticsearch.slowlog.search_type
-
Search type
type: keyword
example: QUERY_THEN_FETCH
-
elasticsearch.slowlog.source_query
-
Slow query
type: keyword
example: {"query":{"match_all":{"boost":1.0}}}
-
elasticsearch.slowlog.extra_source
-
Extra source information
type: keyword
example:
-
elasticsearch.slowlog.total_hits
-
Total hits
type: keyword
example: 42
-
elasticsearch.slowlog.total_shards
-
Total queried shards
type: keyword
example: 22
-
elasticsearch.slowlog.routing
-
Routing
type: keyword
example: s01HZ2QBk9jw4gtgaFtn
-
elasticsearch.slowlog.id
-
Id
type: keyword
example:
-
elasticsearch.slowlog.type
-
Type
type: keyword
example: doc
-
elasticsearch.slowlog.source
-
Source of document that was indexed
type: keyword