- Filebeat Reference: other versions:
- Overview
- Getting Started With Filebeat
- Step 1: Install Filebeat
- Step 2: Configure Filebeat
- Step 3: Configure Filebeat to use Logstash
- Step 4: Load the index template in Elasticsearch
- Step 5: Set up the Kibana dashboards
- Step 6: Start Filebeat
- Step 7: View the sample Kibana dashboards
- Quick start: modules for common log formats
- Repositories for APT and YUM
- Setting up and running Filebeat
- Upgrading Filebeat
- How Filebeat works
- Configuring Filebeat
- Specify which modules to run
- Configure inputs
- Manage multiline messages
- Specify general settings
- Load external configuration files
- Configure the internal queue
- Configure the output
- Load balance the output hosts
- Specify SSL settings
- Filter and enhance the exported data
- Parse data by using ingest node
- Set up project paths
- Set up the Kibana endpoint
- Load the Kibana dashboards
- Load the Elasticsearch index template
- Configure logging
- Use environment variables in the configuration
- Autodiscover
- YAML tips and gotchas
- Regular expression support
- filebeat.reference.yml
- Modules
- Exported fields
- Apache2 fields
- Auditd fields
- Beat fields
- Cloud provider metadata fields
- Docker fields
- Host fields
- Icinga fields
- IIS fields
- Kafka fields
- Kubernetes fields
- Log file content fields
- logstash fields
- mongodb fields
- MySQL fields
- Nginx fields
- Osquery fields
- PostgreSQL fields
- Redis fields
- System fields
- Traefik fields
- Monitoring Filebeat
- Securing Filebeat
- Troubleshooting
- Migrating from Logstash Forwarder to Filebeat
- Contributing to Beats
Add cloud metadata
editAdd cloud metadata
editThe add_cloud_metadata processor enriches each event with instance metadata
from the machine’s hosting provider. At startup it will detect the hosting
provider and cache the instance metadata.
The following cloud providers are supported:
- Amazon Elastic Compute Cloud (EC2)
- Digital Ocean
- Google Compute Engine (GCE)
- Tencent Cloud (QCloud)
- Alibaba Cloud (ECS)
- Azure Virtual Machine
The simple configuration below enables the processor.
processors: - add_cloud_metadata: ~
The add_cloud_metadata processor has one optional configuration setting named
timeout that specifies the maximum amount of time to wait for a successful
response when detecting the hosting provider. The default timeout value is
3s.
If a timeout occurs then no instance metadata will be added to the events. This makes it possible to enable this processor for all your deployments (in the cloud or on-premise).
The metadata that is added to events varies by hosting provider. Below are examples for each of the supported providers.
EC2
{ "meta": { "cloud": { "availability_zone": "us-east-1c", "instance_id": "i-4e123456", "machine_type": "t2.medium", "provider": "ec2", "region": "us-east-1" } } }
Digital Ocean
{ "meta": { "cloud": { "instance_id": "1234567", "provider": "digitalocean", "region": "nyc2" } } }
GCE
{ "meta": { "cloud": { "availability_zone": "projects/1234567890/zones/us-east1-b", "instance_id": "1234556778987654321", "machine_type": "projects/1234567890/machineTypes/f1-micro", "project_id": "my-dev", "provider": "gce" } } }
Tencent Cloud
{ "meta": { "cloud": { "availability_zone": "gz-azone2", "instance_id": "ins-qcloudv5", "provider": "qcloud", "region": "china-south-gz" } } }
Alibaba Cloud
This metadata is only available when VPC is selected as the network type of the ECS instance.
{ "meta": { "cloud": { "availability_zone": "cn-shenzhen", "instance_id": "i-wz9g2hqiikg0aliyun2b", "provider": "ecs", "region": "cn-shenzhen-a" } } }
Azure Virtual Machine
{ "meta": { "cloud": { "provider": "az", "instance_id": "04ab04c3-63de-4709-a9f9-9ab8c0411d5e", "instance_name": "test-az-vm", "machine_type": "Standard_D3_v2", "region": "eastus2" } } }