- Metricbeat Reference: other versions:
- Overview
- Getting started with Metricbeat
- Setting up and running Metricbeat
- Upgrading Metricbeat
- How Metricbeat works
- Configuring Metricbeat
- Specify which modules to run
- Specify general settings
- Load external configuration files
- Configure the internal queue
- Configure the output
- Configure index lifecycle management
- Specify SSL settings
- Filter and enhance the exported data
- Define processors
- Add cloud metadata
- Add Docker metadata
- Add fields
- Add Host metadata
- Add Kubernetes metadata
- Add labels
- Add the local time zone
- Add Observer metadata
- Add process metadata
- Add tags
- Community ID Network Flow Hash
- Convert
- Decode Base64 fields
- Decode JSON fields
- Decompress gzip fields
- Dissect strings
- DNS Reverse Lookup
- Drop events
- Drop fields from events
- Extract array
- Keep fields from events
- Registered Domain
- Rename fields from events
- Parse data by using ingest node
- Enrich events with geoIP information
- Configure project paths
- Configure the Kibana endpoint
- Load the Kibana dashboards
- Load the Elasticsearch index template
- Configure logging
- Use environment variables in the configuration
- Autodiscover
- YAML tips and gotchas
- Regular expression support
- HTTP Endpoint
- metricbeat.reference.yml
- Beats central management
- Modules
- Aerospike module
- Apache module
- App Search module
- aws module
- azure module
- Beat module
- Ceph module
- CockroachDB module
- consul module
- coredns module
- Couchbase module
- couchdb module
- Docker module
- Dropwizard module
- Elasticsearch module
- Elasticsearch ccr metricset
- Elasticsearch cluster_stats metricset
- Elasticsearch enrich metricset
- Elasticsearch index metricset
- Elasticsearch index_recovery metricset
- Elasticsearch index_summary metricset
- Elasticsearch ml_job metricset
- Elasticsearch node metricset
- Elasticsearch node_stats metricset
- Elasticsearch pending_tasks metricset
- Elasticsearch shard metricset
- envoyproxy module
- Etcd module
- Golang module
- Graphite module
- HAProxy module
- HTTP module
- Jolokia module
- Kafka module
- Kibana module
- Kubernetes module
- Kubernetes apiserver metricset
- Kubernetes container metricset
- Kubernetes controllermanager metricset
- Kubernetes event metricset
- Kubernetes node metricset
- Kubernetes pod metricset
- Kubernetes proxy metricset
- Kubernetes scheduler metricset
- Kubernetes state_container metricset
- Kubernetes state_cronjob metricset
- Kubernetes state_deployment metricset
- Kubernetes state_node metricset
- Kubernetes state_pod metricset
- Kubernetes state_replicaset metricset
- Kubernetes state_resourcequota metricset
- Kubernetes state_statefulset metricset
- Kubernetes system metricset
- Kubernetes volume metricset
- kvm module
- Logstash module
- Memcached module
- MongoDB module
- MSSQL module
- Munin module
- MySQL module
- Nats module
- Nginx module
- Oracle module
- PHP_FPM module
- PostgreSQL module
- Prometheus module
- RabbitMQ module
- Redis module
- Statsd module
- System module
- System core metricset
- System cpu metricset
- System diskio metricset
- System entropy metricset
- System filesystem metricset
- System fsstat metricset
- System load metricset
- System memory metricset
- System network metricset
- System process metricset
- System process_summary metricset
- System raid metricset
- System socket metricset
- System socket_summary metricset
- System uptime metricset
- Tomcat module
- traefik module
- uwsgi module
- vSphere module
- Windows module
- ZooKeeper module
- Exported fields
- Aerospike fields
- Apache fields
- App Search fields
- aws fields
- azure fields
- Beat fields
- Beat fields
- Ceph fields
- Cloud provider metadata fields
- CockroachDB fields
- Common fields
- consul fields
- coredns fields
- Couchbase fields
- couchdb fields
- Docker fields
- Docker fields
- Dropwizard fields
- ECS fields
- Elasticsearch fields
- envoyproxy fields
- Etcd fields
- Golang fields
- Graphite fields
- HAProxy fields
- Host fields
- HTTP fields
- Jolokia fields
- Jolokia Discovery autodiscover provider fields
- Kafka fields
- Kibana fields
- Kubernetes fields
- Kubernetes fields
- kvm fields
- Logstash fields
- Memcached fields
- MongoDB fields
- MSSQL fields
- Munin fields
- MySQL fields
- Nats fields
- Nginx fields
- Oracle fields
- PHP_FPM fields
- PostgreSQL fields
- Process fields
- Prometheus fields
- RabbitMQ fields
- Redis fields
- Statsd fields
- System fields
- Tomcat fields
- traefik fields
- uwsgi fields
- vSphere fields
- Windows fields
- ZooKeeper fields
- Monitoring Metricbeat
- Securing Metricbeat
- Troubleshooting
- Get help
- Debug
- Common problems
- "open /compat/linux/proc: no such file or directory" error on FreeBSD
- Metricbeat collects system metrics for interfaces you didn’t configure
- Metricbeat uses too much bandwidth
- Error loading config file
- Found unexpected or unknown characters
- Logstash connection doesn’t work
- @metadata is missing in Logstash
- Not sure whether to use Logstash or Beats
- SSL client fails to connect to Logstash
- Monitoring UI shows fewer Beats than expected
- Contributing to Beats
azure monitor metricset
editazure monitor metricset
editThis functionality is in beta and is subject to change. The design and code is less mature than official GA features and is being provided as-is with no warranties. Beta features are not subject to the support SLA of official GA features.
This is the monitor metricset of the module azure.
This metricset allows users to retrieve metrics from specified resources. Added filters can apply here as the interval of retrieving these metrics, metric names, aggregation list, namespaces and metric dimensions.
Metricset-specific configuration notes
edit-
refresh_list_interval
-
Resources will be retrieved at each fetch call (
period
interval), this means a number of Azure REST calls will be executed each time. This will be helpful if the azure users will be adding/removing resources that could match the configuration options so they will not added/removed to the list. To reduce on the number of API calls we are executing to retrieve the resources each time, users can configure this setting and make sure the list or resources will not be refreshed as often. This is also beneficial for performance and rate/ cost reasons (https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-manager-request-limits). -
resources
- This will contain all options for identifying resources and configuring the desired metrics
Config options to identify resources
edit-
resource_id
- ([]string) The fully qualified ID’s of the resource, including the resource name and resource type. Has the format /subscriptions/{guid}/resourceGroups/{resource-group-name}/providers/{resource-provider-namespace}/{resource-type}/{resource-name}. Should return a list of resources.
But users might have large number of resources they would like to gather metrics from so, in order to reduce the verbosity users will have the options of entering a resource group and filtering by resource type, or type in a “resource_query” where the user can filter resources from their entire subscription. Source for the resource API’s: https://docs.microsoft.com/en-us/rest/api/resources/resources/list https://docs.microsoft.com/en-us/rest/api/resources/resources/listbyresourcegroup
-
resource_group
- ([]string) Using the resource_type configuration option as a filter is required for the resource groups entered. This option should return a list resources we want to apply our metric configuration options on.
-
resource_type
- (string) As mentioned above this will be a filter option for the resource group api, will check for all resources under the specified group that are the type under this configuration.
-
resource_query
- (string) Should contain a filter entered by the user, the output will be a list of resources
Resource metric configurations
edit-
metrics
- List of different metrics to collect information
-
namespace
- (string) Namespaces are a way to categorize or group similar metrics together. By using namespaces, users can achieve isolation between groups of metrics that might collect different insights or performance indicators.
-
name
- ([]string) Name of the metrics that’s being reported. Usually, the name is descriptive enough to help identify what’s measured. A list of metric names can be entered as well
-
aggregations
- ([]string) List of supported aggregations. Azure Monitor stores all metrics at one-minute granularity intervals. During a given minute, a metric might need to be sampled several times or it might need to be measured for many discrete events. To limit the number of raw values we have to emit and pay for in Azure Monitor, they will locally pre-aggregate and emit the values: Minimum: The minimum observed value from all the samples and measurements during the minute. Maximum: The maximum observed value from all the samples and measurements during the minute. Sum: The summation of all the observed values from all the samples and measurements during the minute. Count: The number of samples and measurements taken during the minute. Total: The total number of all the observed values from all the samples and measurements during the minute.
-
dimensions
- List of metric dimensions. Dimensions are optional, not all metrics may have dimensions. A custom metric can have up to 10 dimensions. A dimension is a key or value pair that helps describe additional characteristics about the metric being collected. By using the additional characteristics, you can collect more information about the metric, which allows for deeper insights. By using this key, you can filter the metric to see how much memory specific processes use or to identify the top five processes by memory usage. Metrics with dimensions are exported as flattened single dimensional metrics, aggregated across dimension values.
-
name
- Dimension key
-
value
- Dimension value. (Users can select * to return metric values for each dimension)
Users can select the options to retrieve all metrics from a specific namespace using the following:
metrics: - name: ["*"] namespace: "Microsoft.Storage/storageAccounts"
If no aggregations are entered under a metric level the metricset will retrieve the primary aggregation assigned for this metric.
This is a default metricset. If the host module is unconfigured, this metricset is enabled by default.
Fields
editFor a description of each field in the metricset, see the exported fields section.
Here is an example document generated by this metricset:
{ "@timestamp":"2019-08-20T13:52:00.000Z", "service":{ "type":"azure" }, "azure":{ "subscriptionID":"123456-qwer-1234-5678-12345678", "metrics":{ "percentage_cpu":{ "min":1.86, "max":2.8, "avg":2.535 } }, "resource":{ "name":"vmwindows", "type":"Microsoft.Compute/virtualMachines" }, "namespace":"Microsoft.Compute/virtualMachines" }, "cloud":{ "provider":"azure", "region":"westeurope" }, "metricset":{ "name":"monitor" }, "event":{ "module":"azure", "duration":40189255100, "dataset":"azure.monitor" }, "ecs":{ "version":"1.0.1" }, "host":{ "name":"DESKTOP-RFOOE09", "architecture":"x86_64", "os":{ "family":"windows", "name":"Windows 10 Pro", "kernel":"10.0.17134.950 (WinBuild.160101.0800)", "build":"17134.950", "platform":"windows", "version":"10.0" }, "id":"123456-qwer-1234-5678-12345678", "hostname":"DESKTOP-RFOOE09" }, "agent":{ "ephemeral_id":"123456-qwer-1234-5678-12345678", "hostname":"DESKTOP-RFOOE09", "id":"123456-qwer-1234-5678-12345678", "version":"8.0.0", "type":"metricbeat" } }
On this page
ElasticON events are back!
Learn about the Elastic Search AI Platform from the experts at our live events.
Register now