Google Cloud Platform dataproc metricset
editGoogle Cloud Platform dataproc metricset
editThis functionality is in beta and is subject to change. The design and code is less mature than official GA features and is being provided as-is with no warranties. Beta features are not subject to the support SLA of official GA features.
Dataproc metricset fetches metrics from Dataproc in Google Cloud Platform.
The dataproc metricset contains all metrics exported from the GCP Dataproc Monitoring API. The field names are aligned to Beats naming conventions with minor modifications to their GCP metrics name counterpart.
You can specify a single region to fetch metrics like us-central1. Be aware that GCP Storage does not use zones so us-central1-a will return nothing. If no region is specified, metrics are returned from all buckets.
For a description of each field in the metricset, see the exported fields section.
Here is an example document generated by this metricset:
{
"@timestamp": "2016-05-23T08:05:34.853Z",
"cloud": {
"account": {
"id": "elastic-apm"
},
"provider": "gcp"
},
"event": {
"dataset": "gcp.dataproc",
"duration": 115000,
"module": "gcp"
},
"gcp": {
"labels": {
"metrics": {
"storage_class": "MULTI_REGIONAL"
},
"resource": {
"bucket_name": "artifacts.elastic-apm.appspot.com",
"location": "us"
}
},
"dataproc": {
"cluster": {
"hdfs": {
"datanodes": {
"value": 15
}
}
}
}
},
"metricset": {
"name": "dataproc",
"period": 10000
},
"service": {
"type": "gcp"
}
}