Google Cloud Platform billing metricset
editGoogle Cloud Platform billing metricset
editThis functionality is in beta and is subject to change. The design and code is less mature than official GA features and is being provided as-is with no warranties. Beta features are not subject to the support SLA of official GA features.
billing metricset is designed for collecting billing metrics from Google Cloud
BigQuery daily cost detail table. BigQuery is a fully-managed, serverless data
warehouse.
Cloud Billing export to BigQuery enables you to export detailed Google Cloud
billing data (such as usage, cost estimates, and pricing data) automatically
throughout the day to a BigQuery dataset that you specify. Then you can access
your Cloud Billing data from BigQuery for detailed analysis using Metricbeat.
Please see export
cloud billing data to BigQuery for more details on how to export billing data.
In BigQuery dataset, detailed Google Cloud daily cost data is loaded into a data
table named gcp_billing_export_v1_<BILLING_ACCOUNT_ID>. There is a defined
schema for Google Cloud daily cost data that is exported to BigQuery. Please see
daily cost detail data schema for more details.
Metricset-specific configuration notes
edit- dataset_id: (Required) Dataset ID that points to the top-level container which contains the actual billing tables.
-
table_pattern: (Optional) Daily cost detail billing table name prefix.
Default to
gcp_billing_export_v1. -
cost_type: (Optional) The type of cost this line item represents: regular,
tax, adjustment, or rounding error. Default to
regular.
Configuration example
edit- module: gcp
metricsets:
- billing
period: 24h
project_id: "your project id"
credentials_file_path: "your JSON credentials file path"
dataset_id: "dataset id"
table_pattern: "table pattern"
cost_type: "regular"
For a description of each field in the metricset, see the exported fields section.
Here is an example document generated by this metricset:
{
"@timestamp": "2017-10-12T08:05:34.853Z",
"cloud.account.id": "01475F-5B1080-1137E7",
"cloud.project.id": "elastic-bi",
"cloud.project.name": "elastic-containerlib-prod",
"cloud.provider": "gcp",
"event": {
"dataset": "gcp.billing",
"duration": 115000,
"module": "gcp"
},
"gcp": {
"billing": {
"billing_account_id": "01475F-5B1080-1137E7",
"cost_type": "regular",
"invoice_month": "202106",
"project_id": "containerlib-prod-12763",
"project_name": "elastic-containerlib-prod",
"total": 4717.170681
}
},
"metricset": {
"name": "billing",
"period": 10000
},
"service": {
"type": "gcp"
}
}