Google Cloud Platform billing metricset
editGoogle Cloud Platform billing metricset
editbilling
metricset is designed for collecting billing metrics from Google Cloud
BigQuery daily cost detail table. BigQuery is a fully-managed, serverless data
warehouse.
Cloud Billing export to BigQuery enables you to export standard and detailed Google Cloud
billing data (such as usage, cost estimates, and pricing data) automatically
throughout the day to a BigQuery dataset that you specify. Then you can access
your Cloud Billing data from BigQuery for detailed analysis using Metricbeat.
Please see export
cloud billing data to BigQuery for more details on how to export billing data.
In BigQuery, Google Cloud daily cost data is categorized into two formats: standard and detailed. Each format is stored within a designated dataset and follows a structured schema for precise cost analysis. For a comprehensive understanding of these formats, consult the standard and detailed data schema documentation.
For standard usage cost data, set the table pattern format to
gcp_billing_export_v1
. This table pattern is set as the default when no other
is specified.
For detailed usage cost data, set the table pattern to gcp_billing_export_resource_v1
.
Detailed tables include the standard fields and additional fields, such as
effective_price
, enabling a more granular view of expenses.
Metricset-specific configuration notes
edit- dataset_id: (Required) Dataset ID that points to the top-level container which contains the actual billing tables.
-
table_pattern: (Optional) Daily cost detail billing table name prefix.
Default to
gcp_billing_export_v1
. -
cost_type: (Optional) The type of cost this line item represents: regular,
tax, adjustment, or rounding error. Default to
regular
.
Configuration example
edit- module: gcp metricsets: - billing period: 24h project_id: "your project id" credentials_file_path: "your JSON credentials file path" dataset_id: "dataset id" table_pattern: "table pattern" cost_type: "regular"
For a description of each field in the metricset, see the exported fields section.
Here is an example document generated by this metricset:
{ "@timestamp": "2017-10-12T08:05:34.853Z", "cloud.account.id": "01475F-5B1080-1137E7", "cloud.project.id": "elastic-bi", "cloud.project.name": "elastic-containerlib-prod", "cloud.provider": "gcp", "event": { "dataset": "gcp.billing", "duration": 115000, "module": "gcp" }, "gcp": { "billing": { "billing_account_id": "01475F-5B1080-1137E7", "cost_type": "regular", "invoice_month": "202106", "project_id": "containerlib-prod-12763", "project_name": "elastic-containerlib-prod", "total": 4717.170681, "sku_id": "0D56-2F80-52A5", "service_id": "6F81-5844-456A", "sku_description": "Network Inter Region Ingress from Jakarta to Americas", "service_description": "Compute Engine", "effective_price": 0.00292353, "tags": [ { "key": "size", "value": "standard" } ] } }, "metricset": { "name": "billing", "period": 10000 }, "service": { "type": "gcp" } }