Configure Google Functions
editConfigure Google Functions
editThis functionality is in beta and is subject to change. The design and code is less mature than official GA features and is being provided as-is with no warranties. Beta features are not subject to the support SLA of official GA features.
Functionbeat runs as a Google Function on Google Cloud Platform (GCP).
Before deploying Functionbeat, you need to configure one or more functions and specify details about the services that will trigger the functions.
You configure the functions in the the functionbeat.yml
configuration file.
When you’re done, you can deploy the functions
to your serverless environment.
The following example configures two functions: pubsub
and storage
. The
pubsub
function collects log events from Google
Pub/Sub. The storage
function collects log events from
Google Cloud Storage. Both functions in the
example forward the events to Elasticsearch.
functionbeat.provider.gcp.location_id: "europe-west2" functionbeat.provider.gcp.project_id: "my-project-123456" functionbeat.provider.gcp.storage_name: "functionbeat-deploy" functionbeat.provider.gcp.functions: - name: pubsub enabled: true type: pubsub description: "Google Cloud Function for Pub/Sub" trigger: resource: "projects/_/pubsub/myPubSub" #service: "pubsub.googleapis.com" - name: storage enabled: true type: storage description: "Google Cloud Function for Cloud Storage" trigger: resource: "projects/my-project/buckets/my-storage" event_type: "google.storage.object.finalize" cloud.id: "MyESDeployment:SomeLongString==" cloud.auth: "elastic:mypassword"
Configuration options
editSpecify the following options to configure the functions that you want to deploy to Google Cloud Platform (GCP).
If you change the configuration after deploying the function, use
the update
command to update your deployment.
provider.gcp.location_id
editThe region where your GCP project is located.
provider.gcp.project_id
editThe ID of the GCP project where the function artifacts will be deployed. See the Google Cloud Function documentation to verify that Cloud Functions are supported in the region you specify.
provider.gcp.storage_name
editThe name of the Google Cloud storage bucket where the function artifacts will be
deployed. If the bucket doesn’t exist, it will be created, if you have the
correct project permissions (storage.objects.create
).
functionbeat.provider.gcp.functions
editA list of functions that are available for deployment.
name
editA unique name for the Google function.
type
editThe type of GCP service to monitor. For this release, the supported types are:
|
Collect log events from Google Pub/Sub. |
|
Collect log events from Google Cloud storage buckets. |
description
editA description of the function. This description is useful when you are running multiple functions and need more context about how each function is used.
memory_size
editThe maximum amount of memory to allocate for this function.
The default is 256MB
.
timeout
editThe execution timeout in seconds. If the function does not finish in time,
it is considered failed and terminated. The default is 60s
. Increase this
value if you see timeout messages is the Google Stackdriver logs.
service_account_email
editThe email of the service account that the function will assume as its identity. The default is {projectid}@appspot.gserviceaccount.com.email.
labels
editOne or more labels to apply to the function. A label is a key-value pair that helps you organize your Google Cloud resources.
vpc_connector
editA VPC connector that the function can connect to when sending requests to resources in your VPC network.
Use the format projects/*/locations/*/connectors/*
or a fully qualified
URI.
maximum_instances
editThe maximum instances that can be running at the same time. The default is unlimited.
trigger
editThe trigger that will cause the function to execute.
-
If
type
ispubsub
, specify the name of the Pub/Sub topic to watch for messages. -
If
type
isstorage
, specify the Cloud Storage bucket to watch for object events. Forevent_type
, specify the type of object event that will trigger the function. See the Google Cloud docs for a list of available event types.
keep_null
editIf true
, fields with null values will be published in the output document. By
default, keep_null
is false
.
fields
editOptional fields that you can specify to add additional information to the output. Fields can be scalar values, arrays, dictionaries, or any nested combination of these.
processors
editDefine custom processors for this function. For example, you can specify a dissect processor to tokenize a string:
processors: - dissect: tokenizer: "%{key1} %{key2}"