Configure Google Functions

edit

This functionality is in beta and is subject to change. The design and code is less mature than official GA features and is being provided as-is with no warranties. Beta features are not subject to the support SLA of official GA features.

Functionbeat runs as a Google Function on Google Cloud Platform (GCP).

Before deploying Functionbeat, you need to configure one or more functions and specify details about the services that will trigger the functions.

You configure the functions in the the functionbeat.yml configuration file. When you’re done, you can deploy the functions to your serverless environment.

The following example configures two functions: pubsub and storage. The pubsub function collects log events from Google Pub/Sub. The storage function collects log events from Google Cloud Storage. Both functions in the example forward the events to Elasticsearch.

functionbeat.provider.gcp.location_id: "europe-west2"
functionbeat.provider.gcp.project_id: "my-project-123456"
functionbeat.provider.gcp.storage_name: "functionbeat-deploy"
functionbeat.provider.gcp.functions:
  - name: pubsub
    enabled: true
    type: pubsub
    description: "Google Cloud Function for Pub/Sub"
    trigger:
      resource: "projects/_/pubsub/myPubSub"
      #service: "pubsub.googleapis.com"
  - name: storage
    enabled: true
    type: storage
    description: "Google Cloud Function for Cloud Storage"
    trigger:
      resource: "projects/my-project/buckets/my-storage"
      event_type: "google.storage.object.finalize"

cloud.id: "MyESDeployment:SomeLongString=="
cloud.auth: "elastic:mypassword"

Configuration options

edit

Specify the following options to configure the functions that you want to deploy to Google Cloud Platform (GCP).

If you change the configuration after deploying the function, use the update command to update your deployment.

provider.gcp.location_id

edit

The region where your GCP project is located.

provider.gcp.project_id

edit

The ID of the GCP project where the function artifacts will be deployed. See the Google Cloud Function documentation to verify that Cloud Functions are supported in the region you specify.

provider.gcp.storage_name

edit

The name of the Google Cloud storage bucket where the function artifacts will be deployed. If the bucket doesn’t exist, it will be created, if you have the correct project permissions (storage.objects.create).

functionbeat.provider.gcp.functions

edit

A list of functions that are available for deployment.

name
edit

A unique name for the Google function.

type
edit

The type of GCP service to monitor. For this release, the supported types are:

pubsub

Collect log events from Google Pub/Sub.

storage

Collect log events from Google Cloud storage buckets.

description
edit

A description of the function. This description is useful when you are running multiple functions and need more context about how each function is used.

memory_size

edit

The maximum amount of memory to allocate for this function. The default is 256MB.

timeout

edit

The execution timeout in seconds. If the function does not finish in time, it is considered failed and terminated. The default is 60s. Increase this value if you see timeout messages is the Google Stackdriver logs.

service_account_email

edit

The email of the service account that the function will assume as its identity. The default is {projectid}@appspot.gserviceaccount.com.email.

labels

edit

One or more labels to apply to the function. A label is a key-value pair that helps you organize your Google Cloud resources.

vpc_connector

edit

A VPC connector that the function can connect to when sending requests to resources in your VPC network.

Use the format projects/*/locations/*/connectors/* or a fully qualified URI.

maximum_instances

edit

The maximum instances that can be running at the same time. The default is unlimited.

trigger
edit

The trigger that will cause the function to execute.

  • If type is pubsub, specify the name of the Pub/Sub topic to watch for messages.
  • If type is storage, specify the Cloud Storage bucket to watch for object events. For event_type, specify the type of object event that will trigger the function. See the Google Cloud docs for a list of available event types.

keep_null

edit

If true, fields with null values will be published in the output document. By default, keep_null is false.

fields

edit

Optional fields that you can specify to add additional information to the output. Fields can be scalar values, arrays, dictionaries, or any nested combination of these.

processors

edit

Define custom processors for this function. For example, you can specify a dissect processor to tokenize a string:

processors:
  - dissect:
      tokenizer: "%{key1} %{key2}"