IMPORTANT: No additional bug fixes or documentation updates will be released for this version. For the latest information, see the current release documentation.
« Create and upload config.yaml to S3 bucket Define deployment parameters »
Elastic Docs ›Observability Guide [8.7] ›Send data to Elasticsearch ›Elastic Serverless Forwarder for AWS ›Deploy Elastic Serverless Forwarder ›Create and upload config.yaml to S3 bucket

Fields

edit
Fields
edit

inputs.[]:

A list of inputs (i.e. triggers) for the Elastic Serverless Forwarder Lambda function.

inputs.[].type:

The type of trigger input (cloudwatch-logs, kinesis-data-stream, sqs and s3-sqs are currently supported).

inputs.[].id:

The ARN of the trigger input according to the type. Multiple input entries can have different unique ids with the same type. Inputs of type cloudwatch-logs accept both CloudWatch Logs Log Group and CloudWatch Logs Log Stream ARNs.

inputs.[].outputs:

A list of outputs (i.e. forwarding targets) for the Elastic Serverless Forwarder Lambda function. You can have multiple outputs for an input, but only one output can be defined per type.

inputs.[].outputs.[].type:

The type of the forwarding target output. Currently only the following outputs are supported:

  • elasticsearch
  • [preview] This functionality is in technical preview and may be changed or removed in a future release. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features. logstash

If Logstash is chosen as an output, Elastic Serverless Forwarder expects the elastic_serverless_forwarder Logstash input to be installed, enabled, and properly configured. For more information about installing Logstash plugins, refer to the Logstash documentation.

inputs.[].outputs.[].args: Custom init arguments for the specified forwarding target output.

For elasticsearch the following arguments are supported:

  • args.elasticsearch_url: URL of elasticsearch endpoint in the format http(s)://domain.tld:port. Mandatory when args.cloud_id is not provided. Will take precedence over args.cloud_id if both are defined.
  • args.cloud_id: Cloud ID of elasticsearch endpoint. Mandatory when args.elasticsearch_url is not provided. Will be ignored if args.elasticsearch_url is defined.
  • args.username: Username of the elasticsearch instance to connect to. Mandatory when args.api_key is not provided. Will take precedence over args.api_key if both are defined.
  • args.password Password of the elasticsearch instance to connect to. Mandatory when args.api_key is not provided. Will take precedence over args.api_key if both are defined.
  • args.api_key: API key of elasticsearch endpoint in the format base64encode(api_key_id:api_key_secret). Mandatory when args.username and args.password are not provided. Will be ignored if args.username/args.password are defined.
  • args.es_datastream_name: Name of data stream or index where logs should be forwarded to. Lambda supports automatic routing of various AWS service logs to the corresponding data streams for further processing and storage in the Elasticsearch cluster. It supports automatic routing of aws.cloudtrail, aws.cloudwatch_logs, aws.elb_logs, aws.firewall_logs, aws.vpcflow, and aws.waf logs. For other log types, if using data streams, you can optionally set its value in the configuration file according to the naming convention for data streams and available integrations. If the es_datastream_name is not specified and it cannot be matched with any of the above AWS services, then the value will be set to logs-generic-default. In versions v0.29.1 and below, this configuration parameter was named es_index_or_datastream_name. Rename the configuration parameter to es_datastream_name in your config.yaml file on the S3 bucket to continue using it in the future version. The older name es_index_or_datastream_name is deprecated as of version v0.30.0. The related backward compatibility code is removed from version v1.0.0.
  • args.batch_max_actions: (Optional) Maximum number of actions to send in a single bulk request. Default value: 500.
  • args.batch_max_bytes: (Optional) Maximum size in bytes to send in a single bulk request. Default value: 10485760 (10MB).
  • args.ssl_assert_fingerprint: (Optional) SSL fingerprint for self-signed SSL certificate on HTTPS transport.

For logstash the following arguments are supported:

  • args.logstash_url: URL of Logstash endpoint in the format http(s)://host:port
  • args.username: (Optional) Username of the Logstash instance to connect to. Mandatory if HTTP Basic authentication is enabled in Logstash.
  • args.password: (Optional) Password of the Logstash instance to connect to. Mandatory if HTTP Basic authentication is enabled in Logstash.
  • args.max_batch_size: (Optional) Maximum number of events to send in a single HTTP(s) request. Default value: 500
  • args.compression_level: (Optional) The GZIP compression level for HTTP(s) requests towards Logstash. It can be any integer value between 1 (minimum compression, best performance, highest amount of bytes sent) and 9 (maximum compression, worst performance, lowest amount of bytes sent). Default value: 1
  • args.ssl_assert_fingerprint: (Optional) SSL fingerprint for self-signed SSL certificate on HTTPS transport.
« Create and upload config.yaml to S3 bucket Define deployment parameters »