IMPORTANT: No additional bug fixes or documentation updates will be released for this version. For the latest information, see the current release documentation.

› ›

AWS module

edit

AWS module

edit

This functionality is in beta and is subject to change. The design and code is less mature than official GA features and is being provided as-is with no warranties. Beta features are not subject to the support SLA of official GA features.

This is a module for aws logs. It uses filebeat s3 input to get log files from AWS S3 buckets with SQS notification. This module supports reading s3 server access logs with s3access fileset, ELB access logs with elb fileset, VPC flow logs with vpcflow fileset, and CloudTrail logs with cloudtrail fileset.

Access logs contain detailed information about the requests made to these services. VPC flow logs captures information about the IP traffic going to and from network interfaces in AWS VPC. ELB access logs captures detailed information about requests sent to the load balancer. CloudTrail logs contain events that represent actions taken by a user, role or AWS service.

Read the quick start to learn how to set up and run modules.

Module configuration

edit

Example config:

- module: aws
  s3access:
    enabled: false

    # AWS SQS queue url
    #var.queue_url: https://sqs.myregion.amazonaws.com/123456/myqueue

    # Filename of AWS credential file
    # If not set "$HOME/.aws/credentials" is used on Linux/Mac
    # "%UserProfile%\.aws\credentials" is used on Windows
    # var.shared_credential_file: /etc/filebeat/aws_credentials

    # Profile name for aws credential
    # If not set the default profile is used
    # var.credential_profile_name: fb-aws

  elb:
    enabled: false

    # AWS SQS queue url
    #var.queue_url: https://sqs.myregion.amazonaws.com/123456/myqueue

    # Filename of AWS credential file
    # If not set "$HOME/.aws/credentials" is used on Linux/Mac
    # "%UserProfile%\.aws\credentials" is used on Windows
    # var.shared_credential_file: /etc/filebeat/aws_credentials

    # Profile name for aws credential
    # If not set the default profile is used
    # var.credential_profile_name: fb-aws

  vpcflow:
    enabled: false

    # AWS SQS queue url
    #var.queue_url: https://sqs.myregion.amazonaws.com/123456/myqueue

    # Filename of AWS credential file
    # If not set "$HOME/.aws/credentials" is used on Linux/Mac
    # "%UserProfile%\.aws\credentials" is used on Windows
    # var.shared_credential_file: /etc/filebeat/aws_credentials

    # Profile name for aws credential
    # If not set the default profile is used
    # var.credential_profile_name: fb-aws

  cloudtrail:
    enabled: false

    # AWS SQS queue url
    #var.queue_url: https://sqs.myregion.amazonaws.com/123456/myqueue

    # Filename of AWS credential file
    # If not set "$HOME/.aws/credentials" is used on Linux/Mac
    # "%UserProfile%\.aws\credentials" is used on Windows
    # var.shared_credential_file: /etc/filebeat/aws_credentials

    # Profile name for aws credential
    # If not set the default profile is used
    # var.credential_profile_name: fb-aws

var.queue_url: AWS SQS queue url.
var.shared_credential_file: Filename of AWS credential file.
var.credential_profile_name: AWS credential profile name.

cloudtrail fileset

edit

CloudTrail monitors events for the account. If user creates a trail, it delivers those events as log files to a specific Amazon S3 bucket. The cloudtrail fileset does not read the CloudTrail Digest files that are delivered to the S3 bucket when Log File Integrity is turned on, it only reads the CloudTrail logs.

elb fileset

edit

Elastic Load Balancing provides access logs that capture detailed information about requests sent to the load balancer. Each log contains information such as the time the request was received, the client’s IP address, latencies, request paths, and server responses. Users can use these access logs to analyze traffic patterns and to troubleshoot issues.

Please follow enable access logs for classic load balancer for sending Classic ELB access logs to S3 bucket. For application load balancer, please follow enable access log for application load balancer. For network load balancer, please follow enable access log for network load balancer.

This fileset comes with a predefined dashboard:

s3access fileset

edit

Server access logging provides detailed records for the requests that are made to a bucket. Server access logs are useful for many applications. For example, access log information can be useful in security and access audits. It can also help you learn about customer base and understand Amazon S3 bill.

Please follow how to enable server access logging for sending server access logs to S3 bucket.

This fileset comes with a predefined dashboard:

vpcflow fileset

edit

VPC Flow Logs is a feature in AWS that enables users to capture information about the IP traffic going to and from network interfaces in VPC. Flow log data needs to be published to Amazon S3 in order for vpcflow fileset to retrieve. Flow logs can help users to monitor traffic that is reaching each instance and determine the direction of the traffic to and from the network interfaces.

This fileset comes with a predefined dashboard:

Fields

edit

For a description of each field in the module, see the exported fields section.

« Auditd module Azure module »