New

The executive guide to generative AI

Read more
About usPartnersSupport|Login
IMPORTANT: No additional bug fixes or documentation updates will be released for this version. For the latest information, see the current release documentation.
« Step 1: Install Filebeat Step 3: Load the index template in Elasticsearch »
Elastic Docs ›Filebeat Reference [7.4] ›Getting Started With Filebeat

Step 2: Configure Filebeat

edit

Step 2: Configure Filebeat

edit

Filebeat modules provide the fastest getting started experience for common log formats. If you are using Filebeat modules, skip this section, including the remaining getting started steps, and go directly to Quick start: modules for common log formats.

To configure Filebeat, you edit the configuration file. The default configuration file is called filebeat.yml. The location of the file varies by platform. To locate the file, see Directory layout.

There’s also a full example configuration file called filebeat.reference.yml that shows all non-deprecated options.

See the Config File Format section of the Beats Platform Reference for more about the structure of the config file.

Here is a sample of the filebeat section of the filebeat.yml file. Filebeat uses predefined default values for most configuration options.

filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /var/log/*.log
    #- c:\programdata\elasticsearch\logs\*

To configure Filebeat:

  1. Define the path (or paths) to your log files.

    For the most basic Filebeat configuration, you can define a single input with a single path. For example:

    filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /var/log/*.log

    The input in this example harvests all files in the path /var/log/*.log, which means that Filebeat will harvest all files in the directory /var/log/ that end with .log. All patterns supported by Go Glob are also supported here.

    To fetch all files from a predefined level of subdirectories, the following pattern can be used: /var/log/*/*.log. This fetches all .log files from the subfolders of /var/log. It does not fetch log files from the /var/log folder itself. Currently it is not possible to recursively fetch all files in all subdirectories of a directory.

  2. Configure the output. Filebeat supports a variety of outputs, but typically you’ll either send events directly to Elasticsearch, or to Logstash for additional processing.

    To send output directly to Elasticsearch (without using Logstash), set the location of the Elasticsearch installation:

    • If you’re running our hosted Elasticsearch Service on Elastic Cloud, specify your Cloud ID. For example:

      cloud.id: "staging:dXMtZWFzdC0xLmF3cy5mb3VuZC5pbyRjZWM2ZjI2MWE3NGJmMjRjZTMzYmI4ODExYjg0Mjk0ZiRjNmMyY2E2ZDA0MjI0OWFmMGNjN2Q3YTllOTYyNTc0Mw=="

    • If you’re running Elasticsearch on your own hardware, set the host and port where Filebeat can find the Elasticsearch installation. For example:

      output.elasticsearch:
  hosts: ["myEShost:9200"]

      To send output to Logstash, Configure the Logstash output instead. For all other outputs, see Configure the output.

  3. If you plan to use the sample Kibana dashboards provided with Filebeat, configure the Kibana endpoint. You can skip this step if Kibana is running on the same host as Elasticsearch.

    setup.kibana:
  host: "mykibanahost:5601"

    The hostname and port of the machine where Kibana is running, for example, mykibanahost:5601. If you specify a path after the port number, include the scheme and port: http://mykibanahost:5601/path.

  4. If Elasticsearch and Kibana are secured, set credentials in the filebeat.yml config file before you run the commands that set up and start Filebeat.

    • If you’re running our hosted Elasticsearch Service on Elastic Cloud, specify your cloud auth credentials. For example:

      cloud.auth: "elastic:YOUR_PASSWORD"

    • If you’re running Elasticsearch on your own hardware, specify your Elasticsearch and Kibana credentials:

      output.elasticsearch:
  hosts: ["myEShost:9200"]
  username: "filebeat_internal"
  password: "YOUR_PASSWORD" 
setup.kibana:
  host: "mykibanahost:5601"
  username: "my_kibana_user"  
  password: "YOUR_PASSWORD"

      This examples shows a hard-coded password, but you should store sensitive values in the secrets keystore.

      The username and password settings for Kibana are optional. If you don’t specify credentials for Kibana, Filebeat uses the username and password specified for the Elasticsearch output.

      To use the pre-built Kibana dashboards, this user must have the kibana_user built-in role or equivalent privileges.

      For more information, see Securing Filebeat.

To test your configuration file, change to the directory where the Filebeat binary is installed, and run Filebeat in the foreground with the following options specified: ./filebeat test config -e. Make sure your config files are in the path expected by Filebeat (see Directory layout), or use the -c flag to specify the path to the config file.

For more information about configuring Filebeat, see Configuring Filebeat.

Starting with Filebeat 6.5, you can define and manage Filebeat configurations in a central location in Kibana. For more information, see Beats central management.

« Step 1: Install Filebeat Step 3: Load the index template in Elasticsearch »
Was this helpful?
Feedback