---
title: GoFlow2 logs Integration for Elastic
description: The GoFlow2 logs integration for Elastic enables you to ingest and normalize network flow data, specifically focusing on the sFlow protocol. By using...
url: https://www.elastic.co/docs/reference/integrations/goflow2
products:
  - Elastic integrations
applies_to:
  - Serverless Observability projects: Preview
  - Serverless Security projects: Preview
  - Elastic Stack: Preview since 9.0
---

# GoFlow2 logs Integration for Elastic
|                                                                                                 |                                                                   |
|-------------------------------------------------------------------------------------------------|-------------------------------------------------------------------|
| **Version**                                                                                     | 0.7.1 <applies-to>: Preview</applies-to> ([View all](#changelog)) |
| **Subscription level**[What's this?](https://www.elastic.co/subscriptions)                      | Basic                                                             |
| **Developed by**[What's this?](https://www.elastic.co/docs/reference/integrations/developed-by) | Community                                                         |
| **Ingestion method(s)**                                                                         | File                                                              |
| **Minimum Kibana version(s)**                                                                   | 9.0.08.11.0                                                       |

<admonition title="The GoFlow2 logs integration v0.7.1 is in technical preview">
  To use pre-release integrations, go to the **Integrations** page in Kibana, scroll down, and toggle on the _Display beta integrations_ option.
</admonition>


## Overview

The GoFlow2 logs integration for Elastic enables you to ingest and normalize network flow data, specifically focusing on the sFlow protocol. By using this integration, you can gain deep visibility into network traffic patterns, bandwidth usage, and potential security threats.

### Compatibility

This integration is compatible with the following:
- GoFlow2 binary versions that support the `-mapping` and `-transport.file` flags, such as version 2.1.2 or later.
- Network infrastructure devices (switches, routers, firewalls) capable of exporting sFlow (versions 2, 4, or 5).

<note>
  This integration specifically supports sFlow normalization only; NetFlow and IPFIX protocols are currently unsupported for normalization.
</note>


### How it works

This integration collects network flow data by monitoring JSON log files generated by the GoFlow2 collector. GoFlow2 acts as a flow collector that receives sFlow packets from network devices and converts them into a structured JSON format.
The Elastic Agent monitors the configured log path (defaulting to `/var/log/sflow/goflow2/*.log`) to ingest these formatted JSON logs. The integration then maps the data to the Elastic Common Schema (ECS), ensuring that fields like source and destination IP addresses, ports, and VLAN tags are normalized for consistent analysis across your Elastic deployment.

## What data does this integration collect?

The GoFlow2 logs integration collects network flow data by monitoring JSON log files generated by the GoFlow2 collector. The data is ingested into the `goflow2.sflow` dataset.
The integration collects the following types of data:
- **sFlow logs**: Detailed records of sampled network traffic, capturing packet headers and flow metadata.
- **Network Metadata**: Information including source and destination IP addresses, source and destination ports, and VLAN tags.
- **Flow Statistics**: Metrics regarding the volume of traffic, such as the number of bytes and packets transferred in a specific flow.


### Supported use cases

Integrating GoFlow2 logs with Elastic provides visibility into network traffic patterns and performance. Key use cases include:
- **Network traffic analysis**: Analyze traffic flows to understand bandwidth usage, identify top talkers, and optimize network performance. Gain real-time visibility into high-volume network traffic by collecting sFlow data from routers and switches, allowing for detailed analysis of source and destination patterns.
- **Security analysis and monitoring**: Detect anomalous traffic patterns, potential DDoS attacks, or unauthorized data transfers by monitoring source and destination IPs.
- **Bandwidth management**: Track bandwidth consumption across different interfaces and VLANs to identify top talkers and optimize network resource allocation. Use flow statistics to track long-term trends in network usage and plan for future infrastructure needs.
- **Troubleshooting**: Investigate network connectivity issues by examining specific flow records and metadata.
- **Compliance and auditing**: Maintain long-term records of network flows for regulatory compliance and forensic investigations following security incidents.


## What do I need to use this integration?

To use this integration, you need:
- **Elastic Stack version**: Ensure you are running an Elastic Stack version that supports Elastic Agent and Fleet (version 8.11.0 or later is required).
- **Elastic Agent**: The agent must be installed and enrolled in a policy via Fleet. It requires network access to the Elasticsearch cluster to ship logs.
- **Administrative access**: You must have root or sudo privileges on the Linux server where GoFlow2 and the Elastic Agent will be installed.
- **GoFlow2 binary**: The GoFlow2 binary must be downloaded and placed in the system execution path (for example, `/usr/local/bin/`).
- **Directory permissions**: The directory `/var/log/sflow/goflow2/` must exist. It must be writable by the GoFlow2 process and readable by the Elastic Agent.
- **Network connectivity**: Port `6343/UDP` must be open on the host firewall to receive sFlow traffic from network devices.
- **Network device configuration**: Access to your network switches or routers is required to configure them to export sFlow data to the collector's IP address.


## How do I deploy this integration?


### Agent-based deployment

Elastic Agent must be installed. For more details, check the Elastic Agent [installation instructions](https://www.elastic.co/guide/en/fleet/current/elastic-agent-installation.html). You can install only one Elastic Agent per host.
Elastic Agent is required to stream data from the syslog or log file receiver and ship the data to Elastic, where the events will then be processed via the integration's ingest pipelines.

### Set up steps in GoFlow2 logs

Follow these steps to install and configure GoFlow2 to collect sFlow data and write it to a log file that Elastic Agent can read.

#### 1. Install GoFlow2 Binary

Download the latest release and move it to your bin directory:
```bash
wget https://github.com/netsampler/goflow2/releases/download/v2.2.6/goflow2_2.2.6_linux_amd64.tar.gz
tar -xzvf goflow2_2.2.6_linux_amd64.tar.gz
sudo mv goflow2 /usr/local/bin/
sudo chmod +x /usr/local/bin/goflow2
```


#### 2. Prepare Directories

Create the configuration and log directories:
```bash
sudo mkdir -p /etc/goflow2
sudo mkdir -p /var/log/sflow/goflow2
```


#### 3. Configure the Mapping File

Create `/etc/goflow2/mapping.yaml` to ensure JSON fields match the integration's expectations:
```yaml
formatter:
    fields:
        - type
        - time_flow_start_ns
        - sampler_address
        - sequence_num
        - in_if
        - out_if
        - src_addr
        - dst_addr
        - etype
        - proto
        - src_port
        - dst_port
        - src_vlan
        - dst_vlan
        - sampling_rate
        - bytes
```


#### 4. Setup Systemd Service

Create `/etc/systemd/system/goflow2.service` with the following configuration:
```ini
[Unit]
Description=GoFlow2 sFlow Collector
After=network.target

[Service]
Type=simple
ExecStart=/usr/local/bin/goflow2 \
    -format json \
    -listen "sflow://:6343" \
    -mapping /etc/goflow2/mapping.yaml \
    -transport.file /var/log/sflow/goflow2/goflow2.log
Restart=always
RestartSec=5

[Install]
WantedBy=multi-user.target
```


#### 5. Start the Collector

Reload systemd, enable, and start the service:
```bash
sudo systemctl daemon-reload
sudo systemctl enable --now goflow2
```


#### 6. Configure Network Exporters

Log into your network devices (switches/routers) and configure the sFlow destination to the IP of this server on UDP port `6343`.

#### Vendor resources

- [GoFlow2 GitHub Repository](https://github.com/netsampler/goflow2)
- [GoFlow2 Configuration Guide - DeepWiki](https://deepwiki.com/netsampler/goflow2/2.2-configuration)


### Set up steps in Kibana

1. In Kibana, navigate to **Management > Integrations**.
2. Search for and select **GoFlow2**.
3. Click **Add GoFlow2**.
4. Configure the **Collecting logs via log file** input:
   - **Paths**: Specify the list of paths to the log files that contain sFlow logs. Default: `['/var/log/sflow/goflow2/*.log']`.
- **Preserve original event**: Toggle this to preserve a raw copy of the original event in the `event.original` field. Default: `False`.
- **Ignore events older than**: Set the duration to ignore events older than a specific time. Valid units: `ns`, `us`, `ms`, `s`, `m`, `h`. Default: `72h`.
- **Exclude files**: Add [RE2 syntax](https://github.com/google/re2/wiki/Syntax) regular expressions to match files that should be excluded. See [exclude_files](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-log.html#filebeat-input-log-exclude-files) for details. Default: `['\\.gz$']`.
- **Tags**: Add custom tags to append to the events. Default: `['sflow', 'forwarded']`.
- **Custom Filestream Options**: Provide additional configuration options for the filestream input. See [filestream input](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-filestream.html) for details.
- **Processors**: Define processors to enhance or filter data at the agent level. See [Processors](https://www.elastic.co/guide/en/beats/filebeat/current/filtering-and-enhancing-data.html) for details. Default: `- add_locale: ~`.
5. Provide a **Name** and **Description** for the integration instance.
6. Select the **Agent Policy** to which you want to add the integration.
7. Click **Save and continue**.


### Validation

After configuration is complete, verify that data is flowing correctly.

#### 1. Trigger Data Flow on goflow2

1. **Generate Network Traffic:** Initiate traffic through a network device configured to export sFlow (e.g., run a ping sweep or a large file transfer through the switch).
2. **Verify Log Generation:** Check that GoFlow2 is actively writing to the log file by running the following command:
   ```bash
   tail -f /var/log/sflow/goflow2/goflow2.log
   ```
3. **Restart Service:** Restart the GoFlow2 service to generate initial startup logs:
   ```bash
   sudo systemctl restart goflow2
   ```
4. **Check UDP Reception:** Confirm the server is receiving traffic on the sFlow port:
   ```bash
   sudo tcpdump -i any udp port 6343
   ```


#### 2. Check Data in Kibana

1. Navigate to **Analytics > Discover**.
2. Select the `logs-*` data view.
3. Enter the KQL filter: `data_stream.dataset : "goflow2.sflow"`
4. Verify logs appear. Expand a log entry and confirm these fields are populated:
   - `event.dataset` (should be `goflow2.sflow`)
- `source.ip`
- `destination.ip`
- `event.action` or `event.outcome`
- `message`
5. Navigate to **Analytics > Dashboards** and search for "GoFlow2" to view pre-built visualizations.


## Troubleshooting

For help with Elastic ingest tools, check [Common problems](https://www.elastic.co/docs/troubleshoot/ingest/fleet/common-problems).

### Common configuration issues

- Log file permission issues:
  - Ensure the user running the Elastic Agent has read access to `/var/log/sflow/goflow2/` and that the GoFlow2 process has write permissions.
- UDP port conflict:
  - If the GoFlow2 service fails to start, check if another process is already bound to port `6343/UDP` using the command `sudo netstat -tulpn | grep 6343`.
- Mapping.yaml formatting errors:
  - YAML is sensitive to indentation. If GoFlow2 fails to parse the mapping file, ensure it matches the exact structure provided in the setup steps.
- Mismatched file paths:
  - Ensure the **Paths** variable provided in the Kibana integration configuration matches the `-transport.file` path in the GoFlow2 systemd service.
- Parsing failures:
  - If logs appear in Kibana but fields are not correctly mapped, check the `error.message` field in the log entry. This often indicates that the GoFlow2 output does not match the integration's expected JSON structure.
- Empty log files:
  - If the log file exists but is empty, verify that the network devices are successfully reaching the collector on port 6343/UDP using the command `tcpdump -i any udp port 6343`.
- Unsupported protocols:
  - If you attempt to send NetFlow or IPFIX data, GoFlow2 may log errors or output data that the integration pipeline cannot process, as only sFlow is currently supported.


## Performance and scaling

To ensure optimal performance in high-volume environments, consider the following strategies:
- **Transport and collection:** The integration uses the `filestream` input to monitor local log files generated by the GoFlow2 binary. Ensure that the disk subsystem has sufficient I/O performance to handle simultaneous writes from the GoFlow2 collector and reads by the Elastic Agent. Use a robust log rotation strategy (e.g., `logrotate`) to prevent disk exhaustion, ensuring that the Elastic Agent has sufficient time to harvest logs before they are purged.
- **Data volume management:** In high-density network environments, it is critical to tune the sFlow sampling rate at the source (network hardware). A sampling rate of 1 in 1000 or 1 in 5000 is often sufficient for traffic analysis while significantly reducing the CPU load on the GoFlow2 collector and the event volume processed by the Elastic Agent.
- **Elastic Agent scaling:** For high-throughput environments, deploy the Elastic Agent on a dedicated host with multiple CPU cores to handle JSON parsing. If a single collector becomes a bottleneck, distribute your network devices across multiple GoFlow2 collectors and Elastic Agents to load-balance the ingest pipeline.

For more information on architectures that can be used for scaling this integration, check the [Ingest Architectures](https://www.elastic.co/docs/manage-data/ingest/ingest-reference-architectures) documentation.

## Reference


### Inputs used

These inputs can be used with this integration:
<dropdown title="filestream">
  ## SetupFor more details about the Filestream input settings, check the [Filebeat documentation](https://www.elastic.co/docs/reference/beats/filebeat/filebeat-input-filestream).


  ### Collecting logs from FilestreamTo collect logs via Filestream, select **Collect logs via Filestream** and configure the following parameters:

  - Filestream paths: The full path to the related log file.
</dropdown>


### Data streams


#### sflow

The `sflow` data stream collects sFlow network traffic data processed by GoFlow2.

##### sflow fields

<dropdown title="Exported fields">
  | Field                 | Description                                     | Type             |
  |-----------------------|-------------------------------------------------|------------------|
  | @timestamp            | Event timestamp.                                | date             |
  | data_stream.dataset   | Data stream dataset.                            | constant_keyword |
  | data_stream.namespace | Data stream namespace.                          | constant_keyword |
  | data_stream.type      | Data stream type.                               | constant_keyword |
  | input.type            | Input type                                      | keyword          |
  | log.file.device_id    | Device Id of the log file this event came from. | keyword          |
  | log.file.inode        | Inode of the log file this event came from.     | keyword          |
  | log.offset            | Log offset                                      | long             |
  | sflow.bytes           | Original size in  bytes of the sample packet.   | long             |
  | sflow.sample_rate     | sample rate.                                    | long             |
  | sflow.sequence_num    | flow sequence number.                           | long             |
</dropdown>


##### sflow sample event

<dropdown title="Example">
  ```json
  {
      "@timestamp": "2024-07-31T00:00:59.314Z",
      "destination": {
          "address": [
              "216.160.83.58"
          ],
          "as": {
              "number": 209
          },
          "geo": {
              "city_name": "Milton",
              "continent_name": "North America",
              "country_iso_code": "US",
              "country_name": "United States",
              "location": {
                  "lat": 47.2513,
                  "lon": -122.3149
              },
              "region_iso_code": "US-WA",
              "region_name": "Washington"
          },
          "ip": "216.160.83.58",
          "port": 443
      },
      "ecs": {
          "version": "8.17.0"
      },
      "event": {
          "action": "SFLOW_5",
          "category": [
              "network"
          ],
          "kind": "event",
          "original": "{\"type\":\"SFLOW_5\",\"time_flow_start_ns\":1722384059314899647,\"sampler_address\":\"67.43.156.1\",\"sequence_num\":44555,\"in_if\":563,\"out_if\":573,\"src_addr\":\"216.160.83.57\",\"dst_addr\":\"216.160.83.58\",\"etype\":\"IPv4\",\"proto\":\"TCP\",\"src_port\":10876,\"dst_port\":443,\"src_vlan\":1500,\"dst_vlan\":1500,\"sampling_rate\":1000,\"bytes\":70}",
          "timezone": "+00:00",
          "type": [
              "connection"
          ]
      },
      "network": {
          "bytes": 70000,
          "packets": 1000,
          "transport": "tcp",
          "type": "ipv4"
      },
      "observer": {
          "egress": {
              "interface": {
                  "id": "573"
              },
              "vlan": {
                  "id": "1500"
              }
          },
          "ingress": {
              "interface": {
                  "id": "563"
              },
              "vlan": {
                  "id": "1500"
              }
          },
          "ip": [
              "67.43.156.1"
          ]
      },
      "related": {
          "ip": [
              "216.160.83.57",
              "216.160.83.58"
          ]
      },
      "sflow": {
          "bytes": 70,
          "sample_rate": 1000,
          "sequence_num": 44555
      },
      "source": {
          "address": [
              "216.160.83.57"
          ],
          "as": {
              "number": 209
          },
          "geo": {
              "city_name": "Milton",
              "continent_name": "North America",
              "country_iso_code": "US",
              "country_name": "United States",
              "location": {
                  "lat": 47.2513,
                  "lon": -122.3149
              },
              "region_iso_code": "US-WA",
              "region_name": "Washington"
          },
          "ip": "216.160.83.57",
          "port": 10876
      },
      "tags": [
          "preserve_original_event",
          "forwarded",
          "sflow"
      ]
  }
  ```
</dropdown>


### Vendor documentation links

See the following resources for more information:
- [GoFlow2 GitHub Repository](https://github.com/netsampler/goflow2)


## Changelog

<dropdown title="Changelog">
  | Version   | Details                                                                                                                                                                                             | Minimum Kibana version |
  |-----------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------|
  | **0.7.1** | **Bug fix** ([View pull request](https://github.com/elastic/integrations/pull/18969))Remove top level note from docs                                                                                | 9.0.08.11.0            |
  | **0.7.0** | **Enhancement** ([View pull request](https://github.com/elastic/integrations/pull/17348))Update documentation.                                                                                      | 9.0.08.11.0            |
  | **0.6.0** | **Enhancement** ([View pull request](https://github.com/elastic/integrations/pull/15803))Preserve event.original on pipeline error.                                                                 | 9.0.08.11.0            |
  | **0.5.2** | **Enhancement** ([View pull request](https://github.com/elastic/integrations/pull/15545))Normalize error handler.                                                                                   | 9.0.08.11.0            |
  | **0.5.1** | **Enhancement** ([View pull request](https://github.com/elastic/integrations/pull/14753))Changed owners.                                                                                            | 9.0.08.11.0            |
  | **0.5.0** | **Enhancement** ([View pull request](https://github.com/elastic/integrations/pull/13521))Allow @custom pipeline access to event.original without setting preserve_original_event.                   | 9.0.08.11.0            |
  | **0.4.0** | **Enhancement** ([View pull request](https://github.com/elastic/integrations/pull/13042))Support stack version 9.0.                                                                                 | 9.0.08.11.0            |
  | **0.3.0** | **Enhancement** ([View pull request](https://github.com/elastic/integrations/pull/12571))ECS version updated to 8.17.0.                                                                             | 8.11.0                 |
  | **0.2.0** | **Enhancement** ([View pull request](https://github.com/elastic/integrations/pull/12349))Add configuration for custom filestream options. Ensure filestream fingerprint is disabled in system test. | 8.11.0                 |
  | **0.1.1** | **Bug fix** ([View pull request](https://github.com/elastic/integrations/pull/11315))Use triple-brace Mustache templating when referencing variables in ingest pipelines.                           | 8.11.0                 |
  | **0.1.0** | **Enhancement** ([View pull request](https://github.com/elastic/integrations/pull/10561))Initial version of the package                                                                             | 8.11.0                 |
</dropdown>