Introducing Auditbeat: Ship Linux Audit Logs to Elasticsearch and More

By
Andrew Kroh

Auditbeat is an exciting new addition to the Beats family for 6.0. Its purpose is to audit the activities of users and processes on your systems. Auditbeat is a beta project and currently has two main monitoring capabilities: file integrity monitoring and Linux audit framework monitoring. We expect to add more features as the project matures. Let's take a closer look at its capabilities.

Linux Audit Framework Monitoring

Auditbeat receives events from the audit framework in the Linux kernel and sends them to Elasticsearch. This is similar to auditd, but with some additional logic features that makes it really simple to get the data into Elasticsearch. It will automatically group related messages into a single event even if they arrive out of order from the kernel. It also parses and normalizes the messages so that the data can be delivered to Elasticsearch in a structured format. Let's take a look at an example.

What follows are three audit messages generated by the kernel and received by Auditbeat when a user unsuccessfully attempted to open the /etc/gshadow file. I configured Auditbeat with a rule for generating events when an open syscall fails due to EACCES (permission denied).

  1. type=SYSCALL msg=audit(1501619655.636:32796540): arch=c000003e syscall=2 success=no exit=-13 a0=7fffdd0a98e2 a1=0 a2=7fffdd0a8210 a3=7fffdd0a7c60 items=1 ppid=6637 pid=6654 auid=700 uid=700 gid=700 euid=700 suid=700 fsuid=700 egid=700 sgid=700 fsgid=700 tty=pts0 ses=15663 comm="cat" exe="/bin/cat" key="open"
  2. type=CWD msg=audit(1501619655.636:32796540): cwd="/home/admin"
  3. type=PATH msg=audit(1501619655.636:32796540): item=0 name="/etc/gshadow" inode=138686 dev=fc:00 mode=0100000 ouid=0 ogid=0 rdev=00:00 nametype=NORMAL

These messages are then turned into a single event that can be sent to Elasticsearch.

{
  "@timestamp": "2017-08-01T20:34:15.636Z",
  "audit": {
    "kernel": {
      "sequence": 32796540,
      "category": "audit-rule",
      "record_type": "syscall",
      "session": "15663",
      "result": "fail",
      "actor": {
        "primary": "admin",
        "secondary": "admin",
        "attrs": {
          "suid": "admin",
          "auid": "admin",
          "euid": "admin",
          "fsgid": "sysadmins",
          "gid": "sysadmins",
          "sgid": "sysadmins",
          "egid": "sysadmins",
          "fsuid": "admin",
          "uid": "admin"
        }
      },
      "action": "opened-file",
      "thing": {
        "what": "file",
        "primary": "/etc/gshadow",
        "secondary": "138686"
      },
      "how": "/bin/cat",
      "key": "open",
      "data": {
        "exe": "/bin/cat",
        "exit": "EACCES",
        "pid": "6654",
        "ppid": "6637",
        "syscall": "open",
        "tty": "pts0",
        "arch": "x86_64",
        "cwd": "/home/admin",
        "a2": "7fffdd0a8210",
        "a3": "7fffdd0a7c60",
        "comm": "cat",
        "a0": "7fffdd0a98e2",
        "a1": "0"
      },
      "paths": [
        {
          "item": "0",
          "name": "/etc/gshadow",
          "ouid": "root",
          "dev": "fc:00",
          "mode": "0100000",
          "nametype": "NORMAL",
          "ogid": "root",
          "rdev": "00:00",
          "inode": "138686"
        }
      ]
    }
  }
}

This event answers the important questions of

  • who did it (actor)?
  • what did they do (action)?
  • when did they do it (@timestamp)?
  • how did they do it (how)?
  • to whom did they do it (thing)?
  • what was the outcome (result)?.

Plus Auditbeat retains all of the original syscall data and the associated paths. Ingesting audit data from Linux has never been so painless!

Check out the documentation for more details on how it works and how to configure it.

File Integrity Monitoring

Another feature in this first release of Auditbeat is the ability to monitor files for changes. Security policies often have requirement for this type of monitoring. Auditbeat can watch a list of directories or files and send events in realtime to Elasticsearch when changes occur. The events contain file metadata and cryptographic hashes of the file contents.

The file hashes can be useful in several ways. They can be used to identify whether a specific file is in compliance across a fleet of machines. Or the hashes could be cross-referenced with threat intelligence sources to identify potential malware.

The setup is simple. You just need to specify the paths to the directories that you want Auditbeat to watch. This feature can be used on Linux, macOS, and Windows. Check out the documentation for all the details about how it works and how to configure it.

When a file is created, modified, or deleted in one of the watched directories Auditbeat generates an event like this.

{
  "@timestamp": "2017-08-01T15:24:07.217Z",
  "audit": {
    "file": {
      "action": "created",
      "type": "file",
      "path": "/etc/sudoers.d/superusers",
      "gid": 0,
      "uid": 0,
      "owner": "root",
      "group": "root",
      "inode": "196640",
      "size": 42,
      "mode": "0440",
      "mtime": "2017-08-01T15:24:07.215Z",
      "ctime": "2017-08-01T15:24:07.215Z",
      "atime": "2017-08-01T15:24:07.214Z",
      "hashed": true,
      "md5": "7944a620fcc4dc850fed30643e7a1401",
      "sha1": "a827656e9aecd6f8ed822b90cb76412b94540e0f",
      "sha256": "287b8753a398536b063fa54741bcd42c440535a16f7369e9d453fbb166ecfcfd"
    }   
  }
}

Below is a screenshot of a sample Kibana dashboard showing an overview of the file integrity monitoring data.

Kibana dashboard

Seeing Isn't Believing

You have seen some examples of what Auditbeat can do, but you really should try it for yourself. You can download the Auditbeat beta version from here.

Please become an Elastic Pioneer -- when you are trying Auditbeat give us your feedback for an opportunity to win some cool Elastic swag.

Image Credit: Nick Youngson