›

System Audit Integration [Beta]

Version	1.10.3 (View all)
Compatible Kibana version(s)	8.7.1 or higher
Supported Serverless project types What’s this?	Security Observability
Subscription level What’s this?	Basic
Level of support What’s this?	Elastic

Overview

edit

The System Audit integration collects various security-related information about a system. All data streams send both periodic state information (e.g. all currently installed packages) and real-time changes (e.g. when a new package is installed/uninstalled or an existing package is updated). Currently, the only implemented data stream is the package data stream, which collects various information about system packages. In the future, more data streams like (process, socket, hosts .. etc) will be added.

How it works

edit

Each data stream sends two kinds of information: state and events.

State information is sent periodically. A state update will consist of events for each package that is installed or has had its state change in the polling period. All events belonging to the same state update will share the same UUID in event.id.

The frequency of state updates can be controlled for all data streams using the state.period configuration option. The default is 12h.

Event information is sent as the events occur (e.g. a package is installed, uninstalled or updated). All data streams are currently using a poll model to retrieve their data. The frequency of these polls is controlled by the period configuration parameter.

Entity IDs

edit

This module populates entity_id fields to uniquely identify entities (packages) within a host. This requires the module to obtain a unique identifier for the host:

macOS: Uses the value returned by gethostuuid(2) system call.
Linux: Uses the content of one of the following files, created by either systemd or dbus:
/etc/machine-id
/var/lib/dbus/machine-id
/var/db/dbus/machine-id

Under CentOS 6.x, it’s possible that none of the files above exist. In that case, running dbus-uuidgen --ensure (provided by the dbus package) will generate one for you. One more thing to consider is that at the moment this integration is not supported on Windows systems.

Requirements

edit

You need Elasticsearch for storing and searching your data and Kibana for visualizing and managing it. You can use our hosted Elasticsearch Service on Elastic Cloud, which is recommended, or self-manage the Elastic Stack on your own hardware.

Setup

edit

For step-by-step instructions on how to set up an integration, see the Getting Started guide.

If you want to supress host related information, please consider adding the tag: forwarded. Adding this tag to the tag list will remove any host-related data from the output, this will also stop certain dashboards from displaying host/os-related information/charts.

Data Streams

edit

The data streams which are currently supported are:-

package

Package helps you keep a record of events and changes happening to different packages on your system. The fields & events associated with the data stream are as follows:-

Exported fields

Field	Description	Type
@timestamp	Date/time when the event originated. This is the date/time extracted from the event, typically representing when the event was generated by the source. If the event source has no original timestamp, this value is typically populated by the first time the event was received by the pipeline. Required field for all events.	date
data_stream.dataset	The field can contain anything that makes sense to signify the source of the data. Examples include `nginx.access`, `prometheus`, `endpoint` etc. For data streams that otherwise fit, but that do not have dataset set we use the value "generic" for the dataset value. `event.dataset` should have the same value as `data_stream.dataset`. Beyond the Elasticsearch data stream naming criteria noted above, the `dataset` value has additional restrictions: * Must not contain `-` * No longer than 100 characters	constant_keyword
data_stream.namespace	A user defined namespace. Namespaces are useful to allow grouping of data. Many users already organize their indices this way, and the data stream naming scheme now provides this best practice as a default. Many users will populate this field with `default`. If no value is used, it falls back to `default`. Beyond the Elasticsearch index naming criteria noted above, `namespace` value has the additional restrictions: * Must not contain `-` * No longer than 100 characters	constant_keyword
data_stream.type	An overarching type for the data stream. Currently allowed values are "logs" and "metrics". We expect to also add "traces" and "synthetics" in the near future.	constant_keyword
ecs.version	ECS version this event conforms to. `ecs.version` is a required field and must exist in all events. When querying across multiple indices — which may conform to slightly different ECS versions — this field lets integrations adjust to the schema version of the events.	keyword
event.dataset	Event dataset	constant_keyword
event.module	Event module	constant_keyword
event.original	Raw text message of entire event. Used to demonstrate log integrity or where the full log message (before splitting it up in multiple parts) may be required, e.g. for reindex. This field is not indexed and doc_values are disabled. It cannot be searched, but it can be retrieved from `_source`. If users wish to override this and index this field, please see `Field data types` in the `Elasticsearch Reference`.	keyword
host.architecture	Operating system architecture.	keyword
host.domain	Name of the domain of which the host is a member. For example, on Windows this could be the host’s Active Directory domain or NetBIOS domain name. For Linux this could be the domain of the host’s LDAP provider.	keyword
host.hostname	Hostname of the host. It normally contains what the `hostname` command returns on the host machine.	keyword
host.id	Unique host id. As hostname is not always unique, use values that are meaningful in your environment. Example: The current usage of `beat.name`.	keyword
host.ip	Host ip addresses.	ip
host.mac	Host MAC addresses. The notation format from RFC 7042 is suggested: Each octet (that is, 8-bit byte) is represented by two [uppercase] hexadecimal digits giving the value of the octet as an unsigned integer. Successive octets are separated by a hyphen.	keyword
host.name	Name of the host. It can contain what hostname returns on Unix systems, the fully qualified domain name (FQDN), or a name specified by the user. The recommended value is the lowercase FQDN of the host.	keyword
host.os.family	OS family (such as redhat, debian, freebsd, windows).	keyword
host.os.kernel	Operating system kernel version as a raw string.	keyword
host.os.name	Operating system name, without the version.	keyword
host.os.name.text	Multi-field of `host.os.name`.	match_only_text
host.os.platform	Operating system platform (such centos, ubuntu, windows).	keyword
host.os.version	Operating system version as a raw string.	keyword
host.type	Type of host. For Cloud providers this can be the machine type like `t2.medium`. If vm, this could be the container, for example, or other information meaningful in your environment.	keyword
input.type	Type of Auditbeat input.	keyword
log.level	Original log level of the log event. If the source of the event provides a log level or textual severity, this is the one that goes in `log.level`. If your source doesn’t specify one, you may put your event transport’s severity here (e.g. Syslog severity). Some examples are `warn`, `err`, `i`, `informational`.	keyword
message	For log events the message field contains the log message, optimized for viewing in a log viewer. For structured logs without an original message field, other fields can be concatenated to form a human-readable summary of the event. If multiple messages exist, they can be combined into one message.	match_only_text
package.architecture	Package architecture.	keyword
package.build_version	Additional information about the build version of the installed package. For example use the commit SHA of a non-released package.	keyword
package.checksum	Checksum of the installed package for verification.	keyword
package.description	Description of the package.	keyword
package.install_scope	Indicating how the package was installed, e.g. user-local, global.	keyword
package.installed	Time when package was installed.	date
package.license	License under which the package was released. Use a short name, e.g. the license identifier from SPDX License List where possible (https://spdx.org/licenses/).	keyword
package.name	Package name	keyword
package.path	Path where the package is installed.	keyword
package.reference	Home page or reference URL of the software in this package, if available.	keyword
package.size	Package size in bytes.	long
package.type	Type of package. This should contain the package file type, rather than the package manager name. Examples: rpm, dpkg, brew, npm, gem, nupkg, jar.	keyword
package.version	Package version	keyword
system_audit.package.arch	Package architecture.	keyword
system_audit.package.entity_id	ID uniquely identifying the package. It is computed as a SHA-256 hash of the host ID, package name, and package version.	keyword
system_audit.package.installtime	Package install time.	date
system_audit.package.license	Package license.	keyword
system_audit.package.name	Package name.	keyword
system_audit.package.release	Package release.	keyword
system_audit.package.size	Package size.	long
system_audit.package.summary	Package summary.
system_audit.package.url	Package URL.	keyword
system_audit.package.version	Package version.	keyword
tags	User defined tags	keyword

Example

An example event for package looks as following:

{
    "@timestamp": "2023-01-31T11:44:38.695Z",
    "agent": {
        "ephemeral_id": "c9a6d8c0-780c-4b96-81f2-5a8c850bd0cc",
        "id": "027bc354-85a6-40d6-be9d-7eb4533fbd18",
        "name": "docker-fleet-agent",
        "type": "auditbeat",
        "version": "8.5.1"
    },
    "data_stream": {
        "dataset": "system_audit.package",
        "namespace": "ep",
        "type": "logs"
    },
    "ecs": {
        "version": "8.11.0"
    },
    "elastic_agent": {
        "id": "027bc354-85a6-40d6-be9d-7eb4533fbd18",
        "snapshot": false,
        "version": "8.5.1"
    },
    "event": {
        "action": "existing_package",
        "agent_id_status": "verified",
        "category": [
            "package"
        ],
        "dataset": "system_audit.package",
        "id": "f2b5baf6-fd22-490a-82fd-a044ff7075cb",
        "ingested": "2023-01-31T11:44:40Z",
        "kind": "state",
        "module": "system",
        "type": [
            "info"
        ]
    },
    "host": {
        "architecture": "x86_64",
        "containerized": false,
        "hostname": "docker-fleet-agent",
        "id": "75e38940166b4dbc90b6f5610e8e9c39",
        "ip": [
            "192.168.80.7"
        ],
        "mac": [
            "02-42-C0-A8-50-07"
        ],
        "name": "docker-fleet-agent",
        "os": {
            "codename": "focal",
            "family": "debian",
            "kernel": "5.15.49-linuxkit",
            "name": "Ubuntu",
            "platform": "ubuntu",
            "type": "linux",
            "version": "20.04.5 LTS (Focal Fossa)"
        }
    },
    "input": {
        "type": "audit/system"
    },
    "message": "Package adduser (3.118ubuntu2) is already installed",
    "package": {
        "architecture": "all",
        "description": "add and remove users and groups",
        "name": "adduser",
        "size": 624,
        "type": "dpkg",
        "version": "3.118ubuntu2"
    },
    "system_audit": {
        "package": {
            "arch": "all",
            "entity_id": "OnUSNhuUQkyYgoKf",
            "name": "adduser",
            "size": 624,
            "summary": "add and remove users and groups",
            "version": "3.118ubuntu2"
        }
    },
    "tags": [
        "audit-system-package"
    ]
}

Example dashboard

edit

The integration comes with a package & audit system dashboard for easy identification of events and data overview :

Package Dashboard:

System Audit Dashboard:

Reference

edit

For further information, please look at the Auditbeat System Module documentation.

Changelog

edit

Changelog

Version	Details	Kibana version(s)
1.10.3	Bug fix (View pull request) Use triple-brace Mustache templating when referencing variables in ingest pipelines.	8.7.1 or higher
1.10.2	Enhancement (View pull request) capture root requirement	8.7.1 or higher
1.10.1	Enhancement (View pull request) Changed owners	8.7.1 or higher
1.10.0	Enhancement (View pull request) ECS version updated to 8.11.0.	8.7.1 or higher
1.9.0	Enhancement (View pull request) ECS version updated to 8.10.0.	8.7.1 or higher
1.8.0	Enhancement (View pull request) The format_version in the package manifest changed from 2.11.0 to 3.0.0. Removed dotted YAML keys from package manifest. Added owner.type: elastic to package manifest.	8.7.1 or higher
1.7.0	Enhancement (View pull request) Add tags.yml file so that integration’s dashboards and saved searches are tagged with "Security Solution" and displayed in the Security Solution UI.	8.7.1 or higher
1.6.0	Enhancement (View pull request) Update package to ECS 8.9.0.	8.7.1 or higher
1.5.1	Bug fix (View pull request) Fix processor UI handling.	8.7.1 or higher
1.5.0	Enhancement (View pull request) Convert dashboards to Lens.	8.7.1 or higher
1.4.0	Enhancement (View pull request) Ensure event.kind is correctly set for pipeline errors.	8.5.0 or higher
1.3.0	Enhancement (View pull request) Update package to ECS 8.8.0.	8.5.0 or higher
1.2.1	Bug fix (View pull request) Improved the documentation and explicitly mentioned that Windows is not supported.	8.5.0 or higher
1.2.0	Enhancement (View pull request) Update package-spec version to 2.7.0.	8.5.0 or higher
1.1.0	Enhancement (View pull request) Update package to ECS 8.7.0.	8.5.0 or higher
1.0.1	Enhancement (View pull request) Fix documentation bug	8.5.0 or higher
1.0.0	Enhancement (View pull request) Initial Release	8.5.0 or higher

« System Integration Tanium »