« Juniper SRX integration StatsD input »
Elastic Docs ›Elastic integrations

STAN integration

edit

STAN integration

edit

Version

1.7.0 (View all)

Compatible Kibana version(s)

8.13.0 or higher

Supported Serverless project types
What’s this?

Security
Observability

Subscription level
What’s this?

Basic

This integration is used to collect logs and metrics from STAN servers. The integration collects metrics from STAN monitoring server APIs.

Compatibility

edit

The STAN package is tested with Stan 0.15.1.

Logs

edit

log

edit

The log dataset collects the STAN logs.

Example

An example event for log looks as following:

{
    "@timestamp": "2024-06-18T10:41:22.515Z",
    "agent": {
        "ephemeral_id": "e925a178-f704-43e6-899e-b69b164425d8",
        "id": "21652d23-59b4-4d65-a341-ede15f919642",
        "name": "docker-fleet-agent",
        "type": "filebeat",
        "version": "8.13.0"
    },
    "data_stream": {
        "dataset": "stan.log",
        "namespace": "ep",
        "type": "logs"
    },
    "ecs": {
        "version": "8.11.0"
    },
    "elastic_agent": {
        "id": "21652d23-59b4-4d65-a341-ede15f919642",
        "snapshot": false,
        "version": "8.13.0"
    },
    "event": {
        "agent_id_status": "verified",
        "created": "2024-06-18T10:41:36.382Z",
        "dataset": "stan.log",
        "ingested": "2024-06-18T10:41:38Z",
        "kind": "event",
        "type": [
            "info"
        ]
    },
    "input": {
        "type": "log"
    },
    "log": {
        "file": {
            "path": "/tmp/service_logs/stan.log"
        },
        "level": "info",
        "offset": 0
    },
    "message": "STREAM: Starting nats-streaming-server[test-cluster] version 0.15.1",
    "process": {
        "pid": 7
    },
    "stan": {
        "log": {
            "msg": {}
        }
    },
    "tags": [
        "forwarded",
        "stan-log"
    ]
}

ECS Field Reference

Please refer to the following document for detailed information on ECS fields.

Exported fields
Field Description Type

@timestamp

Event timestamp.

date

data_stream.dataset

Data stream dataset.

constant_keyword

data_stream.namespace

Data stream namespace.

constant_keyword

data_stream.type

Data stream type.

constant_keyword

event.dataset

Event dataset

constant_keyword

event.module

Event module

constant_keyword

input.type

Type of Filebeat input.

keyword

log.offset

Offset of the entry in the log file.

long

stan.log.client.id

The id of the client

integer

stan.log.msg.bytes

Size of the payload in bytes

long

stan.log.msg.error.message

Details about the error occurred

text

stan.log.msg.max_messages

An optional number of messages to wait for before automatically unsubscribing

integer

stan.log.msg.queue_group

The queue group which subscriber will join

text

stan.log.msg.reply_to

The inbox subject on which the publisher is listening for responses

keyword

stan.log.msg.sid

The unique alphanumeric subscription ID of the subject

integer

stan.log.msg.subject

Subject name this message was received on

keyword

stan.log.msg.type

The protocol message type

keyword

Metrics

edit

The default datasets are stats, channels, and subscriptions.

stats

edit

This is the stats dataset of the STAN package, in charge of retrieving generic metrics from a STAN instance.

Example

An example event for stats looks as following:

{
    "@timestamp": "2024-06-18T10:42:35.470Z",
    "agent": {
        "ephemeral_id": "b30d1b07-36f0-4e5f-9f80-56b6015fb855",
        "id": "21652d23-59b4-4d65-a341-ede15f919642",
        "name": "docker-fleet-agent",
        "type": "metricbeat",
        "version": "8.13.0"
    },
    "data_stream": {
        "dataset": "stan.stats",
        "namespace": "ep",
        "type": "metrics"
    },
    "ecs": {
        "version": "8.0.0"
    },
    "elastic_agent": {
        "id": "21652d23-59b4-4d65-a341-ede15f919642",
        "snapshot": false,
        "version": "8.13.0"
    },
    "event": {
        "agent_id_status": "verified",
        "dataset": "stan.stats",
        "duration": 1823495,
        "ingested": "2024-06-18T10:42:47Z",
        "module": "stan"
    },
    "host": {
        "architecture": "x86_64",
        "containerized": true,
        "hostname": "docker-fleet-agent",
        "id": "8259e024976a406e8a54cdbffeb84fec",
        "ip": "192.168.252.5",
        "mac": "02-42-C0-A8-FC-05",
        "name": "docker-fleet-agent",
        "os": {
            "codename": "focal",
            "family": "debian",
            "kernel": "3.10.0-1160.102.1.el7.x86_64",
            "name": "Ubuntu",
            "platform": "ubuntu",
            "type": "linux",
            "version": "20.04.6 LTS (Focal Fossa)"
        }
    },
    "metricset": {
        "name": "stats",
        "period": 60000
    },
    "service": {
        "address": "http://elastic-package-service-stan-1:8222/streaming/serverz",
        "type": "stan"
    },
    "stan": {
        "cluster": {
            "id": "test-cluster"
        },
        "server": {
            "id": "I9vNI3muOuNoem5vuoyo5z"
        },
        "stats": {
            "bytes": 0,
            "channels": 1,
            "clients": 100,
            "messages": 0,
            "state": "STANDALONE",
            "subscriptions": 100
        }
    }
}

ECS Field Reference

Please refer to the following document for detailed information on ECS fields.

Exported fields
Field Description Type Metric Type

@timestamp

Event timestamp.

date

agent.id

Unique identifier of this agent (if one exists). Example: For Beats this would be beat.id.

keyword

cloud.account.id

The cloud account or organization id used to identify different entities in a multi-tenant environment. Examples: AWS account id, Google Cloud ORG Id, or other unique identifier.

keyword

cloud.availability_zone

Availability zone in which this host, resource, or service is located.

keyword

cloud.instance.id

Instance ID of the host machine.

keyword

cloud.provider

Name of the cloud provider. Example values are aws, azure, gcp, or digitalocean.

keyword

cloud.region

Region in which this host, resource, or service is located.

keyword

container.id

Unique container id.

keyword

data_stream.dataset

Data stream dataset.

constant_keyword

data_stream.namespace

Data stream namespace.

constant_keyword

data_stream.type

Data stream type.

constant_keyword

event.dataset

Event dataset

constant_keyword

event.module

Event module

constant_keyword

host.name

Name of the host. It can contain what hostname returns on Unix systems, the fully qualified domain name (FQDN), or a name specified by the user. The recommended value is the lowercase FQDN of the host.

keyword

service.address

Address where data about this service was collected from. This should be a URI, network address (ipv4:port or [ipv6]:port) or a resource path (sockets).

keyword

stan.cluster.id

The cluster ID

keyword

stan.server.id

The server ID

keyword

stan.stats.bytes

Number of bytes consumed across all STAN queues

long

counter

stan.stats.channels

The number of STAN channels

integer

gauge

stan.stats.clients

The number of STAN clients

integer

gauge

stan.stats.messages

Number of messages across all STAN queues

long

counter

stan.stats.role

If clustered, role of this node in the cluster (Leader, Follower, Candidate)

keyword

stan.stats.state

The cluster / streaming configuration state (STANDALONE, CLUSTERED)

keyword

stan.stats.subscriptions

The number of STAN streaming subscriptions

integer

gauge

channels

edit

This is the channels dataset of the STAN package, in charge of retrieving metrics about channels from a STAN instance.

Example

An example event for channels looks as following:

{
    "@timestamp": "2024-06-18T10:40:33.161Z",
    "agent": {
        "ephemeral_id": "2f2befc0-8f03-49d7-b5bf-8131709857b6",
        "id": "21652d23-59b4-4d65-a341-ede15f919642",
        "name": "docker-fleet-agent",
        "type": "metricbeat",
        "version": "8.13.0"
    },
    "data_stream": {
        "dataset": "stan.channels",
        "namespace": "ep",
        "type": "metrics"
    },
    "ecs": {
        "version": "8.0.0"
    },
    "elastic_agent": {
        "id": "21652d23-59b4-4d65-a341-ede15f919642",
        "snapshot": false,
        "version": "8.13.0"
    },
    "event": {
        "agent_id_status": "verified",
        "dataset": "stan.channels",
        "duration": 3556501,
        "ingested": "2024-06-18T10:40:45Z",
        "module": "stan"
    },
    "host": {
        "architecture": "x86_64",
        "containerized": true,
        "hostname": "docker-fleet-agent",
        "id": "8259e024976a406e8a54cdbffeb84fec",
        "ip": "192.168.252.5",
        "mac": "02-42-C0-A8-FC-05",
        "name": "docker-fleet-agent",
        "os": {
            "codename": "focal",
            "family": "debian",
            "kernel": "3.10.0-1160.102.1.el7.x86_64",
            "name": "Ubuntu",
            "platform": "ubuntu",
            "type": "linux",
            "version": "20.04.6 LTS (Focal Fossa)"
        }
    },
    "metricset": {
        "name": "channels",
        "period": 60000
    },
    "service": {
        "address": "http://elastic-package-service-stan-1:8222/streaming/channelsz?subs=1",
        "type": "stan"
    },
    "stan": {
        "channels": {
            "bytes": 0,
            "depth": 0,
            "first_seq": 0,
            "last_seq": 0,
            "messages": 0,
            "name": "foo"
        },
        "cluster": {
            "id": "test-cluster"
        },
        "server": {
            "id": "KJXiShxwxsi2oNAUH1Rlzn"
        }
    }
}

ECS Field Reference

Please refer to the following document for detailed information on ECS fields.

Exported fields
Field Description Type Metric Type

@timestamp

Event timestamp.

date

agent.id

Unique identifier of this agent (if one exists). Example: For Beats this would be beat.id.

keyword

cloud.account.id

The cloud account or organization id used to identify different entities in a multi-tenant environment. Examples: AWS account id, Google Cloud ORG Id, or other unique identifier.

keyword

cloud.availability_zone

Availability zone in which this host, resource, or service is located.

keyword

cloud.instance.id

Instance ID of the host machine.

keyword

cloud.provider

Name of the cloud provider. Example values are aws, azure, gcp, or digitalocean.

keyword

cloud.region

Region in which this host, resource, or service is located.

keyword

container.id

Unique container id.

keyword

data_stream.dataset

Data stream dataset.

constant_keyword

data_stream.namespace

Data stream namespace.

constant_keyword

data_stream.type

Data stream type.

constant_keyword

event.dataset

Event dataset

constant_keyword

event.module

Event module

constant_keyword

host.name

Name of the host. It can contain what hostname returns on Unix systems, the fully qualified domain name (FQDN), or a name specified by the user. The recommended value is the lowercase FQDN of the host.

keyword

service.address

Address where data about this service was collected from. This should be a URI, network address (ipv4:port or [ipv6]:port) or a resource path (sockets).

keyword

stan.channels.bytes

The number of STAN bytes in the channel

long

counter

stan.channels.depth

Queue depth based upon current sequence number and highest reported subscriber sequence number

long

gauge

stan.channels.first_seq

First sequence number stored in the channel. If first_seq > min([seq in subscriptions]) data loss has possibly occurred

long

stan.channels.last_seq

Last sequence number stored in the channel

long

counter

stan.channels.messages

The number of STAN streaming messages

long

counter

stan.channels.name

The name of the STAN streaming channel

keyword

stan.cluster.id

The cluster ID

keyword

stan.server.id

The server ID

keyword

subscriptions

edit

This is the subscriptions dataset of the STAN package, in charge of retrieving metrics about subscriptions from a STAN instance.

Example

An example event for subscriptions looks as following:

{
    "@timestamp": "2024-06-18T10:43:38.692Z",
    "agent": {
        "ephemeral_id": "871b52f0-2644-4638-811c-1b0befe0ee13",
        "id": "21652d23-59b4-4d65-a341-ede15f919642",
        "name": "docker-fleet-agent",
        "type": "metricbeat",
        "version": "8.13.0"
    },
    "data_stream": {
        "dataset": "stan.subscriptions",
        "namespace": "ep",
        "type": "metrics"
    },
    "ecs": {
        "version": "8.0.0"
    },
    "elastic_agent": {
        "id": "21652d23-59b4-4d65-a341-ede15f919642",
        "snapshot": false,
        "version": "8.13.0"
    },
    "event": {
        "agent_id_status": "verified",
        "dataset": "stan.subscriptions",
        "duration": 1947519984,
        "ingested": "2024-06-18T10:43:50Z",
        "module": "stan"
    },
    "host": {
        "architecture": "x86_64",
        "containerized": true,
        "hostname": "docker-fleet-agent",
        "id": "8259e024976a406e8a54cdbffeb84fec",
        "ip": "192.168.252.5",
        "mac": "02-42-C0-A8-FC-05",
        "name": "docker-fleet-agent",
        "os": {
            "codename": "focal",
            "family": "debian",
            "kernel": "3.10.0-1160.102.1.el7.x86_64",
            "name": "Ubuntu",
            "platform": "ubuntu",
            "type": "linux",
            "version": "20.04.6 LTS (Focal Fossa)"
        }
    },
    "metricset": {
        "name": "subscriptions",
        "period": 60000
    },
    "service": {
        "address": "http://elastic-package-service-stan-1:8222/streaming/channelsz?subs=1",
        "type": "stan"
    },
    "stan": {
        "cluster": {
            "id": "test-cluster"
        },
        "server": {
            "id": "SwiO2nzNV8CW27j45QPnFz"
        },
        "subscriptions": {
            "channel": "foo",
            "id": "benchmark-sub-23",
            "last_sent": 0,
            "offline": false,
            "pending": 0,
            "queue": "T",
            "stalled": false
        }
    }
}

ECS Field Reference

Please refer to the following document for detailed information on ECS fields.

Exported fields
Field Description Type Metric Type

@timestamp

Event timestamp.

date

agent.id

Unique identifier of this agent (if one exists). Example: For Beats this would be beat.id.

keyword

cloud.account.id

The cloud account or organization id used to identify different entities in a multi-tenant environment. Examples: AWS account id, Google Cloud ORG Id, or other unique identifier.

keyword

cloud.availability_zone

Availability zone in which this host, resource, or service is located.

keyword

cloud.instance.id

Instance ID of the host machine.

keyword

cloud.provider

Name of the cloud provider. Example values are aws, azure, gcp, or digitalocean.

keyword

cloud.region

Region in which this host, resource, or service is located.

keyword

container.id

Unique container id.

keyword

data_stream.dataset

Data stream dataset.

constant_keyword

data_stream.namespace

Data stream namespace.

constant_keyword

data_stream.type

Data stream type.

constant_keyword

event.dataset

Event dataset

constant_keyword

event.module

Event module

constant_keyword

host.name

Name of the host. It can contain what hostname returns on Unix systems, the fully qualified domain name (FQDN), or a name specified by the user. The recommended value is the lowercase FQDN of the host.

keyword

service.address

Address where data about this service was collected from. This should be a URI, network address (ipv4:port or [ipv6]:port) or a resource path (sockets).

keyword

stan.cluster.id

The cluster ID

keyword

stan.server.id

The server ID

keyword

stan.subscriptions.channel

The name of the STAN channel the subscription is associated with

keyword

stan.subscriptions.id

The name of the STAN channel subscription (client_id)

keyword

stan.subscriptions.last_sent

Last known sequence number of the subscription that was acked

long

counter

stan.subscriptions.offline

Is the subscriber marked as offline?

boolean

stan.subscriptions.pending

Number of pending messages from / to the subscriber

long

gauge

stan.subscriptions.queue

The name of the NATS queue that the STAN channel subscription is associated with, if any

keyword

stan.subscriptions.stalled

Is the subscriber known to be stalled?

boolean

Changelog

edit
Changelog
Version Details Kibana version(s)

1.7.0

Enhancement (View pull request)
ECS version updated to 8.11.0. Update the kibana constraint to ^8.13.0. Modified the field definitions to remove ECS fields made redundant by the ecs@mappings component template.

8.13.0 or higher

1.6.0

Enhancement (View pull request)
Add global filter on data_stream.dataset to improve performance.

8.10.2 or higher

1.5.0

Enhancement (View pull request)
Enable time series data for metrics data streams. This dramatically reduces storage for metrics and is expected to progressively improve query [performance](https://www.elastic.co/blog/70-percent-storage-savings-for-metrics-with-elastic-observability). For more details, see https://www.elastic.co/guide/en/elasticsearch/reference/current/tsds.html.

8.10.2 or higher

1.4.2

Enhancement (View pull request)
Migrate Client IP Count Timeline visualization to lens.

8.10.2 or higher

1.4.1

Enhancement (View pull request)
Add subscriptions data stream metric types mappings.

8.10.2 or higher

1.4.0

Enhancement (View pull request)
Add subscriptions data stream dimensions mappings and ECS fields.

8.10.2 or higher

1.3.9

Enhancement (View pull request)
Add stats data stream metric types mappings.

8.10.2 or higher

1.3.8

Enhancement (View pull request)
Add stats data stream dimensions mappings and ECS fields.

8.10.2 or higher

1.3.7

Enhancement (View pull request)
Add channels data stream metric types mappings.

8.10.2 or higher

1.3.6

Enhancement (View pull request)
Add channels data stream dimensions mappings and ECS fields.

8.10.2 or higher

1.3.5

Enhancement (View pull request)
Migrate Logs Overview dashboard visualizations to lens.

8.10.2 or higher

1.3.4

Enhancement (View pull request)
Migrate Metrics Overview dashboard visualizations to lens.

8.10.2 or higher

1.3.3

Bug fix (View pull request)
Fix reference error in Channel Overview dashboard.

7.14.0 or higher
8.0.0 or higher

1.3.2

Bug fix (View pull request)
Add null check to the rename processor

7.14.0 or higher
8.0.0 or higher

1.3.1

Enhancement (View pull request)
Added categories and/or subcategories.

7.14.0 or higher
8.0.0 or higher

1.3.0

Enhancement (View pull request)
Update to ECS 8.0

7.14.0 or higher
8.0.0 or higher

1.2.0

Enhancement (View pull request)
Release stan package for v8.0.0

7.14.0 or higher
8.0.0 or higher

1.1.2

Enhancement (View pull request)
Uniform with guidelines

—

1.1.1

Bug fix (View pull request)
Fix logic that checks for the forwarded tag

—

1.1.0

Enhancement (View pull request)
Update to ECS 1.12.0

7.14.0 or higher

1.0.0

Enhancement (View pull request)
Release Stan as GA

—

0.5.3

Enhancement (View pull request)
Convert to generated ECS fields

—

0.5.2

Enhancement (View pull request)
update to ECS 1.11.0

—

0.5.1

Enhancement (View pull request)
Escape special characters in docs

—

0.5.0

Enhancement (View pull request)
Update integration description

—

0.4.0

Enhancement (View pull request)
Set "event.module" and "event.dataset"

—

0.3.0

Enhancement (View pull request)
update to ECS 1.10.0 and adding event.original options

—

0.2.0

Bug fix (View pull request)
Fix stack compatability

—

0.1.3

Enhancement (View pull request)
update to ECS 1.9.0

—

0.1.2

Bug fix (View pull request)
Change kibana.version constraint to be more conservative.

—

0.1.0

Enhancement (View pull request)
initial release

—
« Juniper SRX integration StatsD input »