« Custom Threat Intelligence integration EclecticIQ Integration »

› ›

Cybersixgill Darkfeed TAXII Integration

Version	1.33.0 (View all)
Compatible Kibana version(s)	8.13.0 or higher 9.0.0 or higher
Supported Serverless project types What’s this?	Security Observability
Subscription level What’s this?	Basic
Level of support What’s this?	Elastic

This integration connects with the commercial Cybersixgill Darkfeed TAXII server.

Logs

edit

Threat

edit

The Cybersixgill Darkfeed integration collects threat intelligence from the Darkfeed TAXII service available using the credentials provided from Cybersixgill.

Expiration of Indicators of Compromise (IOCs)

edit

The ingested IOCs are expired after the duration configured by IOC Expiration Duration integration setting. An Elastic Transform is created to faciliate only active IOCs be available to the end users. This transform creates destination indices named logs-ti_cybersixgill_latest.dest_threat-* which only contains active and unexpired IOCs. The latest destination index also has an alias named logs-ti_cybersixgill_latest.threat. When querying for active indicators or setting up indicator match rules, only use the latest destination indices or the alias to avoid false positives from expired IOCs. Dashboards are also pointing to the latest destination indices containing active IOC. Please read ILM Policy below which is added to avoid unbounded growth on source datastream .ds-logs-ti_cybersixgill.threat-* indices.

ILM Policy

edit

To facilitate IOC expiration, source datastream-backed indices .ds-logs-ti_cybersixgill.threat-* are allowed to contain duplicates from each polling interval. ILM policy logs-ti_cybersixgill.threat-default_policy is added to these source indices so it doesn’t lead to unbounded growth. This means data in these source indices will be deleted after 5 days from ingested date.

Exported fields

Field	Description	Type
@timestamp	Event timestamp.	date
cloud.image.id	Image ID for the cloud instance.	keyword
cybersixgill.actor	The related actor for the indicator.	keyword
cybersixgill.deleted_at	The timestamp when indicator is (or will be) expired.	date
cybersixgill.expiration_duration	The configured expiration duration.	keyword
cybersixgill.feedname	Name of the Threat Intel feed.	keyword
cybersixgill.mitre.description	The mitre description of the indicator	keyword
cybersixgill.title	The title of the indicator.	keyword
cybersixgill.valid_from	At what date the indicator is valid from.	date
cybersixgill.virustotal.pr	The Virustotal positive rate.	keyword
cybersixgill.virustotal.url	The related Virustotal URL.	keyword
data_stream.dataset	Data stream dataset name.	constant_keyword
data_stream.namespace	Data stream namespace.	constant_keyword
data_stream.type	Data stream type.	constant_keyword
dataset.name	Dataset name.	constant_keyword
dataset.namespace	Dataset namespace.	constant_keyword
dataset.type	Dataset type.	constant_keyword
event.dataset	Event dataset	constant_keyword
event.module	Event module	constant_keyword
host.containerized	If the host is a container.	boolean
host.os.build	OS build information.	keyword
host.os.codename	OS codename, if any.	keyword
input.type	Input type.	keyword
labels.is_ioc_transform_source	Indicates whether an IOC is in the raw source data stream, or the in latest destination index.	constant_keyword
threat.feed.dashboard_id	Dashboard ID used for Kibana CTI UI	constant_keyword
threat.feed.name	Display friendly feed name	constant_keyword
threat.indicator.first_seen	The date and time when intelligence source first reported sighting this indicator.	date
threat.indicator.last_seen	The date and time when intelligence source last reported sighting this indicator.	date
threat.indicator.modified_at	The date and time when intelligence source last modified information for this indicator.	date

Example

An example event for threat looks as following:

{
    "@timestamp": "2021-12-07T13:58:01.596Z",
    "agent": {
        "ephemeral_id": "9c2e1e11-18ae-413f-9523-290c561a4b61",
        "id": "8299ae35-ee0e-4107-9acb-1b6acfdda1fb",
        "name": "docker-fleet-agent",
        "type": "filebeat",
        "version": "8.13.0"
    },
    "cybersixgill": {
        "actor": "vaedzy",
        "deleted_at": "2021-12-17T13:58:01.596Z",
        "expiration_duration": "10d",
        "feedname": "dark_web_hashes",
        "mitre": {
            "description": "Mitre attack tactics and technique reference"
        },
        "title": "[病毒样本] #Trickbot (2021-12-07)",
        "virustotal": {
            "pr": "medium",
            "url": "https://virustotal.com/#/file/7bdf8b8594ec269da864ee662334f4da53d4820a3f0f8aa665a0fa096ca8f22d"
        }
    },
    "data_stream": {
        "dataset": "ti_cybersixgill.threat",
        "namespace": "78677",
        "type": "logs"
    },
    "ecs": {
        "version": "8.11.0"
    },
    "elastic_agent": {
        "id": "8299ae35-ee0e-4107-9acb-1b6acfdda1fb",
        "snapshot": false,
        "version": "8.13.0"
    },
    "event": {
        "agent_id_status": "verified",
        "category": [
            "threat"
        ],
        "created": "2024-08-02T04:14:26.522Z",
        "dataset": "ti_cybersixgill.threat",
        "ingested": "2024-08-02T04:14:27Z",
        "kind": "enrichment",
        "original": "{\"confidence\":70,\"created\":\"2021-12-07T13:58:01.596Z\",\"description\":\"Hash attributed to malware that was discovered in the dark and deep web\",\"extensions\":{\"extension-definition--3de9ff00-174d-4d41-87c9-05a27a7e117c\":{\"extension_type\":\"toplevel-property-extension\"}},\"external_references\":[{\"positive_rate\":\"medium\",\"source_name\":\"VirusTotal\",\"url\":\"https://virustotal.com/#/file/7bdf8b8594ec269da864ee662334f4da53d4820a3f0f8aa665a0fa096ca8f22d\"},{\"description\":\"Mitre attack tactics and technique reference\",\"mitre_attack_tactic\":\"Build Capabilities\",\"mitre_attack_tactic_id\":\"TA0024\",\"mitre_attack_tactic_url\":\"https://attack.mitre.org/tactics/TA0024/\",\"source_name\":\"mitre-attack\"}],\"id\":\"indicator--302dab0f-64dc-42f5-b99e-702b28c1aaa9\",\"indicator_types\":[\"malicious-activity\"],\"lang\":\"en\",\"modified\":\"2021-12-07T13:58:01.596Z\",\"name\":\"4d0f21919d623bd1631ee15ca7429f28;5ce39ef0700b64bd0c71b55caf64ae45d8400965;7bdf8b8594ec269da864ee662334f4da53d4820a3f0f8aa665a0fa096ca8f22d\",\"pattern\":\"[file:hashes.MD5 = '4d0f21919d623bd1631ee15ca7429f28' OR file:hashes.'SHA-1' = '5ce39ef0700b64bd0c71b55caf64ae45d8400965' OR file:hashes.'SHA-256' = '7bdf8b8594ec269da864ee662334f4da53d4820a3f0f8aa665a0fa096ca8f22d']\",\"pattern_type\":\"stix\",\"sixgill_actor\":\"vaedzy\",\"sixgill_confidence\":70,\"sixgill_feedid\":\"darkfeed_012\",\"sixgill_feedname\":\"dark_web_hashes\",\"sixgill_post_virustotallink\":\"https://virustotal.com/#/file/7bdf8b8594ec269da864ee662334f4da53d4820a3f0f8aa665a0fa096ca8f22d\",\"sixgill_postid\":\"c0c9a0085fb5281cfb40a0ddb62e1d2c6a53eb7a\",\"sixgill_posttitle\":\"[病毒样本] #Trickbot (2021-12-07)\",\"sixgill_severity\":70,\"sixgill_source\":\"forum_kafan\",\"spec_version\":\"2.1\",\"type\":\"indicator\",\"valid_from\":\"2021-12-07T02:55:17Z\"}",
        "severity": 70,
        "type": [
            "indicator"
        ]
    },
    "input": {
        "type": "httpjson"
    },
    "tags": [
        "preserve_original_event",
        "forwarded",
        "ti_cybersixgill"
    ],
    "threat": {
        "indicator": {
            "confidence": "High",
            "description": "Hash attributed to malware that was discovered in the dark and deep web",
            "file": {
                "hash": {
                    "md5": "4d0f21919d623bd1631ee15ca7429f28",
                    "sha1": "5ce39ef0700b64bd0c71b55caf64ae45d8400965",
                    "sha256": "7bdf8b8594ec269da864ee662334f4da53d4820a3f0f8aa665a0fa096ca8f22d"
                }
            },
            "first_seen": "2021-12-07T02:55:17.000Z",
            "last_seen": "2021-12-07T13:58:01.596Z",
            "name": "7bdf8b8594ec269da864ee662334f4da53d4820a3f0f8aa665a0fa096ca8f22d",
            "provider": "forum_kafan",
            "reference": "https://portal.cybersixgill.com/#/search?q=_id:c0c9a0085fb5281cfb40a0ddb62e1d2c6a53eb7a",
            "type": "file"
        },
        "tactic": {
            "id": [
                "TA0024"
            ],
            "name": [
                "Build Capabilities"
            ],
            "reference": [
                "https://attack.mitre.org/tactics/TA0024/"
            ]
        }
    }
}

Changelog

edit

Changelog

Version	Details	Kibana version(s)
1.33.0	Enhancement (View pull request) Update Kibana constraint to support 9.0.0.	8.13.0 or higher 9.0.0 or higher
1.32.1	Bug fix (View pull request) Updated SSL description to be uniform and to include links to documentation.	8.13.0 or higher
1.32.0	Enhancement (View pull request) Do not remove `event.original` in main ingest pipeline.	8.13.0 or higher
1.31.1	Bug fix (View pull request) Fix broken links in Security Service integrations packages.	8.13.0 or higher
1.31.0	Enhancement (View pull request) Add "preserve_original_event" tag to documents with `event.kind` set to "pipeline_error".	8.13.0 or higher
1.30.5	Bug fix (View pull request) Use triple-brace Mustache templating when referencing variables in ingest pipelines.	8.13.0 or higher
1.30.4	Bug fix (View pull request) Use triple-brace Mustache templating when referencing variables in ingest pipelines.	8.13.0 or higher
1.30.3	Bug fix (View pull request) Fix labels.is_ioc_transform_source values	8.13.0 or higher
1.30.2	Bug fix (View pull request) Add missing fields in transform	8.13.0 or higher
1.30.1	Bug fix (View pull request) Fix ECS date mapping on threat fields.	8.13.0 or higher
1.30.0	Enhancement (View pull request) Update the kibana constraint to ^8.13.0. Modified the field definitions to remove ECS fields made redundant by the ecs@mappings component template.	8.13.0 or higher
1.29.1	Bug fix (View pull request) Fix sample event.	8.12.0 or higher
1.29.0	Enhancement (View pull request) Make `event.type` field conform to ECS field definition.	8.12.0 or higher
1.28.1	Bug fix (View pull request) Adjust field mappings for transform destination index.	8.12.0 or higher
1.28.0	Enhancement (View pull request) Improve handling of empty responses.	8.12.0 or higher
1.27.0	Enhancement (View pull request) Add support for IOC expiration.	8.12.0 or higher
1.26.0	Enhancement (View pull request) Set sensitive values as secret.	8.12.0 or higher
1.25.1	Enhancement (View pull request) Changed owners	8.7.1 or higher
1.25.0	Enhancement (View pull request) Limit request tracer log count to five.	8.7.1 or higher
1.24.0	Enhancement (View pull request) ECS version updated to 8.11.0.	8.7.1 or higher
1.23.0	Enhancement (View pull request) Improve event.original check to avoid errors if set.	8.7.1 or higher
1.22.0	Enhancement (View pull request) ECS version updated to 8.10.0.	8.7.1 or higher
1.21.0	Enhancement (View pull request) The format_version in the package manifest changed from 2.11.0 to 3.0.0. Removed dotted YAML keys from package manifest. Added owner.type: elastic to package manifest.	8.7.1 or higher
1.20.0	Enhancement (View pull request) Add tags.yml file so that integration’s dashboards and saved searches are tagged with "Security Solution" and displayed in the Security Solution UI.	8.7.1 or higher
1.19.0	Enhancement (View pull request) Update package-spec to 2.10.0.	8.7.1 or higher
1.18.0	Enhancement (View pull request) Update package to ECS 8.9.0.	8.7.1 or higher
1.17.0	Enhancement (View pull request) Document duration units.	8.7.1 or higher
1.16.0	Enhancement (View pull request) Convert visualizations to lens.	8.7.1 or higher
1.15.0	Enhancement (View pull request) Document valid duration units.	8.7.1 or higher
1.14.0	Enhancement (View pull request) Document valid duration units.	—
1.13.0	Enhancement (View pull request) Ensure event.kind is correctly set for pipeline errors.	8.7.1 or higher
1.12.0	Enhancement (View pull request) Update package to ECS 8.8.0.	8.7.1 or higher
1.11.0	Enhancement (View pull request) Add a new flag to enable request tracing	8.7.1 or higher
1.10.1	Bug fix (View pull request) Fix ingest pipeline to map threat type correctly	8.0.0 or higher
1.10.0	Enhancement (View pull request) Update package to ECS 8.7.0.	8.0.0 or higher
1.9.0	Enhancement (View pull request) Update package to ECS 8.6.0.	8.0.0 or higher
1.8.1	Bug fix (View pull request) Remove duplicate fields.	8.0.0 or higher
1.8.0	Enhancement (View pull request) Update package to ECS 8.5.0.	8.0.0 or higher
1.7.0	Enhancement (View pull request) Update package to ECS 8.4.0	8.0.0 or higher
1.6.1	Bug fix (View pull request) Fix proxy URL documentation rendering.	8.0.0 or higher
1.6.0	Enhancement (View pull request) Update categories to include `threat_intel`.	8.0.0 or higher
1.5.0	Enhancement (View pull request) Update package to ECS 8.3.0.	8.0.0 or higher
1.4.1	Enhancement (View pull request) Update package descriptions	8.0.0 or higher
1.4.0	Enhancement (View pull request) Update to ECS 8.2	8.0.0 or higher
1.3.2	Enhancement (View pull request) Adding field mapping for event.created	8.0.0 or higher
1.3.1	Enhancement (View pull request) Add documentation for multi-fields	8.0.0 or higher
1.3.0	Enhancement (View pull request) Moving integration to use the TAXII service rather than python scripts	8.0.0 or higher
1.2.0	Enhancement (View pull request) Update to ECS 8.0	8.0.0 or higher
1.1.0	Enhancement (View pull request) Adds dashboards, new logo and new threat ECS fields	8.0.0 or higher
1.0.0	Enhancement (View pull request) initial implementation	8.0.0 or higher

« Custom Threat Intelligence integration EclecticIQ Integration »

On this page

Logs
Threat
Expiration of Indicators of Compromise (IOCs)
ILM Policy
Changelog

Was this helpful?

Feedback

The Search AI Company

ELK Stack

Elastic Cloud

Generative AI

Search

Security

Observability

By solution

Industries

Customer spotlight

Research

Build

Learn

Connect

Cybersixgill Darkfeed TAXII Integration