›

Squid Proxy

Version	1.0.3 (View all)
Compatible Kibana version(s)	8.14.1 or higher
Supported Serverless project types What’s this?	Security Observability
Subscription level What’s this?	Basic
Level of support What’s this?	Elastic

Squid is a caching and forwarding HTTP web proxy. Use the Squid Proxy integration to monitor Squid Proxy access logs.

Data streams

edit

This integration supports ingestion of logs from Squid Proxy, via the Filestream, TCP, and/or UDP inputs.

Log is used to retrieve access log messages generated by Squid Proxy. For more details, refer to access.log and Squid native access.log format in detail.

Requirements

edit

You need Elasticsearch for storing and searching your data and Kibana for visualizing and managing it. You can use our hosted Elasticsearch Service on Elastic Cloud, which is recommended, or self-manage the Elastic Stack on your own hardware.

Elastic Agent must be installed. For more information, refer to Install Elastic Agents.

Installing and managing an Elastic Agent:

edit

You have a few options for installing and managing an Elastic Agent:

Install a Fleet-managed Elastic Agent (recommended):

edit

With this approach, you install Elastic Agent and use Fleet in Kibana to define, configure, and manage your agents in a central location. We recommend using Fleet management because it makes the management and upgrade of your agents considerably easier.

Install Elastic Agent in standalone mode (advanced users):

edit

With this approach, you install Elastic Agent and manually configure the agent locally on the system where it’s installed. You are responsible for managing and upgrading the agents. This approach is reserved for advanced users only.

Install Elastic Agent in a containerized environment:

edit

You can run Elastic Agent inside a container, either with Fleet Server or standalone. Docker images for all versions of Elastic Agent are available from the Elastic Docker registry, and we provide deployment manifests for running on Kubernetes.

There are some minimum requirements for running Elastic Agent. For more information, refer to Install Elastic Agents.

The minimum Kibana version required is 8.14.1.

Setup

edit

Configure Squid to export access logs using one of the supported methods (file (Module: Standard I/O), udp (Module: UDP Receiver), or tcp (Module: TCP Receiver)).

The integration supports the following formats:

Native log file

Enabling the integration in Elastic:

edit

In Kibana go to Management > Integrations.
In the Search for integrations bar, type Squid Proxy.
Select the Squid Proxy integration from the search results.
Click Add Squid Proxy to add the integration.
Add all the required integration configuration parameters according to the enabled input type.
Click Save and continue.

Logs reference

edit

Log

edit

The log dataset collects Squid logs.

Sample Event

edit

Example

An example event for log looks as following:

{
    "@timestamp": "2006-09-08T04:21:52.049Z",
    "agent": {
        "ephemeral_id": "703e0801-aef8-4d26-aa48-12c7673f6df0",
        "id": "29b8ade0-b4ef-4ce2-ab55-0acc99bbb914",
        "name": "elastic-agent-52603",
        "type": "filebeat",
        "version": "8.15.0"
    },
    "data_stream": {
        "dataset": "squid.log",
        "namespace": "63238",
        "type": "logs"
    },
    "destination": {
        "address": "175.16.199.115",
        "bytes": 19763,
        "geo": {
            "city_name": "Changchun",
            "continent_name": "Asia",
            "country_iso_code": "CN",
            "country_name": "China",
            "location": {
                "lat": 43.88,
                "lon": 125.3228
            },
            "region_iso_code": "CN-22",
            "region_name": "Jilin Sheng"
        },
        "ip": "175.16.199.115"
    },
    "ecs": {
        "version": "8.11.0"
    },
    "elastic_agent": {
        "id": "29b8ade0-b4ef-4ce2-ab55-0acc99bbb914",
        "snapshot": false,
        "version": "8.15.0"
    },
    "event": {
        "agent_id_status": "verified",
        "category": [
            "web"
        ],
        "dataset": "squid.log",
        "duration": 5006000000,
        "ingested": "2024-09-03T18:27:38Z",
        "kind": "event",
        "original": "1157689312.049   5006 10.105.21.199 TCP_MISS/200 19763 CONNECT login.yahoo.com:443 badeyek DIRECT/175.16.199.115 -",
        "outcome": "success",
        "type": [
            "access"
        ]
    },
    "http": {
        "request": {
            "method": "CONNECT"
        }
    },
    "input": {
        "type": "filestream"
    },
    "log": {
        "file": {
            "device_id": "35",
            "inode": "442644",
            "path": "/tmp/service_logs/squid-log-access.log"
        },
        "offset": 0
    },
    "observer": {
        "product": "Squid",
        "type": "proxy",
        "vendor": "Squid"
    },
    "related": {
        "ip": [
            "10.105.21.199",
            "175.16.199.115"
        ],
        "user": [
            "badeyek"
        ]
    },
    "source": {
        "address": "10.105.21.199",
        "ip": "10.105.21.199",
        "user": {
            "name": "badeyek"
        }
    },
    "squid": {
        "peer_status": "DIRECT",
        "result_code": "TCP_MISS",
        "status_code": 200
    },
    "tags": [
        "preserve_original_event",
        "squid-log",
        "forwarded"
    ],
    "url": {
        "original": "login.yahoo.com:443"
    }
}

Exported Fields

edit

Exported fields

Field	Description	Type
@timestamp	Event timestamp.	date
container.id	Unique container id.	keyword
data_stream.dataset	Data stream dataset.	constant_keyword
data_stream.namespace	Data stream namespace.	constant_keyword
data_stream.type	Data stream type.	constant_keyword
event.dataset	Event dataset	constant_keyword
event.module	Event module	constant_keyword
input.type	Type of Filebeat input.	keyword
log.file.device_id	ID of the device containing the filesystem where the file resides.	keyword
log.file.fingerprint	The sha256 fingerprint identity of the file when fingerprinting is enabled.	keyword
log.file.idxhi	The high-order part of a unique identifier that is associated with a file. (Windows-only)	keyword
log.file.idxlo	The low-order part of a unique identifier that is associated with a file. (Windows-only)	keyword
log.file.inode	Inode number of the log file.	keyword
log.file.path	Full path to the log file this event came from.	keyword
log.file.vol	The serial number of the volume that contains a file. (Windows-only)	keyword
log.flags	Flags for the log file.	keyword
log.offset	Offset of the entry in the log file.	long
log.source.address	Source address from which the log event was read / sent from.	keyword
squid.content_type	The content type as seen in the HTTP reply header.	keyword
squid.peer_status	A code explaining how the request was handled, by forwarding it to a peer or going straight to the source.	keyword
squid.result_code	The outcome of the request.	keyword
squid.status_code	The status of the result.	long

Changelog

edit

Changelog

Version	Details	Kibana version(s)
1.0.3	Bug fix (View pull request) Fix broken link and various formatting issues for the Squid Proxy integration.	8.14.1 or higher
1.0.2	Bug fix (View pull request) Tolerate errors with uri_parts, ensure _tmp fields are removed in case of errors.	8.14.1 or higher
1.0.1	Enhancement (View pull request) Add documentation on supported formats.	8.14.1 or higher
1.0.0	Enhancement (View pull request) Add dashboard and improve documentation. Release integration as GA.	8.14.1 or higher
0.21.0	Bug fix (View pull request) Enhance mappings and add geoip enrichments.	—
0.20.0	Enhancement (View pull request) Rewrite integration with standard ingest pipelines.	—
0.19.3	Bug fix (View pull request) Add http.request.body.bytes to ECS mappings.	—
0.19.2	Enhancement (View pull request) Changed owners	—
0.19.1	Bug fix (View pull request) Fix exclude_files pattern.	—
0.19.0	Enhancement (View pull request) ECS version updated to 8.11.0.	—
0.18.0	Enhancement (View pull request) ECS version updated to 8.10.0.	—
0.17.0	Enhancement (View pull request) Update package to ECS 8.9.0.	—
0.16.0	Enhancement (View pull request) Ensure event.kind is correctly set for pipeline errors.	—
0.15.0	Enhancement (View pull request) Update package to ECS 8.8.0.	—
0.14.0	Enhancement (View pull request) Update package-spec version to 2.7.0.	—
0.13.0	Enhancement (View pull request) Update package to ECS 8.7.0.	—
0.12.1	Enhancement (View pull request) Added categories and/or subcategories.	—
0.12.0	Enhancement (View pull request) Update package to ECS 8.6.0.	—
0.11.2	Bug fix (View pull request) Update docs to match field definitions.	—
0.11.1	Bug fix (View pull request) Remove duplicate fields.	—
0.11.0	Enhancement (View pull request) Update package to ECS 8.5.0.	—
0.10.1	Enhancement (View pull request) Use ECS geo.location definition.	—
0.10.0	Enhancement (View pull request) Update package to ECS 8.4.0	—
0.9.0	Enhancement (View pull request) Update package to ECS 8.3.0.	—
0.8.0	Enhancement (View pull request) Update to ECS 8.2.0	—
0.7.0	Enhancement (View pull request) Update to ECS 8.0.0	—
0.6.1	Bug fix (View pull request) Regenerate test files using the new GeoIP database	—
0.6.0	Enhancement (View pull request) Add 8.0.0 version constraint	—
0.5.4	Enhancement (View pull request) Uniform with guidelines	—
0.5.3	Enhancement (View pull request) Update Title and Description.	—
0.5.2	Bug fix (View pull request) Fixed a bug that prevents the package from working in 7.16.	—
0.5.1	Bug fix (View pull request) Fix logic that checks for the forwarded tag	—
0.5.0	Enhancement (View pull request) Update to ECS 1.12.0	—
0.4.3	Bug fix (View pull request) Requires version 7.14.1 of the stack	—
0.4.2	Enhancement (View pull request) Convert to generated ECS fields	—
0.4.1	Enhancement (View pull request) update to ECS 1.11.0	—
0.4.0	Enhancement (View pull request) Update integration description	—
0.3.0	Enhancement (View pull request) Set "event.module" and "event.dataset"	—
0.2.0	Enhancement (View pull request) update to ECS 1.10.0 and adding event.original options	—
0.1.4	Enhancement (View pull request) update to ECS 1.9.0	—
0.1.0	Enhancement (View pull request) initial release	—

« SQL input Juniper SRX integration »