Collecting Windows telemetry with Elastic: An introduction to the ETW Filebeat input

ETW is a lightweight, high-performance Windows-native mechanism for logging detailed event data about system performance and activity. Unlike traditional Windows event logs, ETW enables users to gather low-level telemetry that can be tailored for different providers and scenarios from system performance insights to application-specific diagnostics.

ETW has been widely used for diagnostics and performance monitoring within Windows environments, but collecting and managing ETW data traditionally required complex setups. By integrating ETW capabilities directly into Elastic, our goal is to simplify this process, making it easier to collect, filter, and analyze ETW events alongside other system and application logs within Elastic.

The centerpiece of Elastic’s new ETW capabilities is the ETW input for Filebeat. This new input allows Filebeat to directly subscribe to ETW providers, simplifying the way ETW data can be gathered, transmitted, and processed. Here’s an in-depth look at how it works and how you can use it.

The Filebeat ETW input subscribes to ETW sessions on a Windows host. An ETW session is a logical grouping of events from ETW providers, which are essentially sources of telemetry data, such as Windows Kernel and application providers.

The ETW input in Filebeat uses this subscription mechanism, gathering data in real time and shipping it to Elasticsearch for analysis and visualization. It can operate in three flexible modes and adapt to a range of data collection scenarios for both real-time and historical data needs:

1. Creating a new session: In this mode, Filebeat initializes a new ETW session on the Windows host to capture events from user-mode providers. This setup is ideal for scenarios requiring real-time data from applications or services currently running on the system. By creating a dedicated session, users can gather specific telemetry without affecting or relying on pre-existing ETW sessions on the system.

2. Attaching to an existing session: Filebeat can also attach to an already active ETW session, collecting ongoing event data generated within that session. This approach is useful for environments where ETW sessions are already in use. It allows administrators to capture events alongside other monitoring workflows without duplicating sessions or impacting system performance.

3. Reading from a prerecorded .etl file: In some cases, ETW data may be recorded in advance and stored in .etl files. Filebeat can parse these files, enabling users to analyze historical event data. This mode is particularly valuable for retrospective analysis, where users may want to investigate previous system or application behavior based on ETW data captured earlier.

This flexibility in operational modes allows Filebeat’s ETW input to adapt to diverse use cases — whether for real-time monitoring, forensic analysis, or operational troubleshooting.

Once the mode is set, the ETW input follows a streamlined process to gather and send data to Elastic:

• Provider subscription: Filebeat initiates a subscription to one or more ETW providers based on user configuration, whether by creating a new session or connecting to an existing one.

• Data collection: Events are captured from the active ETW session or parsed from .etl files. ETW records are collected, parsed, and converted into JSON objects to ensure they are structured consistently for valid ingestion. This transformation prepares the data for smooth indexing in Elasticsearch, allowing it to be seamlessly integrated with other log and metric data within Elastic.

• Transmission to Elasticsearch: Once events are collected, they’re sent to Elasticsearch for storage and analysis — and ready for visualization within Kibana.

To demonstrate how the Filebeat ETW input can be configured to capture data from a specific ETW provider, let’s take the example of monitoring the Microsoft DNS Server provider.

To configure the Filebeat ETW input for monitoring DNS events, we can use the logman command-line utility, which provides insight into available ETW providers and their details. logman allows us to list all available providers, view the manifest of each one, and examine details like filterable keywords, trace levels, and other configurations.

Before configuring Filebeat to collect events from a specific provider, you may want to view all available ETW providers in your system. To see the full list of providers, run the following command:

Beyond the Filebeat ETW input, we’re also introducing two integrations that simplify ETW data collection for specific use cases.

Monitoring DNS server analytical logs is essential for maintaining the security and performance of network infrastructure. Common types of DNS attacks — DNS hijacking, DNS tunneling, various denial-of-service (DoS) attacks, and DNS cache poisoning — mean significant risks to organizations. Windows DNS analytical logging is a vital event source that administrators and security analysts require visibility into. It provides detailed insights into every DNS transaction occurring both internally and externally on a network.

The Microsoft DNS Server integration focuses specifically on gathering DNS server Audit and Analytical logs. With this integration, organizations can easily track and analyze DNS queries, responses, and error rates — facilitating a deeper understanding of DNS activity. The prebuilt dashboards included in the integration allow users to dive into DNS activity analysis without the need for complex configuration or extensive setup.

The new ETW input in Elastic reveals powerful possibilities for monitoring and securing Windows environments. Users can now collect detailed telemetry from Windows applications and services that rely on ETW, allowing for real-time insights into system performance, application behavior, and potential security risks.

This capability is complemented by built-in integrations, such as the Microsoft DNS Server integration, that make it simple to gain immediate, actionable insights without complex setup.

These new features are just the beginning of what ETW can bring to Elastic users. We’re excited for you to try out these features, explore new use cases, and share your feedback to help us continue evolving ETW capabilities within Elastic. To get started, visit the ETW input page for setup instructions, or explore these features with a free trial on Elastic Cloud.