Introducing the new alerting framework for Elastic Observability, Elastic Security, and the Elastic Stack
Editor's Note: With the release of Elastic Stack 7.11, the new alerting framework is now generally available. In addition to existing connectors to 3rd party platforms like Slack PagerDuty, and Servicenow, 7.11 adds Microsoft Teams to the list of built-in alerting integrations. Read more about this update in our alerting release blog.
We’re excited to announce a new alerting framework that delivers a first-class alerting experience natively within the SIEM, Uptime, APM, and Metrics applications as part of the Kibana 7.7 release.
Alerting is a fundamental use case across the Elastic Stack, which is why we’re making it part of the core experience within Kibana. Whether you are monitoring application transactions or tracking brute force login attempts, our goal is to provide a tailored experience that allows you to build powerful alerts in the normal flow of your task. The new alerting framework is built from the ground up and designed to offer more than just convenient interfaces. We understand the need to go beyond just notifying people which is why we’ve also incorporated the ability to trigger predefined actions that can do anything from sending an email to using brand new third-party integrations with platforms like Slack and PagerDuty.
The new alerting framework is being introduced as a beta in the 7.7 release of Kibana and is available immediately on the Elasticsearch Service on Elastic Cloud, or for download.
Alerting everywhere you need it to be
Detecting and understanding significant signal shifts is a fundamental need that cuts across all use cases for organizations that build, maintain, and evolve digital systems. In application performance monitoring (APM), for example, you want to detect increases in latency in your application’s responsiveness as well as in error rates, because both potentially have a direct impact on your service and user experience. Furthermore, you want to be able to see this at an infrastructure level with metric alerts such as CPU or memory usage spikes or even service and network downtime. Each of these could result in application performance degradation if not acted on in a timely manner. In addition to one-time events, there is also a need to detect recurring patterns within logs in an effort to understand and proactively avoid future situations. Pivoting to a system security use case, the need for real-time alerting is vital to spot threats like distributed service requests or large data transfers at suspicious times. More broadly, in business analytics, early detection alerts of dips or spikes that correlate with core KPIs driving company performance may well contribute to the success of your strategy and execution.
It was with these use cases and many more in mind that in September 2019 we shared our vision for a new alerting framework in the Elastic Stack. A key part of that vision included three observations gathered from years of deep customer engagement that ultimately lay the foundation and the approach we are taking in reimagining alerts within the Elastic Stack. Those observations are:
- Alerting needs to be everywhere
- Making sense of alerts is critical
- Alerting should be about detection and action
Introducing a new alerting framework for the Elastic Stack in 7.7
We are excited to announce with 7.7 a major step forward in delivering on our vision of alerting within the Elastic Stack that is tightly integrated within the Elastic Observability and Security solutions and makes integrations that matter to people and companies easy to configure.
- Alerting everywhere: Kibana 7.7 introduces ubiquitous alerting for Elastic Observability, Elastic Security, and the Elastic Stack. Users can now create alerts directly from within the SIEM, APM, Metrics, and Uptime applications as well as for any index. This gives users the power to address their alerting needs in the context of their unique use case. The new alerting framework is designed around the core principle of seamless usability and interoperability across solutions with Kibana as their platform.
- Making sense of alerts: Detecting the signal and eliminating the noise is a core focus. For example, in addition to the solution-specific options for creating alerts, Kibana 7.7 provides a single dedicated master view for managing, listing, searching, and editing all alerts in one place. The new alerting framework helps make sense of alerts throughout their lifecycle, from creation, for example, by offering a visual preview, to taking action.
- Detection and action: The new alerting framework focuses on enabling alerts to do more than simply get a human’s attention. With Kibana 7.7 the goal is to seamlessly pass the alerting context that you need into the systems and workflows that matter the most to you, by making integrating a simple and easy process. This is why you’ll see built-in options for integrating with third-party solutions like Slack and PagerDuty as well as webhooks for customizing additional integrations.
More user-friendly, more powerful, more actionable
The new alerting framework lives and breathes in Kibana. This choice, and the principles on which it is built, define the framework in a number of ways.
First, extra care was taken to provide an intuitive and user-friendly experience when it comes to alert creation and management. As a result, creating and editing alerts is achieved using dropdowns and prompts making it easy for anyone to use the first time. Alerting everywhere means that we offer an array of tailored ways to create advanced, contextual alerts in Kibana. This includes generating multiple alert instances via a single alert definition so one effort translates into multiple outputs. In addition, the user experience is consistent across the APM, Metrics, Uptime and SIEM apps, meaning no matter where someone creates an alert in the Elastic Stack, they’ll find the same easy-to-use controls.
The power of the new alerting framework goes much deeper than its interfaces. Also new in this 7.7 release is a distributed task manager which delivers greater performance and scalability. This means that you can easily scale out alerting capacity simply by adding additional Kibana instances.
Importantly, the new alerting framework comes with a number of integrations which you can use to create actions. The 7.7 release introduces third-party connectors for triggering alerts to email, Slack, PagerDuty, and webhooks. Elastic connectors allow you to easily write alerts to indices and server logs. Setting up these connectors takes only a few seconds. Just enter the endpoint and key or credentials on the designated input fields on the UI, as provided by your email, Slack, PagerDuty, or webhook account, and start channeling your alerts and their context across the systems in your workflows.
Finally, the new alerting framework in Kibana supports multitenancy. This means that you can organize your alerts into Kibana Spaces, and the framework will soon fully support Kibana’s enhanced authorization model.