Introducing Dynamic Observability: A no-code integration between Elastic and Rookout

This guest post is by Oded Keret, Rookout's vice president of product. Oded is an experienced software engineer and product manager. He is a true believer in having fun while working together. He is also the founder of the Board Games in Tel-Aviv Meetup group, and Co-Founder of The Party™ team."

In recent years, Observability has become a de-facto standard when discussing development and maintenance of cloud-native applications. The need to develop an observable system and ensure that as it runs in production, engineers will be able to detect performance issues, downtimes, and service disruptions, has evolved into a rich ecosystem of tools and practices.

The Elastic Observability platform has been a trailblazer in this ecosystem, providing a one-stop shop for collecting logs, trace information, and metrics from a seemingly infinite number of agents, agentless collection methods, and cloud provider integrations. Adding visualization, automated root cause analysis, and alerting capabilities on top of the collected data provides engineers with a single pane of glass, providing a rich APM experience presented in a unified and intuitive user experience.

The latest integration between Elastic and Rookout aims to further enrich this experience by introducing Dynamic Observability - namely, the ability to collect additional logs, trace information and metrics - but without adding code, without waiting for a deployment, and with minimal performance and logging cost impact. This development has pushed observability one step further towards a real-time capability, allowing it to address the rising challenges of managing and deploying distributed and dynamic microservices-based deployments.

In this blog post, we will dive into the new partnership, providing a hands-on guide to exposing the power of Dynamic Observability.

The Rookout Live Platform integrates with the Elastic Observability, offering two key game changing capabilities to Elastic users:

1. “Non Breaking Breakpoints” let you fetch local variables, logs and metrics from any line of code within the app, without stopping the app and without adding code or restarting.
2. Live Logging lets you Change logging verbosity level dynamically and contextually. Drill down to the specific instances, components, accounts, or even individual users to get detailed Debug and Trace logs with no pre-filtering.

To enjoy these capabilities and enhance your Elastic Observability experience, we will first set up your Rookout environment.

Besides the instrumentation agent, Rookout also offers a Web Debugger. Once your application is up and running, you’ll be able to set Non Breaking Breakpoints in a dev-friendly user interface, and instrument your code dynamically.

To make sure everything is set up correctly, and see the magic of Rookout, select the appropriate file and add a non-breaking breakpoint to the relevant line of code. Once you invoke it, you will instantly get a full snapshot of your application, showing the values of the variables, stack trace, and much more!

One more ability provided by Rookout is a dynamic view of your cloud-native deployment. In the Debug Session Configuration page, you will be able to get a view of your environment, and group the instances in which Rookout is deployed. By grouping and filtering your environment by Namespace, Deployment or any other cloud-native parameter, you will be able to slice and dice and dynamically fetch data from any or all of the pods, containers, functions or servers running in your application.

Next, you’ll want to hook Rookout up with your Elastic Observability, so that you can benefit from dynamically collected data in Kibana. Start by clicking Settings (the cogwheel at the bottom left) and select the Targets option. Go ahead and add Elasticsearch as a new target:

Name your new target, configure that cluster’s hostnames, the index name, authentication credentials and more:

Now that you are connected, the Rookout instrumentation agents will listen quietly for instructions. When a Rookout Non-Breaking Breakpoint is set and hit, they will dynamically start streaming live data to your Elastic Observability instance.

Once you have got everything set up, getting new data into your Elastic is a breeze. Start out by selecting the instances you want to collect data from. You can collect data from as many (or as few!) instances as you need. Whether you want to collect data from entire deployments, or a single instance, that’s entirely up to you.

Based on the instances you have selected, the source code will be automatically loaded into your browser, and all you have to do is click on the relevant line. Once that line of code is executed, you will see a snapshot at the bottom of the screen - make sure this is the information you are looking for.

Finally, edit the breakpoint and configure it to send data to Elasticsearch, and you are done!

We have seen how Rookout can be used to instantly switch on a newly created log line, without adding code and without stopping or restarting the app. This is very useful in cases where the data you are missing is just not printed to log. That is - if the log line is just not

there. In some cases, you do have a log line that would have printed the missing data - for example, a log line in DEBUG or INFO level, that will not be printed when the application is running in WARNING or ERROR logging verbosity.

As we have shown, Rookout lets you dynamically troubleshoot and add logs in remote, live environments. The fact that you can gain extra visibility into your detailed running code without changing it and without restarting is what sets Rookout apart from traditional instrumentation methods. And by pipelining extra data dynamically into your Elastic ELK stack, we aim to provide a production-first debugging approach throughout your application development lifecycle.