Monitoring

Enterprise Search and the Elastic Stack provide monitoring features to keep a pulse on the health and performance of your Enterprise Search deployment.

Elastic Cloud provides the easiest path to monitor your deployment. Follow these recommended steps:

Additional monitoring options:

Follow these steps to monitor your self-managed Enterprise Search deployment:

If you need more insights into the low-level details of Enterprise Search itself, you can enable APM for Enterprise Search.

You may also call Enterprise Search monitoring APIs to access internal Enterprise Search system metrics and more product usage details.

To visualize the metrics ingested into Elasticsearch via Cloud monitoring, via the self-monitoring feature, or via a dedicated Enterprise Search Metricbeat module, import pre-built Kibana dashboards into your deployment:

Use Elastic APM to collect in-depth performance metrics and errors for Enterprise Search itself.

To enable APM for Enterprise Search monitoring, follow these steps in your Enterprise Search deployment or in a separate deployment dedicated to collecting APM data.

Find the APM secret_token and server_url on the deployment you are using to collect APM data:

Configure Enterprise Search settings on your main deployment to add the APM secret_token and server_url settings:

apm.enabled: true
apm.server_url: 'your-apm-server-url-here'
apm.secret_token: 'your-token-here'

An example apm.server_url looks like this on Elastic Cloud: https://your-apm-deployment-name.apm.europe-west1.gcp.cloud.es.io:443.

An example apm.secret_token looks like this: 2FHHxzMDyU7G3t6pXb.

After the changes are applied, view Enterprise Search metrics in APM:

Enable Enterprise Search self-monitoring on self-managed deployments to capture metrics including Enterprise Search network, JVM, product usage, and background worker metrics.

To enable self-monitoring, add the following setting to your Enterprise Search configuration and restart Enterprise Search:

monitoring.reporting_enabled: true

This feature is intended for self-managed deployments. It is not necessary to enable self-monitoring on Elastic Cloud deployments. See Monitor Enterprise Search deployments on Elastic Cloud.

The Enterprise Search self-monitoring feature only ships metrics to the same Elasticsearch deployment that Enterprise Search uses.

After self-monitoring is enabled, Enterprise Search will automatically provision a Metricbeat instance with the enterprisesearch module. The module will start reporting a detailed set of monitoring metrics to the underlying Elasticsearch cluster every 10 seconds to be stored in a set of indexes called .monitoring-ent-search-*.

Metricbeat indexes have ILM enabled on them. By default, data will be stored in Elasticsearch for up to 30 days. Indexes will be rotated every 100Mb.

See the monitoring section of the configuration guide for deployment configuration settings to control the frequency of monitoring data reporting, index names used and other aspects of the self-monitoring feature within Enterprise Search.

You can view self-monitoring metrics in Kibana. Use the Kibana dashboards for Enterprise Search monitoring or choose Stack Monitoring from the sidebar menu under Management.

Elastic Metricbeat contains an official Enterprise Search monitoring module that can be used to ingest monitoring data from your Enterprise Search deployments into any Elasticsearch cluster. This module powers the Enterprise Search self-monitoring feature underneath. When using Elastic Cloud, enabling logging and monitoring for your deployment runs a built-in Enterprise Search Metricbeat module internally.

The data produced by the Metricbeat module can then be accessed in Kibana, for example by using the Kibana dashboards for Enterprise Search monitoring.

See the Enterprise Search Metricbeat module documentation for more details on how to use this module.