Manage your logs
editManage your logs
editYour Enterprise Search deployment produces a range of logs, including crawler, API, and audit logs. What gets logged, how it is logged, and where you can find these logs, differs between Elastic Cloud and self-managed deployment options.
Your deployment writes some logs directly to disk, and others to Elasticsearch. For the full list, see the Logs quick reference.
This guide explains how to manage your logs on Elastic Cloud, and on self-managed deployments.
Read this guide to learn how to create data views, and use two Kibana tools, Logs and Discover, to view and query your logs.
Managing logs on Elastic Cloud deployments
editElastic Cloud users can set up logs and monitoring in their Cloud environment, and redirect logs and metrics to the same or a different cluster.
Follow these steps to enable logs and metrics:
- Navigate to Deployments → Deployment → Logs and metrics → Enable.
- Create a new monitoring deployment, or select an existing one.
Learn more about enabling logging and monitoring in Elastic Cloud.
Managing logs on self-managed deployments using Elasticsearch
editA subset of logs are indexed into Elasticsearch automatically, using Filebeat, as part of the Enterprise Search installation process.
This does not include general logs such as app-server.log
.
We encourage self-managed users to index these additional logs into Elasticsearch using a monitoring deployment. Apply the Elastic Enterprise Search Filebeat configuration to ingest Enterprise Search log files into Elasticsearch.
When logs are written to Elasticsearch you can manage log retention policies using index lifecycle management (ILM). Otherwise, you will have to manage the rollover and rotation of your logs yourself. The sooner you automate these policies, the better!
View self-managed deployment logs on-disk
editSelf-managed deployments have access to additional logs that are otherwise not available to Elastic Cloud users.
You’ll find these in the Logs quick reference, where the Cloud Elasticsearch index pattern
is marked N/A
.
To view your log files stored on disk, the process varies by deployment type.
Tar, deb, and rpm package installs:
-
Find the logs in
/log/dir
.
Docker:
-
Use the command
docker exec -it <container-name> /bin/bash
to launch a bash shell in the container. Find the logs in/log/dir
. -
Alternatively, the command
docker logs <container-name>
will show thestdout
of Enterprise Search.
Elastic Cloud Kubernetes
-
Use the command
kubectl exec --stdin --tty <pod-name> -- /bin/bash
to launch a bash shell to a running Enterprise Search pod. Find the logs in the/var/log/enterprise-search
directory in an Enterprise Search pod. -
Alternatively, the command
kubectl logs <pod-name>
will show thestdout
of an Enterprise Search pod.
Learn more about logging in Enterprise Search
editFor production use, you should send your deployment logs and metrics to a dedicated monitoring deployment. Learn more in this guide.
App Search exposes several event logging settings through its log settings API and a corresponding section of the Settings dashboard. Read the Log settings guide for more details.
Learn about log retention and how to disable logging in App Search.
Read this guide to learn how to create data views, and use two Kibana tools, Logs and Discover, to view and query your logs.
Read this guide to learn about audit logging in Enterprise Search.