IMPORTANT: No additional bug fixes or documentation updates will be released for this version. For the latest information, see the current release documentation.
« Enterprise Search server troubleshooting Monitoring »
Elastic Docs › ›Elastic Enterprise Search server

Troubleshooting Enterprise Search setup

edit

Troubleshooting Enterprise Search setup

edit

When navigating to Enterprise Search within Kibana, you may see "Unable to connect". To troubleshoot this, first enable and view logs for Enterprise Search and Kibana. Then review common setup issues to help identify and solve your problem.

Enable and view logs

edit

Enterprise Search and Kibana each log information that may help identify and solve your problem. If necessary, first enable logging. Then view the logs for each service.

Refer to the following documentation for your deployment type:

Review common setup issues

edit

Review the following common issues to help identify and solve your problem:

Enterprise Search’s kibana.external_url setting is incorrect or unreachable

edit

Enterprise Search will not start and logs the following:

[2021-09-28T17:57:33.438+00:00][8][2002][app-server][INFO]: Performing pre-flight checks for Kibana running on http://kibana1:5601...
Unexpected exception while running Enterprise Search:
Error: kibana1: Name or service not known at /usr/share/enterprise-search/lib/war/gems/gems/manticore-0.7.0-java/lib/faraday/adapter/manticore.rb:88:in `block in call'
        /usr/share/enterprise-search/lib/war/gems/gems/manticore-0.7.0-java/lib/manticore/response.rb:79:in `call'
        /usr/share/enterprise-search/lib/war/gems/gems/manticore-0.7.0-java/lib/faraday/adapter/manticore.rb:92:in `call'
        /usr/share/enterprise-search/lib/war/lib/middleware/request_logging_middleware.class:25:in `call'
        /usr/share/enterprise-search/lib/war/gems/gems/faraday-1.3.0/lib/faraday/rack_builder.rb:154:in `build_response'
        /usr/share/enterprise-search/lib/war/gems/gems/faraday-1.3.0/lib/faraday/connection.rb:492:in `run_request'
        /usr/share/enterprise-search/lib/war/gems/gems/faraday-1.3.0/lib/faraday/connection.rb:198:in `get'
        /usr/share/enterprise-search/lib/war/shared_togo/lib/shared_togo/kibana_checks.class:89:in `check_kibana_connection_with_retries!'
        /usr/share/enterprise-search/lib/war/shared_togo/lib/shared_togo/kibana_checks.class:36:in `check_kibana_connection!'
        /usr/share/enterprise-search/lib/war/shared_togo/lib/shared_togo/kibana_checks.class:20:in `block in run!'
        /usr/share/enterprise-search/lib/war/gems/gems/activesupport-5.1.7/lib/active_support/tagged_logging.rb:69:in `block in tagged'
        /usr/share/enterprise-search/lib/war/gems/gems/activesupport-5.1.7/lib/active_support/tagged_logging.rb:26:in `tagged'
        /usr/share/enterprise-search/lib/war/gems/gems/activesupport-5.1.7/lib/active_support/tagged_logging.rb:69:in `tagged'
        /usr/share/enterprise-search/lib/war/shared_togo/lib/shared_togo/kibana_checks.class:19:in `run!'
        /usr/share/enterprise-search/lib/war/shared_togo/lib/shared_togo/kibana_checks.class:13:in `run!'
        /usr/share/enterprise-search/lib/war/shared_togo/lib/shared_togo.class:290:in `configure_kibana!'
        /usr/share/enterprise-search/lib/war/shared_togo/lib/shared_togo.class:248:in `configure!'
        /usr/share/enterprise-search/lib/war/config/application.class:19:in `<main>'
        org/jruby/RubyKernel.java:1016:in `load'
        /usr/share/enterprise-search/lib/war/gems/gems/activesupport-5.1.7/lib/active_support/dependencies.rb:286:in `block in load'
        /usr/share/enterprise-search/lib/war/gems/gems/activesupport-5.1.7/lib/active_support/dependencies.rb:258:in `load_dependency'
        /usr/share/enterprise-search/lib/war/gems/gems/activesupport-5.1.7/lib/active_support/dependencies.rb:286:in `load'
        /usr/share/enterprise-search/lib/war/config/application.rb:1:in `<main>'
        org/jruby/RubyKernel.java:974:in `require'
        /usr/share/enterprise-search/lib/war/gems/gems/activesupport-5.1.7/lib/active_support/dependencies.rb:292:in `block in require'
        /usr/share/enterprise-search/lib/war/gems/gems/activesupport-5.1.7/lib/active_support/dependencies.rb:258:in `load_dependency'
        /usr/share/enterprise-search/lib/war/gems/gems/activesupport-5.1.7/lib/active_support/dependencies.rb:292:in `require'
        /usr/share/enterprise-search/lib/war/shared_togo/lib/shared_togo/cli/command.class:36:in `initialize'
        /usr/share/enterprise-search/lib/war/shared_togo/lib/shared_togo/cli/command.class:10:in `run_and_exit'
        /usr/share/enterprise-search/lib/war/shared_togo/lib/shared_togo/cli.class:143:in `run_supported_command'
        /usr/share/enterprise-search/lib/war/shared_togo/lib/shared_togo/cli.class:125:in `run_command'
        /usr/share/enterprise-search/lib/war/shared_togo/lib/shared_togo/cli.class:112:in `run!'
        bin/enterprise-search-internal:15:in `<main>'
Solution
edit

Set kibana.external_url to an accessible Kibana address.

Kibana’s enterpriseSearch.host setting is incorrect or unreachable

edit

Kibana logs the following:

{"type":"log","@timestamp":"2021-09-28T18:05:43+00:00","tags":["error","plugins","enterpriseSearch"],"pid":1221,"message":"Could not perform access check to Enterprise Search: FetchError: request to https://enterprise-search1:3002/api/ent/v2/internal/client_config failed, reason: getaddrinfo ENOTFOUND enterprise-search1"}
Solution
edit

Set enterpriseSearch.host to an accessible Enterprise Search address. See Configure Kibana.

Kibana’s enterpriseSearch.host is set to http while Enterprise Search is running on https

edit

Kibana logs the following:

{"type":"log","@timestamp":"2021-09-28T17:39:05+00:00","tags":["error","plugins","enterpriseSearch"],"pid":1219,"message":"Could not perform access check to Enterprise Search: FetchError: request to http://enterprise-search:3002/api/ent/v2/internal/client_config failed, reason: Parse Error: Expected HTTP/"}
Solution
edit

Set enterpriseSearch.host to the correct protocol within Kibana. See Configure Kibana.

Kibana’s connection to Enterprise Search is timing out

edit

Kibana logs the following:

{"type":"log","@timestamp":"2021-09-28T17:52:31+00:00","tags":["warning","plugins","enterpriseSearch"],"pid":1219,"message":"Enterprise Search access check took over 300ms. Please ensure your Enterprise Search server is responding normally and not adversely impacting Kibana load speeds."}
{"type":"log","@timestamp":"2021-09-28T17:52:35+00:00","tags":["warning","plugins","enterpriseSearch"],"pid":1219,"message":"Exceeded 5000ms timeout while checking https://enterprise-search:3002. Please consider increasing your enterpriseSearch.accessCheckTimeout value so that users aren't prevented from accessing Enterprise Search plugins due to slow responses."}
Solution
edit

Troubleshoot performance issues in Enterprise Search. To get started, see Monitoring APIs.

If necessary, contact Elastic Support.

Kibana cannot find or does not trust the SSL certificates for Enterprise Search

edit

Kibana logs the following:

{"type":"log","@timestamp":"2021-09-28T17:42:11+00:00","tags":["error","plugins","enterpriseSearch"],"pid":1220,"message":"Could not perform access check to Enterprise Search: FetchError: request to https://enterprise-search:3002/api/ent/v2/internal/client_config failed, reason: unable to verify the first certificate"}
Solution
edit

Configure enterpriseSearch.ssl.certificateAuthorities to reference the correct certificate or certificate chain. See Configure Kibana to Trust Your SSL Certificate Authority.

Enterprise Search does not support requests from the Enterprise Search Server to Kibana using custom CAs.

Enterprise Search has SSL enabled but hostname verification fails

edit

When enterpriseSearch.ssl.verificationMode is configured to perform hostname verification but hostname verification fails.

Kibana logs the following:

{"type":"log","@timestamp":"2021-09-28T18:39:04+00:00","tags":["error","plugins","enterpriseSearch"],"pid":1220,"message":"Could not perform access check to Enterprise Search: FetchError: request to https://enterprise-search:3002/api/ent/v2/internal/client_config failed, reason: Hostname/IP does not match certificate's altnames: Host: enterprise-search. is not in the cert's altnames: DNS:localhost, IP Address:127.0.0.1, DNS:node1"}
Solution
edit

In production or other environments that must be secure, regenerate the certificates with the correct Subject Alternative Names matching enterpriseSearch.host in Kibana.

In development and some other environments, reducing security may be acceptable. In these cases, set enterpriseSearch.ssl.verificationMode to a lower setting. For example, change full to certificate or none. See Configure Kibana to Trust Your SSL Certificate Authority.

Reduce the security of your deployment only as a last resort. Ideally, hostname verification should be enabled in all environments.

Enterprise Search and Kibana versions are incompatible

edit

The Kibana UI displays "Incompatible version error".

Kibana logs the following:

{"type":"log","@timestamp":"2021-09-28T19:03:00+00:00","tags":["warning","plugins","enterpriseSearch"],"pid":1225,"message":"Your Kibana instance (v7.15.0) is not the same version as your Enterprise Search instance (vundefined), which may cause unexpected behavior. Use matching versions for the best experience."}
Solution
edit

Run Enterprise Search and Kibana services of the same major and minor versions.

Enterprise Search in Kibana is "forbidden" when traffic filters are set on Elastic Cloud

edit

The Kibana UI displays "Forbidden".

Kibana logs the following:

ResponseError: security_exception: [security_exception] Reason: action [indices:data/read/search] is unauthorized for user [cloud-internal-enterprise_search-server] with roles [cloud-internal-enterprise_search-server] on indices [.kibana_7.14.0_001,.kibana], this action is granted by the index privileges [read,all]
    at onBody (/usr/share/kibana/node_modules/@elastic/elasticsearch/lib/Transport.js:337:23)
    at IncomingMessage.onEnd (/usr/share/kibana/node_modules/@elastic/elasticsearch/lib/Transport.js:264:11)
    at IncomingMessage.emit (events.js:387:35)
    at endReadableNT (internal/streams/readable.js:1317:12)
    at processTicksAndRejections (internal/process/task_queues.js:82:21) {
  meta: {
    body: { error: [Object], status: 403 },
    statusCode: 403,
    headers: {
      'content-length': '625',
      'content-type': 'application/json; charset=UTF-8',
      'x-cloud-request-id': 'O_v__Y_hRf-I528DechRKQ',
      'x-elastic-product': 'Elasticsearch',
      'x-found-handling-cluster': '<deployment_id>',
      'x-found-handling-instance': 'instance-0000000000',
      'x-opaque-id': 'd77d5906-8317-40c1-86e4-d78a408908b5',
      date: 'Mon, 30 Aug 2021 12:14:08 GMT'
    },
    meta: {
      context: null,
      request: [Object],
      name: 'elasticsearch-js',
      connection: [Object],
      attempts: 0,
      aborted: false
    }
  }
}
Solution
edit

Known issues were resolved via the 8.3.3 release. Please contact Elastic Support if this error is encountered on versions on or after 8.3.3.

« Enterprise Search server troubleshooting Monitoring »