X-Pack security Troubleshooting

This is intentional. Some of the settings are considered to be highly sensitive: all ssl settings, ldap bind_dn, bind_password). For this reason, we filter these settings and do not expose them via the nodes info API rest endpoint. You can also define additional sensitive settings that should be hidden using the xpack.security.hide_settings setting. For example, this snippet hides the url settings of the ldap1 realm and all settings of the ad1 realm.

xpack.security.hide_settings: xpack.security.authc.realms.ldap1.url, xpack.security.authc.realms.ad1.*

Verify that the role names associated with the users match the roles defined in the roles.yml file. You can use the users tool to list all the users. Any unknown roles are marked with *.

bin/xpack/users list
rdeniro        : admin
alpacino       : power_user
jacknich       : monitoring,unknown_role* 

This error occurs when the users tool is parsing the input and finds unexepected arguments. This can happen when there are special characters used in some of the arguments. For example, on Windows systems the , character is considered a parameter separator; in other words -r role1,role2 is translated to -r role1 role2 and the users tool only recognizes role1 as an expected parameter. The solution here is to quote the parameter: -r "role1,role2".

Check your realm configuration; realms are checked serially, one after another. If your Active Directory realm is being checked before other realms and there are usernames that appear in both Active Directory and another realm, a valid login for one realm may be causing failed login attempts in another realm.

For example, if UserA exists in both Active Directory and a file realm, and the Active Directory realm is checked first and file is checked second, an attempt to authenticate as UserA in the file realm would first attempt to authenticate against Active Directory and fail, before successfully authenticating against the file realm. Because authentication is verified on each request, the Active Directory realm would be checked - and fail - on each request for UserA in the file realm. In this case, while the authentication request completed successfully, the account on Active Directory would have received several failed login attempts, and that account may become temporarily locked out. Plan the order of your realms accordingly.

Also note that it is not typically necessary to define multiple Active Directory realms to handle domain controller failures. When using Microsoft DNS, the DNS entry for the domain should always point to an available domain controller.

A number of configuration options can cause this error.

To help track down these possibilities, add the following lines to the end of the log4j2.properties configuration file in the CONFIG_DIR:

logger.authc.name = org.elasticsearch.xpack.security.authc
logger.authc.level = DEBUG

A successful authentication should produce debug statements that list groups and role mappings.

Apple’s integration of curl with their keychain technology disables the --cacert option. See http://curl.haxx.se/mail/archive-2013-10/0036.html for more information.

You can use another tool, such as wget, to test certificates. Alternately, you can add the certificate for the signing certificate authority MacOS system keychain, using a procedure similar to the one detailed at the Apple knowledge base. Be sure to add the signing CA’s certificate and not the server’s certificate.

Indicates that a client connection was made to node01.example.com but the certificate returned did not contain the name node01.example.com. In most cases, the issue can be resolved by ensuring the name is specified during certificate creation. For more information, see Setting Up SSL/TLS on a Cluster. Another scenario is when the environment does not wish to use DNS names in certificates at all. In this scenario, all settings in elasticsearch.yml should only use IP addresses including the network.publish_host setting.

Indicates that a client connection was made to an IP address but the returned certificate did not contain any SubjectAlternativeName entries. IP addresses are only used for hostname verification if they are specified as a SubjectAlternativeName during certificate creation. If the intent was to use IP addresses for hostname verification, then the certificate will need to be regenerated with the appropriate IP address. See Setting Up SSL/TLS on a Cluster.

The SSLHandshakeException above indicates that a self-signed certificate was returned by the client that is not trusted as it cannot be found in the truststore or keystore. The SSLException above is seen on the client side of the connection.

The SunCertPathBuilderException above indicates that a certificate was returned during the handshake that is not trusted. This message is seen on the client side of the connection. The SSLException above is seen on the server side of the connection. The CA certificate that signed the returned certificate was not found in the keystore or truststore and needs to be added to trust this certificate.

Indicates that there was an incoming plaintext http request. This typically occurs when an external applications attempts to make an unencrypted call to the REST interface. Please ensure that all applications are using https when calling the REST interface with SSL enabled.

Indicates that there was incoming plaintext traffic on an SSL connection. This typically occurs when a node is not configured to use encrypted communication and tries to connect to nodes that are using encrypted communication. Please verify that all nodes are using the same setting for xpack.security.transport.ssl.enabled.

Indicates an issue with data received on the transport interface in an unknown format. This can happen when a node with encrypted communication enabled connects to a node that has encrypted communication disabled. Please verify that all nodes are using the same setting for xpack.security.transport.ssl.enabled.

The exception is typically seen when a https request is made to a node that is not using https. If https is desired, please ensure the following setting is in elasticsearch.yml:

xpack.security.http.ssl.enabled: true

This error occurs when a SSL/TLS cipher suite is specified that cannot supported by the JVM that Elasticsearch is running in. Security will try to use the specified cipher suites that are supported by this JVM. This error can occur when using the Security defaults as some distributions of OpenJDK do not enable the PKCS11 provider by default. In this case, we recommend consulting your JVM documentation for details on how to enable the PKCS11 provider.

Another common source of this error is requesting cipher suites that use encrypting with a key length greater than 128 bits when running on an Oracle JDK. In this case, you will need to install the JCE Unlimited Strength Jurisdiction Policy Files.