IMPORTANT: No additional bug fixes or documentation updates will be released for this version. For the latest information, see the current release documentation.

« Kerberos authentication Integrating with other authentication systems »

› › ›

JWT authentication

edit

JWT authentication

edit

This functionality is in beta and is subject to change. The design and code is less mature than official GA features and is being provided as-is with no warranties. Beta features are not subject to the support SLA of official GA features.

Elasticsearch can be configured to trust JSON Web Tokens (JWTs) issued from an external service as bearer tokens for authentication.

When a JWT realm is used to authenticate with Elasticsearch, a distinction is made between the client that is connecting to Elasticsearch, and the user on whose behalf the request should run. The JWT authenticates the user, and a separate credential authenticates the client.

A common scenario for JWTs is when an existing front-end application uses OpenID Connect (OIDC) to authenticate and identify a user, and then accesses Elasticsearch on behalf of the authenticated user.

If the front-end application does not exist, you can use the Token-based authentication services instead.

JWT uses OIDC workflows

edit

JWT authentication in Elasticsearch is derived from OIDC user workflows, where different tokens can be issued by an OIDC Provider (OP). One possible token is an ID token, which uses the JWT format. If the ID token is presented to a JWT realm, Elasticsearch can use it as a bearer token to authenticate, identify, and authorize an individual user.

Because JWTs are obtained external to Elasticsearch, you can define a custom workflow instead of using the OIDC workflow. However, the JWT format must still be JSON Web Signature (JWS). The JWS header and JWS signature are validated using OIDC ID token validation rules.

Elasticsearch supports a separate OpenID Connect realm, which provides stronger security guarantees than the JWT realm, and is preferred for any use case where Elasticsearch can act as an OIDC RP. The OIDC realm is the only supported way to enable OIDC authentication in Kibana.

If JWTs are issued for the front-end application, the application is the realm client and JWT user. That is not supported by OIDC flows, but it may be supported by bespoke JWT issuers. In that case, use the client secret and JWT for the client application, and the es-security-runas-user HTTP request header for the different user. See Applying the run_as privilege to JWT realm users.

Configure Elasticsearch to use a JWT realm

edit

To use JWT authentication, create the realm in the elasticsearch.yml file to configure it within the Elasticsearch authentication chain.

The JWT realm has a few mandatory settings, plus optional settings that are described in JWT realm settings.

Client authentication is enabled by default for the JWT realms. Disabling client authentication is possible, but strongly discouraged.

Add your JWT realm to the elasticsearch.yml file. The following example includes the most common settings, which are not intended for every use case:
```
xpack.security.authc.realms.jwt.jwt1:
  order: 3
  client_authentication.type: shared_secret
  allowed_issuer: "https://issuer.example.com/jwt/"
  allowed_audiences: [ "8fb85eba-979c-496c-8ae2-a57fde3f12d0" ]
  allowed_signature_algorithms: [RS256,HS256]
  pkc_jwkset_path: jwt/jwkset.json
  claims.principal: sub
```
order

Specifies a realm order of 3, which indicates the order in which the configured realm is checked when authenticating a user. Realms are consulted in ascending order, where the realm with the lowest order value is consulted first.

client_authentication.type

Specifies the client authentication type as shared_secret, which means that the client is authenticated using an HTTP request header that must match a pre-configured secret value. The client must provide this shared secret with every request in the ES-Client-Authentication header. The value must be a case-insensitive match to the realm’s client_authentication.shared_secret.

allowed_issuer

Sets a verifiable identifier for your JWT issuer. This value is typically a URL, UUID, or some other case-sensitive string value.

allowed_audiences

Specifies a list of JWT audiences that the realm will allow. These values are typically URLs, UUIDs, or other case-sensitive string values.

allowed_signature_algorithms

Indicates that Elasticsearch should use the RS256 or HS256 signature algorithms to verify the signature of the JWT from the JWT issuer.

pkc_jwkset_path

The file path to a JSON Web Key Set (JWKS) containing the public key material that the JWT realm uses to verify JWT signatures. If a path is provided, then it is resolved relative to the Elasticsearch configuration directory. In Elastic Cloud, use an absolute path starting with /app/config/.

claims.principal

The name of the JWT claim that contains the user’s principal (username).
After defining settings, use the elasticsearch-keystore tool to store values for secure settings in the Elasticsearch keystore.
1. Store the shared_secret value for client_authentication.type:
```
bin/elasticsearch-keystore add xpack.security.authc.realms.jwt.jwt1.client_authentication.shared_secret
```
2. Store the HMAC keys for allowed_signature_algorithms, which use the HMAC SHA-256 algorithm HS256 in the example:
```
bin/elasticsearch-keystore add-file xpack.security.authc.realms.jwt.jwt1.hmac_jwkset <path> 
```
  Path to a JWKS, which is a resource for a set of JSON-encoded secret keys. The file can be removed after you load the contents into the Elasticsearch keystore.
  Using the JWKS is preferred. However, you can add an HMAC key in string format using the following command. This format is compatible with HMAC UTF-8 keys, but only supports a single key with no attributes. You can only use one HMAC format (either hmac_jwkset or hmac_key) simultaneously.
  
  bin/elasticsearch-keystore add xpack.security.authc.realms.jwt.jwt1.hmac_key

JWT encoding and validation

edit

JWTs can be parsed into three pieces:

Header: Provides information about how to validate the token.
Claims: Contains data about the calling user or application.
Signature: The data that’s used to validate the token.

Header: {"typ":"JWT","alg":"HS256"}
Claims: {"aud":"aud8","sub":"security_test_user","iss":"iss8","exp":4070908800,"iat":946684800}
Signature: UnnFmsoFKfNmKMsVoDQmKI_3-j95PCaKdgqqau3jPMY

This example illustrates a partial decoding of a JWT. The validity period is from 2000 to 2099 (inclusive), as defined by the issue time (iat) and expiration time (exp). JWTs typically have a validity period shorter than 100 years, such as 1-2 hours or 1-7 days, not an entire human life.

The signature in this example is deterministic because the header, claims, and HMAC key are fixed. JWTs typically have a nonce claim to make the signature non-deterministic. The supported JWT encoding is JSON Web Signature (JWS), and the JWS Header and Signature are validated using OpenID Connect ID Token validation rules. Some validation is customizable through JWT realm settings.

Header claims

edit

The header claims indicate the token type and the algorithm used to sign the token.

alg: (Required, String) Indicates the algorithm that was used to sign the token, such as HS256. The algorithm must be in the realm’s allow list.
typ: (Optional, String) Indicates the token type, which must be JWT.

Payload claims

edit

OIDC ID tokens contain several claims, which provide information about the user who is issuing the token, and the token itself.

OIDC payload claims

edit

The following claims are validated by a subset of OIDC ID token rules.

Elasticsearch doesn’t validate nonce claims, but a custom JWT issuer can add a random nonce claim to introduce entropy into the signature.

You can relax validation of any of the time-based claims by setting allowed_clock_skew. This value sets the maximum allowed clock skew before validating JWTs with respect to their authentication time (auth_time), creation (iat), not before (nbf), and expiration times (exp).

iss: (Required, String) Denotes the issuer that created the ID token. The value must be an exact, case-sensitive match to the value in the allowed_issuer setting.
aud: (Required, String) Indicates the audiences that the ID token is for, expressed as a comma-separated value (CSV). One of the values must be an exact, case-sensitive match to any of the CSV values in the allowed_audiences setting.
exp: (Required, integer) Expiration time for the ID token, expressed in UTC seconds since epoch.
iat: (Required, integer) Time that the ID token was issued, expressed in UTC seconds since epoch.
nbf: (Optional, integer) Indicates the time before which the JWT must not be accepted, expressed as UTC seconds since epoch.
auth_time: (Optional, integer) Time when the user authenticated to the JWT issuer, expressed as UTC seconds since epoch.

Elasticsearch settings for consuming OIDC claims

edit

Elasticsearch uses OIDC ID token claims for the following settings.

principal: (Required, String) Contains the user’s principal (username). The value is configurable using the realm setting claims.principal. If not set, the value defaults to sub. You can configure an optional regular expression using the claims.principal_pattern to extract a substring.
groups: (Optional, JSON array) Contains the user’s group membership. The value is configurable using the realm setting claims.groups. You can configure an optional regular expression using the realm setting claims.groups_pattern to extract a substring value.
name: (Optional, String) Contains a human-readable identifier that identifies the subject of the token. The value is configurable using the realm setting claims.name. You can configure an optional regular expression using the realm setting claims.name_pattern to extract a substring value.
mail: (Optional, String) Contains the e-mail address to associate with the user. The value is configurable using the realm setting claims.mail. You can configure an optional regular expression using the realm setting claims.mail_pattern to extract a substring value.
dn: (Optional, String) Contains the user’s Distinguished Name (DN), which uniquely identifies a user or group. The value is configurable using the realm setting claims.dn. You can configure an optional regular expression using the realm setting claims.dn_pattern to extract a substring value.

JWT realm authorization

edit

The JWT realm supports authorization with the create or update role mappings API, or delegating authorization to another realm. You cannot use these methods simultaneously, so choose whichever works best for your environment.

You cannot map roles in the JWT realm using the role_mapping.yml file.

Authorizing with the role mapping API

edit

You can use the create or update role mappings API to define role mappings that determine which roles should be assigned to each user based on their username, groups, or other metadata.

PUT /_security/role_mapping/jwt1_users?refresh=true
{
  "roles" : [ "user" ],
  "rules" : { "all" : [
      { "field": { "realm.name": "jwt1" } },
      { "field": { "username": "principalname1" } },
      { "field": { "dn": "CN=Principal Name 1,DC=example.com" } },
      { "field": { "groups": "group1" } },
      { "field": { "metadata.jwt_claim_other": "other1" } }
  ] },
  "enabled": true
}

If you use this API in the JWT realm, the following claims are available for role mapping:

principal: (Required, String) Principal claim that is used as the Elasticsearch user’s username.
dn: (Optional, String) Distinguished Name (DN) that is used as the Elasticsearch user’s DN.
groups: (Optional, String) Comma-separated value (CSV) list that is used as the Elasticsearch user’s list of groups.
metadata: (Optional, object) Additional metadata about the user, such as strings, integers, boolean values, and collections that are used as the Elasticsearch user’s metadata. These values are key value pairs formatted as metadata.jwt_claim_<key> = <value>.

Delegating JWT authorization to another realm

edit

If you delegate authorization to other realms from the JWT realm, only the principal claim is available for role lookup. When delegating the assignment and lookup of roles to another realm from the JWT realm, claims for dn, groups, mail, metadata, and name are not used for the Elasticsearch user’s values. Only the JWT principal claim is passed to the delegated authorization realms. The realms that are delegated for authorization - not the JWT realm - become responsible for populating all of the Elasticsearch user’s values.

The following example shows how you define delegation authorization in the elasticsearch.yml file to multiple other realms from the JWT realm. A JWT realm named jwt2 is delegating authorization to multiple realms:

xpack.security.authc.realms.jwt.jwt2.authorization_realms: file1,native1,ldap1,ad1

You can then use the create or update role mappings API to map roles to the authorizing realm. The following example maps roles in the native1 realm for the principalname1 JWT principal.

PUT /_security/role_mapping/native1_users?refresh=true
{
  "roles" : [ "user" ],
  "rules" : { "all" : [
      { "field": { "realm.name": "native1" } },
      { "field": { "username": "principalname1" } }
  ] },
  "enabled": true
}

If realm jwt2 successfully authenticates a client with a JWT for principal principalname1, and delegates authorization to one of the listed realms (such as native1), then that realm can look up the Elasticsearch user’s values. With this defined role mapping, the realm can also look up this role mapping rule linked to realm native1.

Applying the `run_as` privilege to JWT realm users

edit

Elasticsearch can retrieve roles for a JWT user through either role mapping or delegated authorization. Regardless of which option you choose, you can apply the run_as privilege to a role so that a user can submit authenticated requests to "run as" a different user. To submit requests as another user, include the es-security-runas-user header in your requests. Requests run as if they were issued from that user and Elasticsearch uses their roles.

For example, let’s assume that there’s a user with the username user123_runas. The following request creates a user role named jwt_role1, which specifies a run_as user with the user123_runas username. Any user with the jwt_role1 role can issue requests as the specified run_as user.

POST /_security/role/jwt_role1?refresh=true
{
  "cluster": ["manage"],
  "indices": [ { "names": [ "*" ], "privileges": ["read"] } ],
  "run_as": [ "user123_runas" ],
  "metadata" : { "version" : 1 }
}

You can then map that role to a user in a specific realm. The following request maps the jwt_role1 role to a user with the username user2 in the jwt2 JWT realm. This means that Elasticsearch will use the jwt2 realm to authenticate the user named user2. Because user2 has a role (the jwt_role1 role) that includes the run_as privilege, Elasticsearch retrieves the role mappings for the user123_runas user and uses the roles for that user to submit requests.

POST /_security/role_mapping/jwt_user1?refresh=true
{
  "roles": [ "jwt_role1"],
  "rules" : { "all" : [
      { "field": { "realm.name": "jwt2" } },
      { "field": { "username": "user2" } }
  ] },
  "enabled": true,
  "metadata" : { "version" : 1 }
}

After mapping the roles, you can make an authenticated call to Elasticsearch using a JWT and include the ES-Client-Authentication header:

curl -s -X GET -H "Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJhdWQiOlsiZXMwMSIsImVzMDIiLCJlczAzIl0sInN1YiI6InVzZXIyIiwiaXNzIjoibXktaXNzdWVyIiwiZXhwIjo0MDcwOTA4ODAwLCJpYXQiOjk0NjY4NDgwMCwiZW1haWwiOiJ1c2VyMkBzb21ldGhpbmcuZXhhbXBsZS5jb20ifQ.UgO_9w--EoRyUKcWM5xh9SimTfMzl1aVu6ZBsRWhxQA" -H "ES-Client-Authentication: sharedsecret test-secret" https://localhost:9200/_security/_authenticate

The response includes the user who submitted the request (user2), including the jwt_role1 role that you mapped to this user in the JWT realm:

{"username":"user2","roles":["jwt_role1"],"full_name":null,"email":"user2@something.example.com",
"metadata":{"jwt_claim_email":"user2@something.example.com","jwt_claim_aud":["es01","es02","es03"],
"jwt_claim_sub":"user2","jwt_claim_iss":"my-issuer"},"enabled":true,"authentication_realm":
{"name":"jwt2","type":"jwt"},"lookup_realm":{"name":"jwt2","type":"jwt"},"authentication_type":"realm"}
%

If you want to specify a request as the run_as user, include the the es-security-runas-user header with the name of the user that you want to submit requests as. The following request uses the user123_runas user:

curl -s -X GET -H "Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJhdWQiOlsiZXMwMSIsImVzMDIiLCJlczAzIl0sInN1YiI6InVzZXIyIiwiaXNzIjoibXktaXNzdWVyIiwiZXhwIjo0MDcwOTA4ODAwLCJpYXQiOjk0NjY4NDgwMCwiZW1haWwiOiJ1c2VyMkBzb21ldGhpbmcuZXhhbXBsZS5jb20ifQ.UgO_9w--EoRyUKcWM5xh9SimTfMzl1aVu6ZBsRWhxQA" -H "ES-Client-Authentication: sharedsecret test-secret" -H "es-security-runas-user: user123_runas" https://localhost:9200/_security/_authenticate

In the response, you’ll see that the user123_runas user submitted the request, and Elasticsearch used the jwt_role1 role:

{"username":"user123_runas","roles":["jwt_role1"],"full_name":null,"email":null,"metadata":{},
"enabled":true,"authentication_realm":{"name":"jwt2","type":"jwt"},"lookup_realm":{"name":"native",
"type":"native"},"authentication_type":"realm"}%

PKC JWKS reloading

edit

JWT authentication supports signature verification using PKC (Public Key Cryptography) or HMAC algorithms.

PKC JSON Web Token Key Sets (JWKS) can contain public RSA and EC keys. HMAC JWKS or an HMAC UTF-8 JWK contain secret keys. JWT issuers typically rotate PKC JWKS more frequently (such as daily), because RSA and EC public keys are designed to be easier to distribute than secret keys like HMAC.

JWT realms load a PKC JWKS and an HMAC JWKS or HMAC UTF-8 JWK at startup. JWT realms can also reload PKC JWKS contents at runtime; a reload is triggered by signature validation failures.

HMAC JWKS or HMAC UTF-8 JWK reloading is not supported at this time.

Load failures, parse errors, and configuration errors prevent a node from starting (and restarting). However, runtime PKC reload errors and recoveries are handled gracefully.

All other JWT realm validations are checked before a signature failure can trigger a PKC JWKS reload. If multiple JWT authentication signature failures occur simultaneously with a single Elasticsearch node, reloads are combined to reduce the reloads that are sent externally.

Separate reload requests cannot be combined if JWT signature failures trigger:

PKC JWKS reloads in different Elasticsearch nodes
PKC JWKS reloads in the same Elasticsearch node at different times

Enabling client authentication (client_authentication.type) is strongly recommended. Only trusted client applications and realm-specific JWT users can trigger PKC reload attempts. Additionally, configuring the following JWT security settings is recommended:

allowed_audiences
allowed_clock_skew
allowed_issuer
allowed_signature_algorithms

Authorizing to the JWT realm with an HMAC UTF-8 key

edit

The following settings are for a JWT issuer, Elasticsearch, and a client of Elasticsearch. The example HMAC key is in an OIDC format that’s compatible with HMAC. The key bytes are the UTF-8 encoding of the UNICODE characters.

HMAC UTF-8 keys need to be longer than HMAC random byte keys to achieve the same key strength.

JWT issuer

edit

The following values are for the bespoke JWT issuer.

Issuer:     iss8
Audiences:  aud8
Algorithms: HS256
HMAC UTF-8: hmac-oidc-key-string-for-hs256-algorithm

JWT realm settings

edit

To define a JWT realm, add the following realm settings to elasticsearch.yml.

xpack.security.authc.realms.jwt.jwt8.order: 8 
xpack.security.authc.realms.jwt.jwt8.allowed_issuer: iss8
xpack.security.authc.realms.jwt.jwt8.allowed_audiences: [aud8]
xpack.security.authc.realms.jwt.jwt8.allowed_signature_algorithms: [HS256]
xpack.security.authc.realms.jwt.jwt8.claims.principal: sub
xpack.security.authc.realms.jwt.jwt8.client_authentication.type: shared_secret

In Elastic Cloud, the realm order starts at 2. 0 and 1 are reserved in the realm chain on Elastic Cloud.

JWT realm secure settings

edit

After defining the realm settings, use the elasticsearch-keystore tool to add the following secure settings to the Elasticsearch keystore. In Elastic Cloud, you define settings for the Elasticsearch keystore under Security in your deployment.

xpack.security.authc.realms.jwt.jwt8.hmac_key: hmac-oidc-key-string-for-hs256-algorithm
xpack.security.authc.realms.jwt.jwt8.client_authentication.shared_secret: client-shared-secret-string

JWT realm role mapping rule

edit

The following request creates role mappings for Elasticsearch in the jwt8 realm for the user principalname1:

PUT /_security/role_mapping/jwt8_users?refresh=true
{
  "roles" : [ "user" ],
  "rules" : { "all" : [
      { "field": { "realm.name": "jwt8" } },
      { "field": { "username": "principalname1" } }
  ] },
  "enabled": true
}

Request headers

edit

The following header settings are for an Elasticsearch client.

Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJpc3M4IiwiYXVkIjoiYXVkOCIsInN1YiI6InNlY3VyaXR5X3Rlc3RfdXNlciIsImV4cCI6NDA3MDkwODgwMCwiaWF0Ijo5NDY2ODQ4MDB9.UnnFmsoFKfNmKMsVoDQmKI_3-j95PCaKdgqqau3jPMY
ES-Client-Authentication: SharedSecret client-shared-secret-string

You can use this header in a curl request to make an authenticated call to Elasticsearch. Both the bearer token and the client authorization token must be specified as separate headers with the -H option:

curl -s -X GET -H "Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJpc3M4IiwiYXVkIjoiYXVkOCIsInN1YiI6InNlY3VyaXR5X3Rlc3RfdXNlciIsImV4cCI6NDA3MDkwODgwMCwiaWF0Ijo5NDY2ODQ4MDB9.UnnFmsoFKfNmKMsVoDQmKI_3-j95PCaKdgqqau3jPMY" -H "ES-Client-Authentication: SharedSecret client-shared-secret-string" https://localhost:9200/_security/_authenticate

If you used role mapping in the JWT realm, the response includes the user’s username, their roles, metadata about the user, and the details about the JWT realm itself.

{"username":"user2","roles":["jwt_role1"],"full_name":null,"email":"user2@something.example.com",
"metadata":{"jwt_claim_email":"user2@something.example.com","jwt_claim_aud":["es01","es02","es03"],
"jwt_claim_sub":"user2","jwt_claim_iss":"my-issuer"},"enabled":true,"authentication_realm":
{"name":"jwt2","type":"jwt"},"lookup_realm":{"name":"jwt2","type":"jwt"},"authentication_type":"realm"}
%

« Kerberos authentication Integrating with other authentication systems »

JWT authentication

JWT authentication

JWT uses OIDC workflows

Configure Elasticsearch to use a JWT realm

JWT encoding and validation

Header claims

Payload claims

OIDC payload claims

Elasticsearch settings for consuming OIDC claims

JWT realm authorization

Authorizing with the role mapping API

Delegating JWT authorization to another realm

Applying the run_as privilege to JWT realm users

PKC JWKS reloading

Authorizing to the JWT realm with an HMAC UTF-8 key

JWT issuer

JWT realm settings

JWT realm secure settings

JWT realm role mapping rule

Request headers

Applying the `run_as` privilege to JWT realm users