Security & Users

"How do I manage self managed App Search security, users, and access roles?"

App Search offers three security modes that tie into identity management.

Elasticsearch security features - with a Basic license - are robust.

App Search leverages a similar security model, offering strong security as part of the same Basic Elasticsearch license.

If Elasticsearch security is disabled, anyone with access to the internal address will become an owner of the App Search instance.

The permissive, security disabled setting is not recommended! It’s for experimentation and configuration only.

Please note that certain features described in this guide are only available with an Elasticsearch Platinum License.

Choose from one of three security modes to securely manage access to App Search:

Certain Security and User Management features are only available with an Elasticsearch Platinum license. They are as follows:

Standard security requires configuration of both Elasticsearch and App Search.

It is the default security method when Elasticsearch security is enabled.

Within the Elasticsearch directory, open up the config/elasticsearch.yml file and add:

xpack.security.enabled: true
xpack.security.authc.api_key.enabled: true

Next, you’ll need to setup a password so that App Search can reach Elasticsearch.

Elasticsearch comes with a password setup utility.

It takes two different options:

Either mode will result in a set of passwords for Elasticsearch users.

You’ll need the password for the elastic user — it’s the default user.

bin/elasticsearch-setup-passwords auto

The output will look like so:

Changed password for user apm_system
PASSWORD apm_system = xxxxxxxxxxxxxx

Changed password for user kibana
PASSWORD kibana = xxxxxxxxxxxxxx

Changed password for user logstash_system
PASSWORD logstash_system = xxxxxxxxxxxxxx

Changed password for user beats_system
PASSWORD beats_system = xxxxxxxxxxxxxx

Changed password for user remote_monitoring_user
PASSWORD remote_monitoring_user = xxxxxxxxxxxxxx

Changed password for user elastic <<<
PASSWORD elastic = [ELASTIC_PW]

Retrieve and store the elastic user credential using care and best practices!

To recap, we:

The rest happens within App Search’s configuration files.

Within App Search directory, edit config/app-search.yml and add your credentials:

app_search.auth.source: standard
elasticsearch.username: elastic
elasticsearch.password: [ELASTIC_PW]

That’s it!

Now user settings are managed by App Search.

The default username is: app_search

The default password is generated for you upon start-up. Look into your console log to retrieve it.

You can also pass in your own default password via an environment variable on first start-up:

APP_SEARCH_DEFAULT_PASSWORD=password bin/app-search

Follow the Role Based Access Control guide for more information on how roles work.

And if you haven’t done so, set up a mailer so that users can receive invitations. See Configuring a mailer in the Enterprise Search documentation.

Elasticsearch can manage users using the Elasticsearch internal native realm.

App Search can use the Elasticsearch native realm as its source of truth for identity management.

Within the Elasticsearch directory, open config/elasticsearch.yml and add if it is not present:

xpack.security.enabled: true
xpack.security.authc.api_key.enabled: true
xpack:
  security:
    authc:
      realms:
        native:
          native1:
            order: 0

The internal native realm within Elasticsearch is on by default and its where users will be "stored" and managed.

We’ve also set xpack.security.enabled and xpack.security.authc.api_key.enabled to true.

Follow the standard instructions to use Elasticsearch to create a password.

We’ll set that password and resume the Elasticsearch native realm configuration within App Search.

Within App Search, edit config/app-search.yml and change the app_search.auth.source to elasticsearch-native.

You’ll also need to add a secure password as xpack.security.enabled is now true.

Your config/app-search.yml will look as such:

elasticsearch.username: elastic
elasticsearch.password: [ELASTIC_PW]

app_search.auth.source: elasticsearch-native

Restart App Search.

Upon restart, App Search will now be synchronized with the native realm.

Next, we’ll introduce user management.

User management is done through Elasticsearch.

App Search connects to Elasticsearch user management via Role Mapping.

A Role Mapping makes it so that roles - as App Search understands them - can map to roles as Elasticsearch defines them.

In this way, users are created and managed within the Elasticsearch native realm via API or via Kibana.

Instead of making you manage different sets of users, this allows you to inherit them.

In other words: App Search communicates with Elasticsearch to control user access.

For example, if you wanted to create a user named app-search and provide a password, you’d do that like so:

curl -u elastic:[YOUR_KEY] -X POST "localhost:9200/_security/user/app-search" \
-H 'Content-Type: application/json' \
-d '{
  "password" : "example-password",
  "roles" : [ "app-search-user" ]
}'

A role is required — we provided app-search-user.

Login using this new user: app-search / example-password

Click Users & Roles within the sidebar.

Select Add Mapping.

You’ll see a screen which allows you to select various mapping configurations:

User Management - Create a mapping.

The goal here is to define a Attribute Value for a given External Attribute.

External - in this case - means Elasticsearch.

We created a user above using a role: app-search-user.

If we select "role" as our External Attribute, app-search-user will not be present in the default list.

Elasticsearch accepted the role, but it isn’t considered a "proper role" until the role definition has been added to Elasticsearch via the Role API.

An example API request to create a role looks like so:

curl -u elastic:[YOUR_KEY] -X POST "localhost:9200/_security/role/app-search-user" \
-H 'Content-Type: application/json' \
-d'{
  "cluster": ["all"],
  "indices": [
    {
      "names": [ "index1", "index2" ],
      "privileges": ["all"]
    }
  ],
  "applications": [
    {
      "application": "app-search",
      "privileges": [ "admin", "read" ],
      "resources": [ "*" ]
    }
  ]
}'

In response, you’ll receive:

{
  "role":{
    "created":true
  }
}

The "role" app-search-user will now appear under the list of roles available to the role External Value.

Once you have selected an External Attribute and Attribute Value pair, you must then configure the App Search role.

You can provide a "role", like Admin or Editor and then scope Engine access.

These roles provide access control parameters as App Search understands them.

The mapping is a link between two interpretations of a role:

Click Save Mapping.

Now, any Elasticsearch user which matches the mapping will be able to access App Search.

A user needs to match at least one one mapping so that they can login!

Heads up! If a user matches two mappings, App Search will select the one which is more permissive. In other words, if role A provides read access to one Engine and role B provides write access to all Engines, if a user has both roles, they will inherit the permissions of role B.

SAML support requires an Elasticsearch platinum license.

SAML support leverages the Elasticsearch SAML realm.

It requires a third party SAML provider, like Auth0 or Okta.

We’ll demonstrate how to set-up with Auth0.

Login to Auth0.

Create a new application:

SAML User Management

You’ll want to give it a name - like My Saml App, then select Regular Web Applications:

SAML User Management

Next, click into your application and go into Settings:

SAML User Management

Scroll down until you see Allowed Callback URL.

Enter one that is relative to your app.search.external_url, such as +http://localhost:3002/users/saml_login+, then save:

SAML User Management

Keep scrolling down until you reach Advanced Settings:

SAML User Management

We need the SAML Metadata URL. Securely copy its contents.

That metadata URL contains XML which we need to position close to Elasticsearch. We’ll do so in the Within Elasticsearch step.

The SAML Metadata URL is near the bottom:

SAML User Management

Back in your application menu, head to the Addons section.

From there, locate and activate SAML 2.0:

SAML User Management

Finally, create a user by selecting Users from the main menu and then clicking the + Create User button:

SAML User Management

Fill out the email, a password, and leave connection to its default.

Within the Elasticsearch directory, open config/elasticsearch.yml and add:

(Note the comments!)

xpack.security.enabled: true
xpack.security.authc.api_key.enabled: true
xpack.security.authc.token.enabled: true
xpack.security.transport.ssl.enabled: true

xpack.security.authc.realms.saml.saml_test: # saml_test is our "auth.name" -- we can name anything we'd like: auth0, app_search, etc.
 order: 1
 idp.metadata.path: saml/idp-metadata.xml # We need to create this directory and place the file here.
 idp.entity_id: "urn:dev-abc-1234.auth0.com" # Taken from the SAML Metadata URL
 sp.entity_id: "http://localhost:3002/" # Match these with your external_url.
 sp.acs: "http://localhost:3002/users/saml_login"
 sp.logout: "http://localhost:3002/logout"
 attributes.principal: "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress"
 attributes.mail: "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress"

Within the config directory, create a saml directory and a idp-metadata.xml file within it.

Paste the contents of the SAML Metadata URL file into the idp-metadata.xml file.

The URL has a prefix similar to the example: dev-abc-1234.auth0.com.

Be sure you have added that string as the idp.entity_id, prefixed with urn: as shown above.

Restart Elasticsearch.

Follow the standard instructions to use Elasticsearch to create a password.

We’ll set that password and resume the SAML configuration within App Search.

Within App Search directory, edit config/app-search.yml update the app_search.auth.source:

elasticsearch.username: elastic
elasticsearch.password: [ELASTIC_PW]

app_search.auth.source: elasticsearch-saml
app_search.auth.name: saml_test

Note that the auth.name must match what is within Elasticsearch’s SAML realm.

Visit your App Search URL and try to login.

You will be prompted to do so — if you created an Auth0 user in the step above, use its credentials.

Within your SAML provider, you’ll be able to create users to interact with the Elasticsearch native realm.

You can then use App Search’s Role Mapping to coordinate with Elasticsearch.

That’s it! You’ve successfully connected SAML.

You can find the sp.* host for App Search (the [APP SEARCH ENDPOINT] in the examples below) on Elastic Cloud by clicking on "Copy endpoint" next to the App Search application in the Deployment details page.

An example of how the elasticsearch.yml file would differ on Elastic Cloud. Note that you would change this in the "User setting overrides" section of your edit deployment page.

xpack.security.enabled: true
xpack.security.authc.api_key.enabled: true
xpack.security.authc.token.enabled: true
xpack.security.transport.ssl.enabled: true

xpack.security.authc.realms.saml.saml_test: # saml_test is our "auth.name" -- we can name anything we'd like: auth0, app_search, etc.
 order: 1
 idp.metadata.path: "[METADATA URL]" # This could be a link to your identity provider's metadata, or an XML file.
 idp.entity_id: "[SAML ENTITY ID]" # This is provided by your identity provider, as the "Identity Provider Issuer" or "Domain".
 sp.entity_id: "[APP SEARCH ENDPOINT]/" # Match these with your external_url.
 sp.acs: "[APP SEARCH ENDPOINT]/users/saml_login"
 sp.logout: "[APP SEARCH ENDPOINT]/logout"
 attributes.principal: "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress" # This will depend on your identity provider.
 attributes.groups: "groups"

Run the binary with an attached --reset-auth command: