WARNING: Version 5.4 of the Elastic Stack has passed its EOL date.
This documentation is no longer being maintained and may be removed. If you are running this version, we strongly advise you to upgrade. For the latest information, see the current release documentation.
Migrating from Shield
editMigrating from Shield
editWith the move to X-Pack, the biggest changes to security functionality are the names of the security configuration options, TLS/SSL configuration, and how roles are defined. A few privileges have also been removed. This section describes the changes you need to make when upgrading from Shield.
Also, all endpoints have been renamed from /_shield/XYZ
to /_xpack/security/XYZ
.
Migrating Settings
editMost Shield settings have simply had their prefix changes from shield.
to
xpack.security.
. Hare a few examples of Shield 2.x settings and their new X-Pack
counterparts:
Shield 2.x Setting Name | X-Pack 5.0 Setting Name |
---|---|
|
|
|
|
Persistent cluster state settings used for IP Filtering are not automatically migrated to the new names. You will need to manually migrate these settings.
TLS/SSL Settings
editX-Pack introduces a simpler way of configuring TLS/SSL within your cluster and as part of this effort, some of the settings have changed or been removed. Also, new options have been introduced, which includes the ability to more easily separate default TLS/SSL configurations from the transport configuration.
There is a hierarchy to TLS/SSL settings in X-Pack, which allows for a simple
configuration for everything or customizations to be applied to individual
components. The X-Pack wide configuration are the settings that being with
xpack.ssl
. X-Pack security adds configuration for HTTP under the settings
that begin with xpack.security.http.ssl.
and Transport under the settings that
begin with xpack.security.transport.ssl.
. Transport profiles can still have
their own configurations with settings that begin with
transport.profiles.$PROFILE.xpack.security.ssl.
and they will fallback to
the default transport configuration. Realms can also have their own
configurations, which are available as settings that begin with
xpack.security.authc.realms.$REALM_NAME.ssl.
.
There is also an important change in terms of how hostname verification is done by X-Pack security. Previously, a reverse DNS lookup would be made to determine the name of the address being connected to. This reverse DNS lookup has been removed and the name/address provided when connecting is used. Please make sure the the publish host on your nodes is one of the values in the certificate.
The following settings have been replaced with new settings:
|
Use |
|
Use |
|
Use |
|
This boolean setting has been replaced by |
|
This boolean setting has been replaced by |
The following settings have been removed:
|
This setting controlled the name resolution for SSL traffic, but was not easy to use and hard to set up. If certificates are not set with
the proper DNS names, you should configure |
|
The value for the SSL session cache size should be configured through the use of the system property |
|
The SSL session cache timeout being set incorrectly can negatively impact performance and was not widely used, so it has been removed and the default value of the JDK is used. |
Migrating File-based Users and Roles to the Native Realm
editFrom 5.0 onward, you should use the native
realm to manage roles and local
users. To migrate existing file-based users and roles to the native realm, use
the migrate
tool that’s included with the X-Pack plugin.
When migrating from Shield 2.x, the migrate
tool should be run prior
to upgrading to ensure all roles can be migrated as some may be in a deprecated
format that X-Pack cannot read. The migrate
tool is available in Shield
2.4.0 and higher.
The migrate
tool loads the existing file-based users and roles and calls the
user and roles APIs to add them to the native realm. You can migrate all users
and roles, or specify the ones you want to migrate. Users and roles that
already exist in the native
realm are not replaced or overridden. If
the names you specify with the --users
and --roles
options don’t
exist in the file
realm, they are skipped.
Run the migrate tool after you install the X-Pack plugin. For example:
$ bin/x-pack/migrate native -U http://localhost:9200 -u elastic -p changeme -n lee,foo -r role1,role2,role3,role4,foo starting migration of users and roles... importing users from [/home/es/config/shield/users]... found existing users: [test_user, joe3, joe2] migrating user [lee] {"user":{"created":true}} no user [foo] found, skipping importing roles from [/home/es/config/shield/roles.yml]... found existing roles: [marvel_user, role_query_fields, admin_role, role3, admin, remote_marvel_agent, power_user, role_new_format_name_array, role_run_as, logstash, role_fields, role_run_as1, role_new_format, kibana4_server, user, transport_client, role1.ab, role_query] migrating role [role1] {"role":{"created":true}} migrating role [role2] {"role":{"created":true}} role [role3] already exists, skipping no role [foo] found, skipping users and roles imported.
The native
subcommand supports the following options:
-
-U
,--url
- Endpoint URL of the Elasticsearch cluster to which you want to migrate the file-based users and roles. Required.
-
-u
,--username
- Username to use for authentication.
-
-p
,--password
- Password to use for authentication.
-
-n
,--users
- Comma-separated list of the users you want to migrate. If not specified, all users are migrated.
-
-r
,--roles
- Comma-separated list of the roles you want to migrate. If not specified, all roles are migrated.
Additionally, the -E
flag can be used to specify additional settings. For example
to specify a different configuration directory, the command would look like:
$ bin/x-pack/migrate native -U http://localhost:9200 -u elastic -p changeme -Epath.conf=/etc/elasticsearch
File Configuration Changes
editIn Shield, the location of the users
, users_roles
, roles.yml
, and system_key
files could be changed using settings. In X-Pack, these files must always reside
in the CONFIG_DIR/x-pack
directory.
esusers
Realm Changes
editIn Shield, esusers
was the type for the file based realm. This realm type has been
renamed to file
in X-Pack.
Additionally, the names of users in the file
based realm are not allowed to conflict
with the built-in users.
Role Format Changes
editIn X-Pack, the format for specifying roles in the roles.yml
file is much stricter:
- Role names are not allowed to conflict with the built-in roles.
-
Index names and expressions, such as
.kibana
orlogstash-*
, may no longer be used as thekey
for a set of indices permissions. - Support for comma-separated values and single values has been removed in favor of collections or arrays.
-
Syntax for field level security has changed. Instead of providing a list of allowed
fields
, the allowed fields are now inside an objectfield_security
. The new syntax allows to also define exceptions. See field-level-security for more details.
The easiest way to migrate your roles, will be by converting the roles to native
roles
using the migrate
tool prior to upgrading to X-Pack.
The following role shows how to specify multiple cluster and index permissions using collections.
Using collection notation to define roles.
get_user: cluster: - monitor - manage_security indices: - names: - events_index - results privileges: - read - monitor - names: - user_index privileges: - monitor - read query: term: department_id: 12 field_security: grant: - name - email - contact.* except: - contact.mobile
The name of the role is |
|
The cluster privileges are defined as a collection |
|
The first set of indices permissions contained in this role. This permission
applies to the |
|
The second set of indices permissions contained in this role. This permission
applies to the |
You can also use arrays to specify multiple cluster and index permissions. The following role uses the array notation to define the same role as in the previous example.
Using array notation to define roles.
get_user: cluster: [ monitor, manage_security ] indices: - names: [ events_index, results ] privileges: [ read, monitor ] - names: user_index privileges: [ monitor, read ] query: term: department_id: 12 field_security: grant: [ name, email, contact.* ] except: [ contact.mobile ]
Values for cluster
, privileges
, and the entries in fields_security
must be specified as a collection
or array even if there’s only one value. You can specify a single value for the
names
field.
If you have roles that use an index name or expression as a key, you’ll need to update them. For example, the following role definitions are no longer valid.
# Unsupported role format: index name as key and privilege as value get_user: indices: 'events_index': read
# Unsupported role format: index name as the key with privileges as a value within the object get_user: indices: 'events_index': privileges: read
You can rewrite this role to work with X-Pack using collection notation:
This line is the start of a privilege definition for a set of indices. Note
that this line is indented further underneath |
Similarly, if you have roles that use comma-separated values to specify multiple values, you need to rewrite them to use the collection or array notation. For example, the following role definition is no longer valid.
# Unsupported role format: comma-separated values get_user: cluster: monitor, manage indices: 'events_index,results': - privileges: read, write, create_index - field_security: - grant: fieldA, fieldB
You can rewrite this role using either collections or arrays.
Specifying multiple values with a collection.
get_user: cluster: - monitor - manage indices: - names : - 'events_index' - 'results' privileges: - read - write - create_index field_security: grant: - fieldA - fieldB
Specifying multiple values with an array.
get_user: cluster: [ 'monitor', 'manage' ] indices: - names: [ 'events_index', 'results' ] privileges: [ 'read', 'write', 'create_index' ] field_security: grant: [ 'fieldA', 'fieldB' ]
If you have roles that specify single values for cluster
, privileges
,
and fields
, you need to rewrite them to use the collection or array notation.
For example, the following role definition is no longer valid.
# Unsupported role format: single values get_user: cluster: manage indices: 'events_index': - privileges: read field_security: grant: fieldA
You can rewrite this role using collections or arrays (or a combination of
the two). Note that the names
field still accepts a single value.
Specifying multiple values with collections and arrays.
get_user: cluster: - manage indices: - names: 'events_index' privileges: - read field_security: grant: [ 'fieldA' ]
Removed Privileges
editA few of the privileges that were available in Shield 2.x are no longer available in X-Pack. The following tables list the privileges that have been removed and their suggested replacements.
Table 1. Cluster Privileges
Removed Privilege | Replacement Privilege |
---|---|
|
|
|
|
|
|
Table 2. Indices Privileges
Removed Privilege | Replacement Privilege |
---|---|
|
|
|
|
|
|
|
|
|
|
|
|
Field level security and the percolator
editThe percolator has been refactored and now requires a percolator
field in the mapping.
As a result, when using field level security with the percolator, access to the percolator
field needs to be granted.
Auditing Changes
editA new event type, realm_authentication_failed
, has been added. These events were
previously audited as authentication_failed
events but in some cases produced confusing
output. The realm_authentication_failed
event indicates that the credentials provided by
the user failed authentication for a specific realm; it does not indicate a failed login
attempt as there could be other realms to try to authenticate against.
The logfile
output no longer relies on the logger level to determine
what events to output. The xpack.security.audit.logfile.events.include
setting can be used
to specify the events that should be output; you may also use the
xpack.security.audit.logfile.events.exclude
setting to simply exclude values from the
defaults.
Some audit events would output the content of a REST request if the log level was configured at
the TRACE
level; control over this output has been replaced with the settings
xpack.security.audit.logfile.events.emit_request_body
and
xpack.security.audit.index.events.emit_request_body
for the logfile and index outputs. The
default is false
.
Finally, as part of the move to Log4j2 the logging.yml
file has been replaced with the
log4j2.properties
file.
Run As Changes
editIn X-Pack security, the default header name for submitting requests on behalf of other users has changed
to es-security-runas-user
. Applications making use of the run as functionality will need to use the new header name.