Variables and conditions in input configurations
editVariables and conditions in input configurations
editWhen running Elastic Agent in some environments, you might not know all the input configuration details up front. To solve this problem, the input configuration accepts variables and conditions that get evaluated at runtime using information from the running environment. Similar to autodiscovery, these capabilities allow you to apply configurations dynamically.
Let’s consider a unique agent policy that is deployed on two machines: a Linux
machine named "linux-app" and a Windows machine named "winapp". Notice that
the configuration has some variable references: ${host.name}
and
${host.platform}
:
inputs: - id: unique-logfile-id type: logfile streams: - paths: /var/log/${host.name}/another.log condition: ${host.platform} == "linux" - path: c:/service/app.log condition: ${host.platform} == "windows"
At runtime, Elastic Agent resolves variables and evaluates the conditions based on values provided by the environment, generating two possible input configurations.
On the Windows machine:
inputs: - id: unique-logfile-id type: logfile streams: - path: c:/service/app.log
On the Linux machine:
inputs: - id: unique-logfile-id type: logfile streams: - paths: /var/log/linux-app/another.log
Using variable substitution along with conditions allows you to create concise, but flexible input configurations that adapt to their deployed environment.
Variable substitution
editThe syntax for variable substitution is ${var}
, where var
is the name of a
variable defined by a provider. A provider defines key/value pairs that are
used for variable substitution and conditions.
Elastic Agent supports a variety of providers, such as host
and local
, that
supply variables to Elastic Agent. For example, earlier you saw ${host.name}
used to
resolve the path to the host’s log file based on the {host.platform}
value. Both of these values
were provided by the host
provider.
All providers are enabled by default when Elastic Agent starts. If a provider cannot be configured, its variables are ignored.
Refer to Providers for more detail.
The following agent policy uses a custom key named foo
to resolve a value
defined by a local provider:
inputs: - id: unique-logfile-id type: logfile streams: - paths: /var/log/${foo}/another.log providers: local: vars: foo: bar
The policy generated by this configuration looks like this:
inputs: - id: unique-logfile-id type: logfile streams: - paths: /var/log/bar/another.log
When an input uses a variable substitution that is not present in the current key/value mappings being evaluated, the input is removed in the result.
For example, this agent policy uses an unknown key:
inputs: - id: logfile-foo type: logfile path: "/var/log/foo" - id: logfile-unknown type: logfile path: "${ unknown.key }"
The policy generated by this configuration looks like this:
inputs: - id: logfile-foo type: logfile path: "/var/log/foo"
Alternative variables and constants
editVariable substitution can also define alternative variables or a constant.
To define a constant, use either '
or "
. When a constant is reached during
variable evaluation, any remaining variables are ignored, so a constant should
be the last entry in the substitution.
To define alternatives, use |
followed by the next variable or constant.
The power comes from allowing the input to define the preference order of the
substitution by chaining multiple variables together.
For example, the following agent policy chains together multiple variables to
set the log path based on information provided by the running container
environment. The constant /var/log/other
is used to end of the path, which is
common to both providers:
inputs: - id: logfile-foo type: logfile path: "/var/log/foo" - id: logfile-container type: logfile path: "${docker.paths.log|kubernetes.container.paths.log|'/var/log/other'}"
Conditions
editA condition is a boolean expression that you can specify in your agent policy to control whether a configuration is applied to the running Elastic Agent. You can set a condition on inputs, streams, or even processors.
In this example, the input is applied if the host platform is Linux:
inputs: - id: unique-logfile-id type: logfile streams: - paths: - /var/log/syslog condition: ${host.platform} == 'linux'
In this example, the stream is applied if the host platform is not Windows:
inputs: - id: unique-system-metrics-id type: system/metrics streams: - metricset: load data_stream.dataset: system.cpu condition: ${host.platform} != 'windows'
In this example, the processor is applied if the host platform is not Windows:
inputs: - id: unique-system-metrics-id type: system/metrics streams: - metricset: load data_stream.dataset: system.cpu processors: - add_fields: fields: platform: ${host.platform} to: host condition: ${host.platform} != 'windows'
Condition syntax
editThe conditions supported by Elastic Agent are based on EQL's boolean syntax, but add support for variables from providers and functions to manipulate the values.
Supported operators:
-
Full PEMDAS math support for
+ - * / %
. -
Relational operators
< <= >= > == !=
-
Logical operators
and
andor
Functions:
Types:
-
Booleans
true
andfalse
Condition examples
editRun only when a specific label is included.
arrayContains(${docker.labels}, 'monitor')
Skip on Linux platform or macOS.
${host.platform} != "linux" and ${host.platform} != "darwin"
Run only for specific labels.
arrayContains(${docker.labels}, 'monitor') or arrayContains(${docker.label}, 'production')
Function reference
editThe condition syntax supports the following functions.
add
editadd(Number, Number) Number
Usage:
add(1, 2) == 3 add(5, ${foo}) >= 5
arrayContains
editarrayContains(Array, String) Boolean
Usage:
arrayContains(${docker.labels}, 'monitor')
concat
editconcat(String, String) String
Parameters are coerced into strings before the concatenation.
Usage:
concat("foo", "bar") == "foobar" concat(${var1}, ${var2}) != "foobar"
divide
editdivide(Number, Number) Number
Usage:
divide(25, 5) > 0 divide(${var1}, ${var2}) > 7
endsWith
editendsWith(String, String) Boolean
Usage:
endsWith("hello world", "hello") == true endsWith(${var1}, "hello") != true
hasKey
edithasKey(Dictionary, String) Boolean
Usage:
hasKey(${host}, "platform")
indexOf
editindexOf(String, String, Number?) Number
Returns -1 if the string is not found.
Usage:
indexOf("hello", "llo") == 2 indexOf(${var1}, "hello") >= 0
length
editlength(Array|Dictionary|string)
Usage:
length("foobar") > 2 length(${docker.labels}) > 0 length(${host}) > 2
match
editmatch(String, Regexp) boolean
Regexp
supports Go’s regular expression syntax. Conditions that use
regular expressions are more expensive to run. If speed is critical, consider
using endWiths
or startsWith
.
Usage:
match("hello world", "^hello") == true match(${var1}, "world$") == true
modulo
editmodulo(number, number) Number
Usage:
modulo(25, 5) > 0 modulo(${var1}, ${var2}) == 0
multiply
editmultiply(Number, Number) Number
Usage:
multiply(5, 5) == 25 multiple(${var1}, ${var2}) > x
number
editnumber(String) Integer
Usage:
number("42") == 42 number(${var1}) == 42
startsWith
editstartsWith(String, String) Boolean
Usage:
startsWith("hello world", "hello") == true startsWith(${var1}, "hello") != true
string
editstring(Number) String
Usage:
string(42) == "42" string(${var1}) == "42"
stringContains
editstringContains(String, String) Boolean
Usage:
stringContains("hello world", "hello") == true stringContains(${var1}, "hello") != true
subtract
editsubtract(Number, Number) Number
Usage:
subtract(5, 1) == 4 subtract(${foo}, 2) != 2
Debugging
editTo debug configurations that include variable substitution and conditions, use
the inspect
command. This command shows the configuration that’s generated
after variables are replaced and conditions are applied.
First run the Elastic Agent. For this example, we’ll use the following agent policy:
outputs: default: type: elasticsearch hosts: [127.0.0.1:9200] apikey: <my-api-key> providers: local_dynamic: items: - vars: key: value1 processors: - add_fields: fields: custom: match1 target: dynamic - vars: key: value2 processors: - add_fields: fields: custom: match2 target: dynamic - vars: key: value3 processors: - add_fields: fields: custom: match3 target: dynamic inputs: - id: unique-logfile-id type: logfile enabled: true streams: - paths: - /var/log/${local_dynamic.key}
Then run elastic-agent inspect --variables
to see the generated configuration. For
example:
$ ./elastic-agent inspect --variables inputs: - enabled: true id: unique-logfile-id-local_dynamic-0 original_id: unique-logfile-id processors: - add_fields: fields: custom: match1 target: dynamic streams: - paths: - /var/log/value1 type: logfile - enabled: true id: unique-logfile-id-local_dynamic-1 original_id: unique-logfile-id processors: - add_fields: fields: custom: match2 target: dynamic streams: - paths: - /var/log/value2 type: logfile - enabled: true id: unique-logfile-id-local_dynamic-2 original_id: unique-logfile-id processors: - add_fields: fields: custom: match3 target: dynamic streams: - paths: - /var/log/value3 type: logfile outputs: default: apikey: <my-api-key> hosts: - 127.0.0.1:9200 type: elasticsearch providers: local_dynamic: items: - processors: - add_fields: fields: custom: match1 target: dynamic vars: key: value1 - processors: - add_fields: fields: custom: match2 target: dynamic vars: key: value2 - processors: - add_fields: fields: custom: match3 target: dynamic vars: key: value3 ---