From version 5.0 onward, Watcher is part of X-Pack. For more information, see Alerting on cluster and index events.

› ›

Inputs

A watch input loads data into a watch’s execution context as the initial payload. Watcher supports four input types: simple , search, http and chain

If you don’t define an input for a watch, an empty payload is loaded into the execution context.

Simple Input

edit

An input that enables you to load static data into a watch’s execution context as the initial payload. The simple input is useful when the data you want to work with doesn’t need to be loaded dynamically, but for maintainability you want to store the data centrally and reference it with templates.

You can define the static data as a string (str), numeric value (num), or an object (obj):

{
  "input" : {
    "simple" : {
      "str" : "val1",
      "num" : 23,
      "obj" : {
        "str" : "val2"
      }
    }
  }
  ...
}

For example, the following watch uses the simple input to set the recipient name for a reminder email that’s sent every day at noon.

{
  "trigger" : {
    "schedule" : {
      "daily" : { "at" : "noon" }
    }
  },
  "input" : {
    "simple" : {
      "name" : "John"
    }
  },
  "actions" : {
    "reminder_email" : {
      "email" : {
        "to" : "to@host.domain",
        "subject" : "Reminder",
        "body" : "Dear {{ctx.payload.name}}, by the time you read these lines, I'll be gone"
      }
    }
  }
}

Search Input

edit

An input that enables you to search the Elasticsearch cluster that Watcher is running on and load the response into a watch’s execution context as the initial payload. See Table 2, “Search Input Attributes” for the supported attributes.

Conditions, transforms, and actions can access the search results through the watch execution context. For example:

To load all of the search hits into an email body, use ctx.payload.hits.
To reference the total number of hits, use ctx.payload.hits.total.
To access a particular hit, use its zero-based array index. For example, to get the third hit, use ctx.payload.hits.hits.2.
To get a field value from a particular hit, use ctx.payload.hits.hits.<index>.fields.<fieldname>. For example, to get the message field from the first hit, use ctx.payload.hits.hits.0.fields.message.

Table 2. Search Input Attributes

Name	Required	Default	Description
`request.search_type`	no	`query_then_fetch`	The type of search request to perform. Valid values are: `dfs_query_and_fetch`, `dfs_query_then_fetch`, `query_and_fetch`, `query_then_fetch`, and `scan`. The Elasticsearch default is `query_then_fetch`.
`request.indices`	no	-	The indices to search. If omitted, all indices are searched, which is the default behaviour in Elasticsearch.
`request.types`	no	-	The document types to search for. If omitted, all document types are are searched, which is the default behaviour in Elasticsearch.
`request.body`	no	-	The body of the request. The request body follows the same structure you normally send in the body of a REST `_search` request. The body can be static text or include `mustache` templates.
`request.template`	no	-	The body of the search template. See configure templates for more information.
`request.indices_options.expand_wildcards`	no	`open`	How to expand wildcards. Valid values are: `all`, `open`, `closed`, and `none` See `expand_wildcards` for more information.
`request.indices_options.ignore_unavailable`	no	`true`	Whether the search should ignore unavailable indices. See `ignore_unavailable` for more information.
`request.indices_options.allow_no_indices`	no	`true`	Whether to allow a search where a wildcard indices expression results in no concrete indices. See allow_no_indices for more information.
`extract`	no	-	A array of JSON keys to extract from the search response and load as the payload. When a search generates a large response, you can use `extract` to select the relevant fields instead of loading the entire response.
`timeout`	no	30s	The timeout for waiting for the search api call to return. If no response is returned within this time, the search input times out and fails. This setting overrides the default internal search operations timeouts.

You can reference the following variables in the execution context when specifying the request body:

Name	Description
`ctx.watch_id`	The id of the watch that is currently executing.
`ctx.execution_time`	The time execution of this watch started.
`ctx.trigger.triggered_time`	The time this watch was triggered.
`ctx.trigger.scheduled_time`	The time this watch was supposed to be triggered.
`ctx.metadata.*`	Any metadata associated with the watch.

Submitting Searches

edit

You can use the search input to submit any valid search request to your Elasticsearch cluster. For example, the following snippet returns all event documents in the logs index.

"input" : {
  "search" : {
    "request" : {
      "indices" : [ "logs" ],
      "types" : [ "event" ],
      "body" : {
        "query" : { "match_all" : {}}
      }
    }
  }
}

Extracting Specific Fields

edit

You can specify which fields in the search response you want to load into the watch payload with the extract attribute. This is useful when a search generates a large response and you are only interested in particular fields.

For example, the following input loads only the total number of hits into the watch payload:

"input": {
    "search": {
      "request": {
        "indices": [".watch_history*"]
      },
      "extract": ["hits.total"]
    }
  },

Using Templates

edit

The search input supports search templates. For example, the following snippet references the indexed template called my_template and passes a value of 23 to fill in the template’s value parameter.

{
  "input" : {
    "search" : {
      "request" : {
        "indices" : [ "logs" ],
        "template" : {
          "id" : "my_template",
          "params" : {
            "value" : 23
          }
        }
      }
    }
  }
  ...
}

Applying Conditions

edit

The search input is often used in conjunction with the script condition. For example, the following snippet adds a condition to check if the search returned more than five hits

{
  "input" : {
    "search" : {
      "request" : {
        "indices" : [ "logs" ],
        "body" : {
          "query" : { "match_all" : {} }
        }
      }
    }
  },
  "condition" : {
    "script" : "return ctx.payload.hits.total > 5"
  }
  ...
}

HTTP Input

edit

An input that enables you to query an HTTP endpoint and load theresponse into a watch’s execution context as the initial payload. See Table 3, “HTTP Input Attributes” for the supported attributes.

For example, you can use the http input to:

Query an external Elasticsearch cluster. The query that one can define in the search input can also defined in the http input. This in particular interesting as one can query clusters that are version wise incompatible with the cluster where Watcher is running on. With the search input these cluster would otherwise not be accessible. Also with the http input it is straight forward to run a dedicated Watcher cluster that queries other clusters.
Query Elasticsearch APIs other than the search API. For example, the Nodes Stats, Cluster Health or Cluster State APIs.
Query an external webservice. Any service that exposes an HTTP endpoint can be queried by the http input. This can be very useful when you need to bridge between an Elasticsearch cluster and other systems.

Conditions, transforms, and actions can access the JSON response through the watch execution context. For example, if the response contains a message object, you could use ctx.payload.message to get the message from the payload.

If the body of the response from the HTTP endpoint is in the JSON or YAML format it will be parsed and used as the initial payload. Any other value that is returned will be assigned and accessible to/via the _value variable of the payload.

Table 3. HTTP Input Attributes

Name	Required	Default	Description
`request.scheme`	no	http	Url scheme. Valid values are: `http` or `https`.
`request.host`	yes	-	The host to connect to.
`request.port`	yes	-	The port the http service is listening on.
`request.path`	no	-	The URL path. The path can be static text or contain `mustache` templates. URL query string parameters must be specified via the `request.params` attribute.
`request.method`	no	get	The HTTP method. Supported values are: `head`, `get`, `post`, `put` and `delete`.
`request.headers`	no	-	The HTTP request headers. The header values can be static text or include `mustache` templates.
`request.params`	no	-	The URL query string parameters. The parameter values can be static text or contain `mustache` templates.
`request.url`	no	-	Allows you to set `request.scheme`, `request.host`, `request.port` and `request.params` add once by specifying a real URL, like `https://www.example.org:1234/mypath?foo=bar`. May not be combined with on of those four parameters. As those parameters are set, specifying them individually might overwrite them.
`request.auth`	no	-	Authentication related HTTP headers. Currently, only basic authentication is supported.
`request.proxy.host`	no	-	The proxy host to use when connecting to the host.
`request.proxy.port`	no	-	The proxy port to use when connecting to the host.
`request.connection_timeout`	no	10s	The timeout for setting up the http connection. If the connection could not be set up within this time, the input will timeout and fail. It is also possible to configure the default connection timeout for all http connection timeouts.
`request.read_timeout`	no	10s	The timeout for reading data from http connection. If no response was received within this time, the input will timeout and fail. It is also possible to configure the default read timeout for all http connection timeouts.
`request.body`	no	-	The HTTP request body. The body can be static text or include `mustache` templates.
`extract`	no	-	A array of JSON keys to extract from the input response and use as payload. In cases when an input generates a large response this can be used to filter the relevant piece of the response to be used as payload.
`response_content_type`	no	json	The expected content type the response body will contain. Supported values are `json`, `yaml` and `text`. If the format is `text` the `extract` attribute cannot exist. Note that this overrides the header that is returned in the HTTP response. If this is set to `text` the body of the response will be assigned and accessible to/via the `_value` variable of the payload.

You can reference the following variables in the execution context when specifying the path, params, headers and body values:

Name	Description
`ctx.watch_id`	The id of the watch that is currently executing.
`ctx.execution_time`	The time execution of this watch started.
`ctx.trigger.triggered_time`	The time this watch was triggered.
`ctx.trigger.scheduled_time`	The time this watch was supposed to be triggered.
`ctx.metadata.*`	Any metadata associated with the watch.

Accessing the HTTP Response

edit

If the response body is formatted in JSON or YAML, it is parsed and loaded into the execution context. If the response body is not formatted in JSON or YAML, it is loaded into the payload’s _value field.

Conditions, transforms, and actions access the response data through the execution context. For example, if the response contains a message object, you can use ctx.payload.message to access the message data.

In addition all the headers from the response can be accessed using the ctx.payload._headers field as well as the HTTP status code of the response using ctx.payload._status_code.

Querying External Elasticsearch Clusters

edit

The following snippet shows a basic http input that searches for all documents in the idx index in an external cluster:

"input" : {
  "http" : {
    "request" : {
      "host" : "example.com",
      "port" : 9200,
      "path" : "/idx/_search"
    }
  }
}

You can use the full Elasticsearch Query DSL to perform more sophisticated searches. For example, the following snippet retrieves all documents that contain event in the category field.

"input" : {
  "http" : {
    "request" : {
      "host" : "host.domain",
      "port" : 9200,
      "path" : "/idx/_search",
      "body" :  "{\"query\" :  {  \"match\" : { \"category\" : \"event\"}}}"
    }
  }
}

Using Templates

edit

The http input supports templating. You can use templates when specifying the path, body, header values, and parameter values.

For example, the following snippet uses templates to specify what index to query and restrict the results to documents added within the last five minutes.

"input" : {
  "http" : {
    "request" : {
      "host" : "host.domain",
      "port" : 9200,
      "path" : "/{{ctx.watch_id}}/_search",
      "body" : "{\"query\" : {\"range\": {\"@timestamp\" : {\"from\": \"{{ctx.trigger.triggered_time}}||-5m\",\"to\": \"{{ctx.trigger.triggered_time}}\"}}}}"
      }
    }
  }

Calling Elasticsearch APIs

edit

You can use http input load the data returned by any Elasticsearch API. For example, the following snippet calls the Cluster Stats API and passes in the human query string argument.

Http Input.

"input" : {
  "http" : {
    "request" : {
      "host" : "host.domain",
      "port" : "9200",
      "path" : "/_cluster/stats",
      "params" : {
        "human" : "true" 
      }
    }
  }
}

Enabling this attribute returns the bytes values in the response in human readable format.

Calling External Webservices

edit

You can use http input to get data from any external webservice. The http input supports basic authentication. For example, the following snippet calls myservice and uses basic authentication:

Http Input.

"input" : {
  "http" : {
    "request" : {
      "host" : "host.domain",
      "port" : "9200",
      "path" : "/myservice",
      "auth" : {
        "basic" : {
          "username" : "user",
          "password" : "pass"
        }
      }
    }
  }
}

Chain Input

edit

An input that enables you to string together multiple inputs to load data into the watch execution context. Each input loads data into the execution context and that data is available to subsequent inputs in the chain.

The chain input is useful when you want to perform an action based on data from multiple sources. You can also use the data collected by one input to load data from another source.

To reference data loaded by a particular input, you use the input’s name, ctx.payload.<input-name>.<value>.

For example, the following chain input loads data from an HTTP server using the path set by a simple input.

{
  "input" : {
    "chain" : {
      "inputs" : [ 
        {
          "first" : {
            "simple" : { "path" : "/_search" }
          }
        },
        {
          "second" : {
            "http" : {
              "request" : {
                "host" : "localhost",
                "port" : 9200,
                "path" : "{{ctx.payload.first.path}}" 
              }
            }
          }
        }
      ]
    }
  }
  ...
}

	The inputs in a chain are specified as an array to guarantee the order in which the inputs are processed. (JSON does not guarantee the order of arbitrary objects.)
	Loads the `path` set by the `first` input.

« Reference Triggers »