JSON content discovery
editJSON content discovery
editThe Elastic Serverless Forwarder is able to automatically discover JSON content in the payload of an input and collect the JSON objects contained in the payload.
The JSON objects can either be on a single line or spanning multiple lines. In the second case, the forwarder expects different JSON objects spanning multiple lines to be separated by a newline delimiter.
When JSON objects span multiple lines, a limit of 1000 lines is applied. Every JSON object spanning across more than 1000 lines will not be collected. Every line composing the whole JSON object will be forwarded individually instead.
If you have known payload content which includes single JSON objects that span more than 1000 lines, or if you find that relying on auto-discovery of JSON content has a big impact on performance, you can configure JSON content types within the inputs to address this. This will change the parsing logic and improve performance while overcoming the 1000 lines limit.
Where content is known to be plain text, you can improve overall performance by disabling automatic JSON content discovery completely.
To change this configuration option, set inputs.[].json_content_type to one of the following values:
- single: indicates that the content of a single item in the input payload is a single JSON object. The content can either be on a single line or spanning multiple lines. With this setting the whole content of the payload is decoded as a JSON object, with no limit on the number of lines the JSON object spans.
- ndjson: indicates that the content of a single item in the input payload is a valid NDJSON format. Multiple single JSON objects formatted on a single line should be separated by a newline delimiter. With this setting each line will be decoded as JSON object, which improves the parsing performance.
- disabled: instructs the forwarder not to attempt any automatic JSON content discovery and instead treat the content as plain text, which improves the parsing performance.
There is no need to configure the JSON content type when Expanding events from JSON object lists, unless you have single JSON objects that span more than 1000 lines.