Monitor resources on private networks
editMonitor resources on private networks
editTo monitor resources on private networks you can either:
- Allow Elastic’s global managed infrastructure to access your private endpoints.
- Use Elastic Agent to create a Private Location.
Private Locations via Elastic Agent require only outbound connections from your network, while allowing Elastic’s global managed infrastructure to access a private endpoint requires inbound access, thus posing an additional risk that users must assess.
Allow access to your private network
editTo give Elastic’s global managed infrastructure access to a private endpoint, use IP address filtering, HTTP authentication, or both.
To grant access via IP, use this list of egress IPs. The addresses and locations on this list may change, so automating updates to filtering rules is recommended. IP filtering alone will allow all users of Elastic’s global managed infrastructure access to your endpoints, if this is a concern consider adding additional protection via user/password authentication via a proxy like nginx.
Monitor via a private agent
editPrivate Locations allow you to run monitors from your own premises. Before running a monitor on a Private Location, you’ll need to:
- Set up Fleet Server and Elastic Agent.
- Connect Fleet to the Elastic Stack and enroll an Elastic Agent in Fleet.
- Add a Private Location in the Synthetics app.
Private Locations running through Elastic Agent must have a direct connection to Elasticsearch. Do not configure any ingest pipelines, or output via Logstash as this will prevent Synthetics from working properly and is not supported.
Set up Fleet Server and Elastic Agent
editStart by setting up Fleet Server and Elastic Agent:
- Set up Fleet Server: If you are using Elastic Cloud, Fleet Server will already be provided and you can skip this step. To learn more, refer to Set up Fleet Server.
- Create an agent policy: For more information on agent policies and creating them, refer to Elastic Agent policy.
A Private Location should be set up against an agent policy that runs on a single Elastic Agent. The Elastic Agent must be enrolled in Fleet (Private Locations cannot be set up using standalone Elastic Agents). Do not run the same agent policy on multiple agents being used for Private Locations, as you may end up with duplicate or missing tests. Private Locations do not currently load balance tests across multiple Elastic Agents. See Scaling Private Locations for information on increasing the capacity within a Private Location.
By default Private Locations are configured to allow two simultaneous browser tests and an unlimited number of lightweight checks. As a result, if more than two browser tests are assigned to a particular Private Location, there may be a delay to run them.
Connect to the Elastic Stack
editAfter setting up Fleet, you’ll connect Fleet to the Elastic Stack and enroll an Elastic Agent in Fleet.
Elastic provides Docker images that you can use to run Fleet and an Elastic Agent more easily.
For monitors running on Private Locations, you must use the elastic-agent-complete
Docker image to create a self-hosted Elastic Agent node. The standard Elastic Cloud or self-hosted
Elastic Agent will not work.
The elastic-agent-complete Docker image is the only way to have all available options that you see in the Elastic Cloud.
Version 9.0.0-beta1 has not yet been released.
Then enroll and run an Elastic Agent. You’ll need an enrollment token and the URL of the Fleet Server. You can use the default enrollment token for your policy or create new policies and enrollment tokens as needed.
For more information on running Elastic Agent with Docker, refer to Run Elastic Agent in a container.
Version 9.0.0-beta1 has not yet been released.
The elastic-agent-complete Docker image requires additional capabilities to operate correctly. Ensure
NET_RAW and SETUID are enabled on the container.
You may need to set other environment variables. Learn how in Elastic Agent environment variables guide.
Add a Private Location
editWhen the Elastic Agent is running you can add a new Private Location in Kibana:
-
Find
Syntheticsin the global search field. - Click Settings.
- Click Private Locations.
- Click Add location.
- Give your new location a unique Location name and select the Agent policy you created above.
- Click Save.
It is not currently possible to use custom CAs for synthetics browser tests in private locations without following a workaround. To learn more about the workaround, refer to the following GitHub issue: elastic/synthetics#717.
Scaling Private Locations
editBy default Private Locations are configured to allow two simultaneous browser tests, and an unlimited number of lightweight checks.
These limits can be set via the environment variables SYNTHETICS_LIMIT_{TYPE}, where {TYPE} is one of BROWSER, HTTP, TCP, and ICMP
for the container running the Elastic Agent docker image.
It is critical to allocate enough memory and CPU capacity to handle configured limits. Start by allocating at least 2 GiB of memory and two cores per browser instance to ensure consistent performance and avoid out-of-memory errors. Then adjust as needed. Resource requirements will vary depending on workload. Much less memory is needed for lightweight monitors. Start by allocating at least 512MiB of memory and two cores for lightweight checks. Then increase allocated memory and CPU based on observed usage patterns.
These limits are for simultaneous tests, not total tests. For example, if 60 browser tests were scheduled to run once per hour and each took 1 minute to run, that would fully occupy one execution slot. However, it is a good practice to set up execution slots with extra capacity. A good starting point would be to over-allocate by a factor of 5. In the previous example that would mean allocating 5 slots.
Next steps
editNow you can add monitors to your Private Location in the Synthetics app or using the Elastic Synthetics library’s push method.