Elastic Cloud data ingestion

When it comes to delivering your data into Elastic Cloud, a variety of options are available for data ingestion. We’ve collected a series of resources to make it easier. From video to documentation, below you can check out a list of the fastest ways to get started.

Video thumbnail

Data ingestion methods

By now you've probably spun up a deployment and might be wondering what's next. You likely have data that you want to add, known as ingesting, to Elasticsearch, so let's explore some data ingestion methods.

Enterprise SearchSearch for information across websites, applications, workplace content, and more
ObservabilityMonitor and gain insights across your cloud-native and distributed systems
SecurityDetect and respond to threats by unifying SIEM, endpoint security, and cloud security
Logs
Records and events from application servers, your web farm, IoT devices, or triggered actions by users
Application traces
Specialized logging that records information about your program's execution
Metrics
Point-in-time measurements of your application or system for monitoring, alerting, and analyzing trends
Security information and event management data (SIEM)
Security data from cloud, apps, network devices, hosts, servers, and more to power detection and response
Endpoint detection and response data (EDR)
Host activity data to identify threats and accelerate response
Files and catalogs: Pre-built search solutions
Simple-to-use, curated search experience with tunable relevance controls
Files and catalogs: Custom search solution
Customized data store and search experience leveraging Elasticsearch APIs and clients as well as file upload via Kibana
Logs
Records and events from application servers, your web farm, IoT devices, or triggered actions by users
Application traces
Specialized logging that records information about your program's execution
Metrics
Point-in-time measurements of your application or system for monitoring, alerting, and analyzing trends
Security information and event management data (SIEM)
Security data from cloud, apps, network devices, hosts, servers, and more to power detection and response
Endpoint detection and response data (EDR)
Host activity data to identify threats and accelerate response
Files and catalogs: Pre-built search solutions
Simple-to-use, curated search experience with tunable relevance controls
Files and catalogs: Custom search solution
Customized data store and search experience leveraging Elasticsearch APIs and clients as well as file upload via Kibana
Enterprise SearchSearch for information across websites, applications, workplace content, and more
ObservabilityMonitor and gain insights across your cloud-native and distributed systems
SecurityDetect and respond to threats by unifying SIEM, endpoint security, and cloud security

Can't find the integration you’re looking for? March to your own beat with Elastic Beats!

Wondering what the difference between Beats (i.e., Filebeat, Metricbeat, etc.) and Elastic Agent is? Check out our documentation.

Ship. Visualize. Analyze.

Data ingestion in Elastic Cloud is easy. Learn to ship data quickly so you can create robust visualizations for faster analysis.

  • "en": "Elastic Agent", 
"cn": "Elastic 代理"

    Elastic Agent

    Elastic Agent is a single, unified agent that simplifies the data ingestion process for logs, metrics, APM traces, synthetics, and securing your system. With Elastic Agent, you can quickly deploy across your ecosystem and centrally manage your agents through Kibana.

  • Web crawler

    The Elastic web crawler is an easy yet powerful way to ingest publicly available web content so it becomes instantly searchable on your website. The web crawler fetches content, extracts it, and transforms your web pages into search documents that are automatically indexed. Simply provide a URL and let the web crawler do the rest.

  • File upload

    Did you know you can upload files directly into Kibana for quick data exploration? If you have a log file or delimited CSV, TSV, or JSON file, you can upload it, view its fields and metrics, and optionally import it into Elasticsearch.

  • Elastic connectors

    Leverage Elastic connectors to sync data from an original source to an Elasticsearch index. Both native connectors and connector clients are available to use to extract the original files, records, or objects so they can be transformed into Elasticsearch documents and used AI powered search experiences.

  • Language clients: Prebuilt search solutions

    Elasticsearch provides HTTP APIs. Developers use these APIs to integrate Enterprise Search with their applications and services. However, many developers prefer APIs that feel natural to their chosen programming language.

  • "en": "Dashboard and spyglass", 
"cn": "仪表板和放大镜", 
"de": "Dashboard und Lupe", 
"es": "Dashboard y lupa", 
"fr": "Tableau de bord et loupe", 
"jp": "ダッシュボードと望遠鏡", 
"kr": "대시보드와 망원경", 
"pt": "Dashboard e lupa"

    Language clients: Custom search solutions

    Whether you need client application data, ECS (Elastic Common Schema)-formatted log data, server monitoring metrics, or relational database records to synchronize with Elasticsearch, you get a tailored experience with Elastic’s custom search solutions.

Elastic Cloud, your way

Send data to Elastic Cloud with simplified, native integrations on Microsoft Azure, Google Cloud, or AWS.

  • Natively ingest logs stored in Amazon S3 into Elastic with the Elastic serverless forwarder and view them alongside other logs and metrics for centralized analytics. Natively ingest container logs from Amazon ECS with the native integration of AWS FireLens to collect container and storage logs for faster analysis.

  • Automate the ingestion of logs directly within the Microsoft Azure portal with the native integration. Easily monitor your virtual machines when you stream logs as well as seamlessly ingest logs and metrics from Microsoft Azure Spring Cloud to unify visibility across your Spring Boot Applications.

  • Simplify your architecture when you stream events and logs from Google Pub/Sub, Google Cloud Storage (GCS), and BigQuery into Elastic using Google Dataflow. Leverage the Elastic App Search extension for Google Cloud Firestore to add search functionality to mobile and web-based applications.