Ingest Logs from Elasticsearch Filebeat

Cortex XDR can ingest logs from Elasticsearch Filebeat, a file system logger that logs file activity on your endpoints and servers.
Ingesting logs and data requires a Cortex XDR Pro per TB license.
If you want to ingest logs about file activity on your endpoints and servers and do not use the Cortex XDR agent, you can install Elasticsearch* Filebeat as a system logger and then forward those logs to Cortex XDR. To facilitate log ingestion, Cortex XDR supports the same protocols that Filebeat and Elasticsearch use to communicate.
To provide additional context during investigations, Cortex XDR automatically creates a new XQL dataset from your Filebeat logs. You can then use the XQL dataset to search across the logs Cortex XDR received from Filebeat.
To receive logs, you configure collection settings for Filebeat in Cortex XDR and output settings in your Filebeat installations. As soon as Cortex XDR begins receiving logs, the data is visible in XQL Search queries.
  1. In Cortex XDR, set up Log Collection.
    1. Select
      Settings ( )
      Configurations
      Custom Collections
      .
    2. In the Filebeat configuration, click the
      here
      link.
    3. Enter a descriptive
      Name
      for your Filebeat log collection configuration.
    4. Enter the
      Vendor
      and
      Product
      for the type of logs you are ingesting.
      The vendor and product are used to define the name of your XQL dataset (
      <vendor>
      _
      <product>
      _raw
      ). If you do not define a vendor or product, Cortex XDR examines the log header to identify the type and uses that to define the vendor and product in the dataset. For example, if the type is Acme and you opt to let Cortex XDR determine the values, the dataset name would be
      acme_acme_raw
      .
    5. Save & Generate Token
      .
      Click the copy icon next to the key and record it somewhere safe. You will need to provide this key when you set up output settings on your Filebeat instance. If you forget to record the key and close the window you will need to generate a new key and repeat this process.
  2. Set up Filebeat to forward logs.
    After installing the Filebeat agent, configure an Elasticsearch output:
    1. Under the
      output.elasticsearch
      section, configure the following entities:
      • hosts
        —Copy the API URL from your Filebeat configuration and paste it in this field.
      • compression_level
        —5 (recommended)
      • bulk_max_size
        —1000 (recommended)
      • api_key
        —Paste the key you created in when you configured Filebeat Log Collection in Cortex XDR.
      • proxy_url
        —(
        Optional
        )
        <server_ip>
        :
        <port_number>
        . You can specify your own
        <server_ip>
        or use the broker VM to proxy Filebeat communication using the format
        <broker_VM_ip>
        :
        <port_number>
        . When using the broker VM, ensure that you activate the Local Agent Settings applet with the
        Agent Proxy
        enabled.
    2. Save the changes to your output file.
    After Cortex XDR begins receiving logs from Filebeat, they will be available in XQL Search queries.
  3. (
    Optional
    ) Monitor your Filebeat integration.
    You can return to the
    Settings ( )
    Configurations
    Custom Collectors
    page to monitor the status of your Filebeat configuration. For each instance, Cortex XDR displays the number of logs received in the last hour, day, and week. You can also use the Data Ingestion Dashboard to view general statistics about your data ingestion configurations.
  4. (
    Optional
    ) Set up alert notifications to monitor the following events:
    • A Filebeat agent status changes to disconnected.
    • A Filebeat module has stopped sending logs.
Elasticsearch is a trademark of Elasticsearch B.V., registered in the U.S. and in other countries.

Recommended For You