Set up an HTTP Log Collector to Receive Logs

You can set up Cortex XDR to receive logs from third-party sources, and automatically parse and process these logs.
Ingesting logs and data requires a
Cortex
XDR
Pro per TB license.
In addition to logs from supported vendors, you can set up a custom HTTP log collector to receive logs in Raw, JSON, CEF, or LEEF format.
After
Cortex
XDR
begins receiving logs from the third-party source,
Cortex
XDR
automatically parses the logs and creates a dataset with the name
<vendor>
_
<product>
_raw
. You can then use XQL Search queries to view logs and create new Correlation rules.
To set up an HTTP log collector to receive logs from an external source.
  1. Create an HTTP Log collector in
    Cortex
    XDR
    .
    1. Select
      Settings ( )
      Configurations
      Custom Collections
      .
    2. In the
      HTTP
      configuration, click
      Add Instance
      .
    3. Specify a descriptive
      Name
      for your HTTP log collection configuration.
    4. Select the data object
      Compression
      , either gzip or uncompressed.
    5. Select the
      Log Format
      as
      Raw
      ,
      JSON
      ,
      CEF
      , or
      LEEF
      .
      Cortex
      XDR
      supports logs in single line format or multiline format. For a
      JSON
      format, multiline logs are collected automatically when the
      Log Format
      is configured as
      JSON
      . When configuring a
      Raw
      format, you must also define the
      Multiline Parsing Regex
      as explained below.
      -The
      Vendor
      and
      Product
      defaults to
      Auto-Detect
      when the
      Log Format
      is set to
      CEF
      or
      LEEF
      .
      -For a
      Log Format
      set to
      CEF
      or
      LEEF
      ,
      Cortex
      XDR
      reads events row by row to look for the
      Vendor
      and
      Product
      configured in the logs. When the values are populated in the event log row,
      Cortex
      XDR
      uses these values even if you specified a value in the
      Vendor
      and
      Product
      fields in the HTTP collector settings. Yet, when the values are blank in the event log row,
      Cortex
      XDR
      uses the
      Vendor
      and
      Product
      that you specified in the HTTP collector settings. If you did not specify a
      Vendor
      or
      Product
      in the HTTP collector settings, and the values are blank in the event log row, the values for both fields are set to
      unknown
      .
    6. Specify the
      Vendor
      and
      Product
      for the type of logs you are ingesting.
      The vendor and product are used to define the name of your XQL dataset (
      <vendor>
      _
      <product>
      _raw
      ). If you do not define a vendor or product,
      Cortex
      XDR
      examines the log header to identify the type and uses that to define the vendor and product in the dataset. For example, if the type is Acme and you opt to let
      Cortex
      XDR
      determine the values, the dataset name would be
      acme_acme_raw
      .
    7. (
      Optional
      ) Specify the
      Multiline Parsing Regex
      for logs with multilines.
      This option is only displayed when the
      Log Format
      is set to
      Raw
      , so you can set the regular expression that identifies when the multiline event starts in logs with multilines. It is assumed that when a new event begins, the previous one has ended.
    8. Save & Generate Token
      .
      Click the copy icon next to the key and record it somewhere safe. You will need to provide this key when you configure your HTTP POST request. If you forget to record the key and close the window you will need to generate a new key and repeat this process.
      Click
      Done
      when finished.
  2. Send data to your
    Cortex
    XDR
    HTTP log collector.
    1. Send an HTTP POST request to the URL for your HTTP Log Collector.
      For a sample curl or python request, click
      View Example
      .
    2. Substitute the values specific to your configuration.
      • url
        —You can copy the URL for your HTTP log collector from the
        Custom Collectors
        page. For example:
        https://api-{tenant external URL}/logs/v1/event
        .
      • api_key
        —API key you previously recorded for your HTTP log collector.
      • Content-Type
        —Depending on the data object format you selected during setup, this will be
        application/json
        for JSON format or
        text/plain
        for Text format.
      • Body
        —The body contains the records you want to send to
        Cortex
        XDR
        . Separate records with a
        \n
        (new line) delimiter. The request body can contain up to 10Mib records although 1 Mib is recommended. In the case of a curl command, the records are contained in the
        -d ‘
        <records>
        parameter.
  3. Monitor your HTTP Log Collection integration.
    You can return to the
    Settings ( )
    Configurations
    Custom Collectors
    page to monitor the status of your HTTP Log Collection configuration. For each instance,
    Cortex
    XDR
    displays the number of logs received in the last hour, day, and week. You can also use the Data Ingestion Dashboard to view general statistics about your data ingestion configurations.
  4. After
    Cortex
    XDR
    begins receiving logs, use the XQL Search to search your logs.

Recommended For You