Ingest Generic Logs from AWS CloudTrail and Amazon CloudWatch

Take advantage of Cortex XDR investigation capabilities and set up generic log ingestion for your AWS CloudTrail and Amazon CloudWatch logs.
Ingesting logs and data requires a Cortex XDR Pro per TB license.
You can forward generic logs for the relative service to Cortex XDR from AWS CloudTrail or Amazon CloudWatch.

Ingest Generic Logs from Amazon Kinesis Firehose

Ingesting logs and data requires a Cortex XDR Pro per TB license.
You can ingest generic logs of the raw data from Amazon Kinesis Firehose. To enable log forwarding, you set up Amazon Kinesis Firehose and then add that to your AWS CloudTrail or Amazon CloudWatch configuration. After you complete the set up process, logs from the respective service are then searchable in Cortex XDR to provide additional information and context to your investigations.
To set up AWS integration, you require certain permissions in AWS. You need a role that enables access to configuring Amazon Kinesis Firehose.
  1. Set up the AWS integration in Cortex XDR.
    1. Select
      Settings ( )
      Configurations
      Data Collection
      Collection Integrations
      .
    2. In the AWS configuration, click the
      here
      link to begin a new configuration.
    3. Enter a descriptive
      Name
      for your log collection configuration.
    4. Enter the
      Vendor
      and
      Product
      for the type of logs you are ingesting.
      The vendor and product are used to define the name of your XQL dataset (
      <vendor>
      _
      <product>
      _raw
      ). If you do not define a vendor or product, Cortex XDR uses the default values of Amazon and AWS with the resulting dataset name as
      amazon_aws_raw
      . To uniquely identify the log source, consider changing the values.
    5. Choose the format of the data input source (CloudTrail or CloudWatch) that you will export to Cortex XDR, either
      JSON
      or
      Text
      .
    6. Save & Generate Token
      .
      Click the copy icon next to the key and record it somewhere safe. You will need to provide this key when you set up output settings in AWS Kinesis Firehose. If you forget to record the key and close the window you will need to generate a new key and repeat this process.
    7. Select
      Done
      to close the window.
  2. Create a Kinesis Data Firehose delivery stream to your chosen destination.
    1. Log in to the AWS Management Console, and open the Kinesis console at https://console.aws.amazon.com/kinesis.
    2. Select
      Data Firehose
      Create delivery stream
      .
    3. Define the name and source for your stream.
      • Delivery stream name
        —Enter a descriptive name for your stream configuration.
      • Source
        —Select
        Direct PUT or other sources
        .
      • Server-side encryption for source records in the delivery stream
        —Ensure this option is disabled.
      Click
      Next
      to proceed to the process record configuration.
    4. Define the process records.
      • Transform source records with AWS Lambda—Set the Data Transformation as
        Disabled
        .
      • Convert record format—Set Record format conversion as
        Disabled
        .
      Click
      Next
      to proceed to the destination configuration.
    5. Choose a destination for the logs.
      Choose
      HTTP Endpoint
      as the destination and configure the HTTP endpoint configuration settings:
      • HTTP endpoint name
        —Enter the name you used to identify your AWS log collection configuration in Cortex XDR.
      • HTTP endpoint URL
        —Copy the API URL associated with your log collection from the Cortex XDR management console (
        Settings ( )
        Configurations
        Data Collection
        Custom Collectors
        Copy API URL
        . The URL will include your tenant name (
        https://api-
        <tenant external URL>
        /logs/v1/aws)
        .
      • Access key
        —Paste in the token key you recorded earlier during the configuration of your Cortex XDR log collection settings.
      • Content encoding
        —Select
        GZIP
        . Disabling content encoding may result in high egress costs.
      • Retry duration
        —Enter
        300
        seconds.
      • S3 bucket—Set the
        S3 backup mode
        as
        Failed data only
        . For the S3 bucket, we recommend that you create a dedicated bucket for Cortex XDR integration.
      Click
      Next
      to proceed to the settings configuration.
    6. Configure additional settings.
      • HTTP endpoint buffer conditions—Set the
        Buffer size
        as
        1
        MiB and the
        Buffer interval
        as
        60
        seconds.
      • S3 buffer conditions—Use the default settings for
        Buffer size
        as
        5
        MiB and
        Buffer interval
        as
        300
        seconds unless you have alternative sizing preferences.
      • S3 compression and encryption—Choose your desired compression and encryption settings.
      • Error logging
        —Select
        Enabled
        .
      • Permissions
        Create or update IAM role
        . option
      Select
      Next
      .
    7. Review your configuration and
      Create delivery stream
      .
      When your delivery stream is ready, the status changes from Creating to Active.
  3. To begin forwarding logs, add the Kinesis Firehose instance to your AWS CloudTrail or Amazon CloudWatch configuration.
    To do this, you add a subscription filter for Amazon Kinesis Firehose. See https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/SubscriptionFilters.html.
  4. Verify the status of the integration.
    Return to the
    Integrations
    page and view the statistics for the log collection configuration.

Recommended For You