Manage Datasets

Learn how to import, delete, and interact with custom or third-party datasets in Cortex® XDR™.
This feature requires a Cortex® XDR™ Pro per TB license.
Cortex XDR runs every XQL query against a dataset. A dataset is a collection of column:value sets. You can upload datasets as a CSV
, TSV,
or JSON file that contains the data you are interested in querying. If you do not specify a dataset in your query, Cortex XDR runs the query against the default datasets configured, which is by default configured as
. The
dataset contains all of the endpoint and network data that Cortex XDR collects. You can always change the default datasets using the Set as default option.
To query other datasets, you have two options: you can either set the dataset as default, which enables you to query the datasets without specifying them in the query, or you can name a specific dataset at the beginning of your query with the
stage command. You can add to your list of available datasets by uploading a CSV
, TSV,
or JSON file to Cortex XDR.
You cannot upload a file that contains a byte array (that is, binary data).
Cortex XDR Query Language (XQL) supports using different languages for dataset and field names. In addition, when setting up your XQL query, it is important to keep in mind the refresh times for datasets. All Cortex XDR system datasets, which are created out-of-the-box, are continuously ingested in near real-time as the data comes in, except for the following exceptions.
  • endpoints
    —Refreshed every hour.
  • pan_dss_raw
    —Refreshed daily.
  • Forensics datasets—The Forensics data is not configured to be updated by default. When you enable a collection in the
    Agent Settings
    profile, the data is collected only once unless you specify an interval. If you specify an interval, the data is collected every
    number of hours with the minimum being 12.
Manage datasets from
Cortex XDR
Settings ( )
Data Management
Dataset Management
. In the
Dataset Management
page you can import, view, and interact with your available datasets.
  • Import a dataset.
    1. Select
      + Lookup
    2. Browse
      to your CSV
      , TSV,
      or JSON file, or drag and drop it into the dialog window.
      You can only upload a TSV file that contains a
      file extension.
      When uploading a CSV, TSV, or JSON file, ensure that the file meets the following requirements:
      • Field names are supported using different languages, numbers (
        ), or underscores (
        ). If you use any other characters, Cortex XDR automatically converts them to underscores (
      • Dataset names are supported using different languages. Numbers (
        ) and underscores (
        ) are supported, but not as the first character of the name.
        You can create dataset names using uppercase characters, but in queries dataset names are always treated as if they are lowercase.
      • Must start with a letter or underscore. Cannot use prefixes
        , or
      • Cannot exceed 128 characters.
      • No duplicate names, white spaces, or carriage returns.
    3. (
      ) Rename the file, where only English alphabetical characters are supported.
    4. Add
      the file as a lookup.
    5. After receiving a notification reporting that the upload succeeded,
      ( ) to view it in your list of datasets.
      If the file has the same name as an existing dataset, Cortex XDR will append an underscore and a number to the name to make it unique.
  • Save query results as a dataset.
    You can use the
    stage command to save query results as a dataset. For details about this command, see the XQL Language Reference.
  • Query against a dataset by selecting it with the
    command when you create an XQL query.
  • Right-click a dataset to view the schema of the dataset, set it as default, delete it, copy it, and show or hide datasets. In addition, for a dataset with a
    set to
    , you can also download the JSON file.
    • View Schema
      to view the schema information for every field found in the dataset result set in the
      tab of XQL Search. Each system field in the schema is written with an underscore (_) before the name of the field in the
      column in the table.
    • Set as default
      to query the dataset without having to specify it in your queries in XQL Search as
      dataset = <name of dataset>
      . Once configured, the
      column entry for this dataset is set to
      . By default, this option is not available when right-clicking the
      dataset as this dataset is the only dataset configured as the
      as it contains all of the endpoint and network data that Cortex XDR collects. Once you
      Set as default
      another dataset, you can always remove it by right-clicking the dataset, and selecting
      Remove from defaults
      . When setting multiple default datasets, your query does not need to mention any of the dataset names, and Cortex XDR queries the default datasets using a
    • Delete
      to remove the dataset from Cortex XDR.
    • Download
      the JSON file for a dataset with a
      set to
      . This option is not available for any other dataset type.
      When you download a Lookup dataset with field names in a foreign language, the downloaded JSON file displays the fields as
      as opposed to returning the fields in the foreign language as expected.
    • Copy text to clipboard
      to copy the name of the dataset to your clipboard.
    • Copy entire row
      to copy each cell in a row, separated by tabs, to your clipboard.
    • Show rows with ‘<dataset_name>’
      to create a filter that displays all datasets with the same name.
    • Hide rows with ‘<dataset_name>’
      to create a filter that hides all datasets with the same name.
  • Filter
    your available datasets to specify the ones you want to see.
    1. Select
      An interface for your filter criteria appears.
    2. Select a field, an operator, and a value to match.
    3. Select
      + AND
      + OR
      to add additional filter expressions.
    4. Save
      ( ) your filter to reuse it later.
      After saving, select the three-dot menu ( ) to view your filter.
  • Customize the table.
    Select the three-dot menu ( ) and
    to change the width of rows and columns. You can also select which columns to display. You can always
    Restore default layout
    to go back to displaying the default column layout.

Recommended For You