End-of-Life (EoL)

Indicator Extraction

Indicator extraction extracts indicators from XSOAR incident fields and enriches them with commands and scripts defined for the indicator type.auto extract, auto-extract
Indicator extraction identifies indicators from different text sources in the system (such as War Room entries, email content, etc), extracts them (usually based on regex) and creates indicators in Cortex XSOAR. After extraction, the indicator can be enriched.
Indicator enrichment takes the extracted indicator and provides detailed information about the indicator (from the open ports to
whois
information). It provides a story about the indicator, based on an enrichment feed such as VirusTotal, IPinfo, etc.
In Cortex XSOAR, the indicator extraction feature extracts indicators from incident fields and enriches them using commands and scripts defined for the indicator type. Provided the indicator extraction is enabled, you can configure the extraction logic according to the incident type and according to the associated field.
You can extract indicators in the following scenarios:
  • When fetching incidents
  • When an incident field is updated
  • In a playbook task
  • Using the command line
As your system matures and you start ingesting more events and have more integrations configured, you need to consider customizing your incident type, including how to extract indicators.
Using indicator extraction can adversely affect system performance. Cortex XSOAR recommends that you define extraction settings for each incident type as needed.
For example, in malware you want to extract all IP addresses, but in phishing you only want to extract IP addresses from specific email headers. For attachments, you may want to disable indicator extraction to reduce external API usage and protect restricted data (the hash) from being sent.

Create Indicator Extraction rules

You create indicator extraction rules:
  • You can extract indicators from incident fields on creation of an incident and when a field changes. For Content Pack installed incident types, the indicator extraction rules are set out of the box. For example, in a phishing incident type, in the Destination IP field, IPv6 and IP indicators are extracted. For the Detection URL field, the URL indicator field is extracted, etc.
    Provided the indicator extractions settings are enabled and depending on the rules set in the incident type, indicator extraction is automatic. For example, in a phishing incident, indicator extraction is set to extract the IP indicator (in the incident type). When the incident field updates, the IP indicator field is extracted automatically. In the War Room, you can check that the IP indicator field has been extracted by typing
    1.1.1.1
    . Cortex XSOAR recognizes the indicator as an IP indicator by matching it to the IP indicator’s regex. It then extracts and enriches the indicator using an integration that uses the IP command (such as, AutoFocus, IPinfo etc).
    To edit Incident types installed from the Marketplace or propagated to a Tenant (
    Multi-tenant
    ), you need to detach them. Once detached, the incident type will not be updated with new content provided by Cortex XSOAR. To receive content updates, reattach the incident type. For more information, see customize incident layouts.
  • Running a command during an investigation.

Indicator Extraction Modes

Indicator extraction supports the following modes:
  • None
    : Indicators are not extracted automatically. Use this option when you do not want to further evaluate the indicators.
  • Inline
    : Indicators are extracted within the context that indicator extraction runs (synchronously). The findings are added to the Context Data. For example, if you define indicator extraction for the phishing incident type as inline:
    • For incident creation, the playbook you defined to run by default will not run until the indicators have been extracted.
    • For an on field change, extraction occurs before the next playbook tasks run. Use this option when you need to have the most robust information available per indicator.
    This configuration may delay playbook execution (incident creation).
    While indicator creation is asynchronous, indicator extraction and enrichment is run synchronously. Data is placed into the incident context and is available via the context for subsequent tasks.
  • Out of band
    : Indicators are extracted in parallel (asynchronously) to other actions. The extracted data will be available within the incident, however, it is not available for immediate use in task inputs or outputs since the information is not available in real time.
    For incident creation, out of band is used in rare cases where you do not need the indicators extracted for the proceeding flow of the playbook. You still want to extract them and save them in the system as indicators, so that they can be reviewed at a later stage for manual review. System performance may be better as the playbook flow does not stop to extract, but if the incident contains indicators that are needed or expected in the proceeding playbook execution flow, inline should be used, as it will not execute the playbook before all indicators are extracted from the incident.
    When using Out of band, the extracted indicators do not appear in the context. If you want the extracted indicators to appear select Inline.
  • Use system default
    : Indicators are extracted according to the following server configurations (if you do not change them, default configurations apply):
    Component
    Key
    Value
    Incident creation
    reputation.calc.algorithm
    : Sets the indicator extraction mode for incident creation. It extracts from all associated fields at the point of incident creation. You can change the value when editing an incident type, which overrides this system configuration for this incident type. If set to 1 (none), automatic indicator extraction is not available when manually adding notes to the War Room.
    Default is
    2
    (inline)
    Type one of the following values:
    -
    1
    : none
    -
    2
    : inline
    -
    3
    : out of band
    Incident field change
    reputation.calc.algorithm.fields.change
    : Sets the indicator extraction mode for incident field change. You can change the value when editing an incident type, which overrides this system configuration for this incident type.
    Default is
    3
    (out of band).
    Tasks
    reputation.calc.algorithm.tasks
    : Applies to the result of the task. You can change the value when editing a task, which overrides the system configuration for this task.
    Default is
    none
    .
    Manual
    reputation.calc.algorithm.manual
    : applies to commands triggered from the CLI. You can change the value when using the auto-extract parameter, which overrides the system configuration for this command.
    Default is
    3
    (out of band)

Upgrade Cortex XSOAR

When upgrading from version 6.0.x and below:
  • By default, all incident types (from a Content Pack) are detached.
  • Indicator extraction is enabled for all incident fields, but if you have already selected an extraction mode, this is not affected. For example, if it is set to “none” there will be no extraction on incident creation. It is recommended to review the incident types, and select the required extraction rules.
  • The Incident field change is set to none.

Troubleshoot Indicator Extraction

If indicators are not extracting, check whether the indicator mode is set to none. Even if you select the relevant incident fields and the indicators to extract, if the mode is set to none, indicators do not extract.
When creating new custom fields, if you select
Extract all indicators from all fields
, all fields are extracted including the custom field. If you select
Extract specific indicators by default
, indicator extraction for the new custom field is set to none.
In a Multi-tenant environment, when installing a Content Pack in the Marketplace, the propagation labels enable the entire Content Pack to propagate to the tenant. For example, when installing the Content Pack (which includes an incident type) in the Marketplace, if the propagation label is set to all, it is propagated to the tenant. Even if you change the propagation label in the incident type, it has already been propagated.
The incident type labels can also propagate the incident type to additional tenants for which the Content Pack was not propagated.
If the incident type is not being updated in the tenant’s account, check whether the incident type is detached. If the tenant detaches the incident type, the changes are not updated from the Main account.

Recommended For You