Ingest Incidents from a SIEM Using QRadar - Tutorials - 6.x - Cortex XSOAR - Cortex - Security Operations

Cortex XSOAR Tutorials 6.x

Product
Cortex XSOAR
Version
6.x
Creation date
2022-09-02
Last date published
2023-04-16
Category
Tutorials

The information in this tutorial is intended for Cortex XSOAR (SOC) engineers or architects. The goal of this tutorial is to explain how an XSOAR architect should design the platform and incident life cycle so that SOC analysts can easily identify and digest critical incident data and to quickly and confidently make decisions.

The tutorial walks you step-by-step through designing the incident life cycle, starting with configuring your XSOAR instance through ingesting, processing, and creating the right configuration for investigating, and closing security events. The tutorial uses QRadar as the SIEM, but the general flows are applicable to any SIEM and other data sources.

By the end of this tutorial, you will have configured your integration and set up a basic flow, as well as started to ingest incidents from the SIEM to Cortex XSOAR.

As you progress through configuring Cortex XSOAR, always have the analyst journey in mind. Your end goal is to make the SOC efficient, which means creating flows that require the analyst to spend the least amount of time on each incident, and ensuring they have all the tools and data to make accurate and confident decisions.

By the time an analyst picks up an incident for investigation, a series of actions will already have been taken on the incident, most of which you as the architect will design. Designing flows for an analyst in Cortex XSOAR is a balance between gathering and processing as much relevant data as possible while only displaying the data and suggesting actions that analysts require for resolving incidents.

Your goal should be to automate as much of the process as possible and leave the analyst to make accurate and confident decisions when needed.

This tutorial includes the following sections:

Analyst Flow

Depending on how you design the incident life cycle, a lot of actions will be performed automatically before the analyst even sees an incident. In fact, incidents might be closed as duplicates, closed as false-positives, data will be enriched, and many more.

For most incidents, the analyst will:

  1. Pick up an open incident.

  2. View enough data to be able to make decisions including what steps have already been taken (Incident Summary, Work Plan).

  3. Easily collect additional data by running commands on third-party tools.

  4. Collaborate with SOC team members in the War Room; ask questions, post comments, and more.

  5. Make an informed decision on how to handle the incident. A typical flow is to design the incident page to include a set of buttons that the analyst can select from (Escalate, Close, etc.)

  6. Activate one of several possible responses either within XSOAR or externally (e.g., open a ticket for IT).

  7. Close the incident and specify a close reason.

Architect Flow

To explain the architect flow we will use QRadar to define and design the incident life cycle for the analyst. The process will be similar for any security event source (e.g., Splunk).

Define the Incident Type

The first step in planning your incident identification and response posture is to create a basic outline of the flows that your SOC will handle. For example, grab a piece of scrap paper and jot down the main steps of an incident starting with the first alert all the way through to the response and closure.

Most SOCs handle several types of incidents generated from the SIEM that originate from SIEM alerts. For example:

  • Alert on a high number of unsuccessful login attempts by a user (authentication incident)

  • Alert on a suspicious file triggering endpoint security alerts (Malware incident)

  • Alert on a user that is accessing data from different locations at the same time (Impossible Travel/Access incident)

Each of these incident types will require you to create targeted workflows and automated responses that generate accurate data sets from which you can design incident type-specific layouts. Your goal is to enable analysts to quickly digest event information.

The incident type determines how the incident progresses through and is presented in XSOAR:

  • Playbook: which playbook will run for this incident type.

  • Automation level: Is it possible to automate the entire process, in which case an analyst will never interact with the incident. You can also automate the majority of the incident response but require an analyst to review it before taking a final decision. All of this is determined by the playbook.

  • Default playbook: it is also recommended to create default playbook for all incident types that do not have a specific playbooks associated (in time you will add more playbooks and the default playbook will be triggered less)

  • Incident/Indicator fields: the incident and indicator fields will vary depending on the incident type as well. For example, a file hash field will be relevant for Malware incidents but not for an access incident. Cortex XSOAR comes with several pre-built playbooks and incident types. Each incident type has out-of-the-box fields that comprise the incident and indicator layouts. You can customize these to add required fields and remove fields that you think will not be necessary. When selecting fields to include in a layout it’s important to remember that the analyst will need to view, digest, and act upon the information you present. It’s a good practice to include everything that will accomplish this goal, but not more than is required.

  • Incident assignment: Once the playbooks defines that an analyst intervention is required, you can also decide which analyst will be assigned (e.g. do you want your analysts to pick up the first incident they see, do you want to auto-assign to the most appropriate analyst that XSOAR suggests, etc.

Plan the workflow for each incident type

This section describes the aspects of the workflow that take place before the analyst is assigned.

  • Assign incidents to the flow

    In Cortex XSOAR, incident classification is the process by which an incoming security event/alert is assigned to an XSOAR incident type. When you fetch security events from a third-party service, in our example, QRadar, the event data is ingested in JSON format. You classify the security event based on the fields (keys) from the event JSON.

    In QRadar, the categories key contains the offense (event) type in QRadar. The values from this key can be used to classify the incidents in XSOAR, although you can use any field (key). Most integrations that fetch security incidents include out-of-the-box (default) event classification for the most common security events. The default classifier for QRadar assigns the following values for the categories key as Access incidents in XSOAR.

    • SIM User Authentication

    • User Login Failure

    In addition to these default classifiers, we can add additional categories (security events) to an incident type in Cortex XSOAR.

    For more detailed information, learn about classifying incidents in Cortex XSOAR.Classification and Mapping

    tutorial-siem-incident-classifier.png
  • Pre-process incoming incidents

    In many cases, you’ll want to eliminate false-positives and deduplicate incidents. This enables you to maintain a clean set of incidents for analysts to work on.

    Pre-processing rules enable you to perform certain actions on security events before they are ingested as incidents in XSOAR. You perform actions when the condition of a rule is met, for example, link the incoming incident to an existing incident, or under configured conditions, drop the incoming incident altogether. In addition, you can run a script on the incident when the condition(s) are met.

    For practical application, see the pre-processing tutorial for phishing incidents.Create Pre-Process Rules for Incidents

  • Run a playbook on the incident

    After the security event is ingested, classified to an XSOAR incident type, and filtered through the pre-processing rules, the default playbook for the incident type is run on the incident. In most cases, the default playbook will extract one or a combination of the following:

    • Extract and enrich indicators

    • Determine the incident severity

    • Assign the incident to an analyst for manual review

    Depending on how you design the workflow for the incident type, when the playbook completes its run, will be the first time that the incident data is presented to analysts in the layouts and dashboards.

    1. Plan the incident process per incident type (full-automated, manual, or hybrid).

    2. Define how incidents will be classified (assigned to an incident type) in XSOAR.Classification and Mapping

    3. Pre-processing: apply automations to incidents before they are ingested to eliminate false-positives and duplicate incidents. This keeps a clean set of active incidents for analysts to concentrate on.Create Pre-Process Rules for IncidentsIncident De-Duplication

    4. Steps an incident goes through.

    5. Analyst intervention in the incident (communication tasks, ad-hoc tasks, and manual tasks).

    6. How data is displayed in Cortex XSOAR.

    7. What happens when an investigation is remediated, including post-processing.Post Processing for Incidents

Incident Lifecycle

Using Cortex XSOAR, you can define integrations with your security products, your security services and any other component that make up your security operations environments. You can then ingest events from these integrations and transform them into incidents in Cortex XSOAR. After the incidents are created, you can run playbooks on these incidents to enrich them with information from other products and services, which helps you complete the picture. In most cases, you can use automations to determine if an incident requires further investigation or can be closed based on the findings. This enables your analysts to focus on the minority of incidents that require further investigation.

  • Analyze SOC’s current workflows and categorize the incident types you expect to handle.

  • Plan what the process should be per each type of incident (fully automated or manual or hybrid).

  • Then to put your plan in action, define the following per each incident type:

    • What incidents are going to be included in this flow (called classification in XSOAR).

    • In many cases you might want to eliminate false positives and duplicate incidents early (preprocessing).

    • What steps each incident should go through (playbook).

    • When analyst intervention required if at all (also in the playbook, and dashboards).

    • How the incident data is going to be presented within XSOAR (mapping).

    • How the incident details would be presented to the analyst (layouts, dashboards…).

    • What happens when the incident is over (e.g. close incident and force or encourage analyst to specify close reason).

1. Prerequisites

This tutorial assumes that you already have the following components installed and configured in your Cortex XSOAR instance. It’s important to have this done because the workflows in this guide require their functionality at some point. For example, a playbook task might send an email notification to users, therefore, if you don’t have an outgoing email integration configured, you won’t be able to complete the playbook workflow.

  1. Install the Cortex XSOAR server and verify that it’s up and running.

  2. Configure an incoming email integration.

    For example, Gmail or Microsoft Graph Mail.

  3. Configure an outgoing email integration.

    For example, Mail Sender.

  4. Provision XSOAR users.

    For example, Active Directory or Okta.

  5. Make sure you have your IBM QRadar credentials.

2. Set up your IBM QRadar integration instance

One of the most common use cases in Cortex XSOAR is integrating with a SIEM, such as QRadar or Splunk, to ingest events as XSOAR incidents.

  1. Install the IBM QRadar pack.

    1. Go to the Cortex XSOAR Marketplace.

    2. Search for IBM QRadar v3.

    3. Click the pack and click Install.

    4. Click Install again to confirm the installation.

  2. Configure the IBM QRadar instance.

    Before you configure the instance:

    • Make sure you have your IBM QRadar API key or authentication details

    • Define your email sender integration and the SIEM admin email address.

    1. Go to SettingsInstances.

    2. Search for IBM QRadar v2.

    3. Click Add Instance.

    4. Under Classifier, select N/A.

    5. Under Incident Type, select QRadar Generic

      At this point we are using the generic QRadar incident type. As you become more familiar with Cortex XSOAR, you can create custom incident types as needed.

    6. Under Mapper Incoming, select QRadar - Generic Incoming Mapper.

      tutorial-siem-mapper.png
    7. Enter the URL of your QRadar server, as well as the username and password, or API key.

    8. Select the Long running instance checkbox.

      This ensures the connection with the QRadar server is always kept open so you have the most up-to-date information.

    9. In the Query to fetch offenses section, define which offenses to fetch.

      Before defining your query to fetch incidents, Cortex XSOAR recommends that you check your QRadar offenses and fetch the most recent 5 offenses using the ID field. For example, id=483 where 483 is the fifth most recent ID number in the QRadar offenses.

      tutorial-siem-qradar-config.png
    10. Select the Incident IP Enrichment and Incident Asset Enrichment checkboxes.

      This enrichment provides additional information about IP addresses and assets that are related to the offenses you’re ingesting.

    11. Add or remove fields from the Events fields to return... box.

      This box lists all of the fields from this offense that Cortex XSOAR ingests out-of-the-box. This is a list of fields that exist in IBM QRadar.

    12. Under the Max number of events per incident field, enter the number of events for this offense that you want to ingest.

      Out-of-the-box, we ingest 20 events. When we investigate these incidents, you will see that these are 20 events out of a much larger number. However, ingesting all of the events for each offense would have adverse consequences on your system performance.

      tutorial-siem-max-events.png
  3. Go to the Incidents page and verify that incidents are being ingested.

3. Run a playbook

After you define the pack and start ingesting incidents, the default playbook assigned to the QRadar incident type automatically processes the incoming incidents. No additional steps are required from the analyst to initiate the playbook run/incident-handling process.

By default, the QRadar integration uses the QRadar Generic playbook, which is executed for the QRadar Generic incident type. It performs all the common parts of the investigation, including:

  • Notifying the SOC

  • Enriching data for indicators and users

  • Calculating severity

  • Excluding irrelevant indicators

  • Assigning incidents

  • Tagging malicious indicators

  • Notifying the SIEM admin about false positives and adjusting indicators and rules

  • Generating a report with the playbook findings

Playbook Inputs

Every playbook has inputs that help determine how the playbook will flow. The inputs (and outputs) are configured at the very beginning of the playbook, under Playbook Triggered.

For a full description of each of the inputs for this playbook, refer to the playbook documentation. However, for purposes of this tutorial, we will focus on the following inputs:

  • Enrich - this input is used to determine whether you want the playbook to enrich all of the indicators in the incident. By default, it’s set to true. However, enrichment can be a very resource intensive operation, in which case, you might want to change this setting to false and enrich only specific indicators and only using certain integrations.

    Also, if you do want to enrich indicators, you should ensure that you have enabled at least some of the out-of-the-box enrichment integrations, such as AutoFocus, VirusTotal, etc.

  • RunAdditionalSearch - determines whether to fetch more QRadar logs associated with an offense, in addition to the default 20 logs provided by the integration. To fetch more than the default number, set this input to true and in the QRadar - Get Offense Logs subplaybook, define how many and what kind of logs to fetch.

  • UseCustomSeveritySettings - determines whether the severity mapping that's provided in the mapper is used or not. By default, this input is set to true, meaning that QRadar events are mapped as follows:

    1,1,1,2,2,2,2,3,3,3

    Magnitude 1-3 in QRadar will be given a Low severity in Cortex XSOAR.

    Magnitude 4-7 in QRadar will be given a Medium severity in Cortex XSOAR.

  • Magnitude 8-10 in QRadar will be given a High severity in Cortex XSOAR.

  • UseCalculateSeverity - determines if the severity of the incident is set based on the Calculate Severity playbook, or if you want to set the severity based on a different attribute, by default, incident magnitude. By default, this input is set to true. However, if you set this to false, you should configure the field by which you do want to determine severity in the FieldToSetSeverityFrom input, (by default, magnitude), as well as configure how magnitude values are translated in Cortex XSOAR using the ScaleToSetSeverityFrom input.

  • FieldToSetSeverityFrom - as mentioned above, if you are not calculating the severity based on the Calculate Severity playbook, you must determine the severity by one of the fields from the QRadar incident. QRadar has several fields that can be used to determine severity, and this can be configured in Cortex XSOAR by changing the value in the Get field for this input. As you can see in the screenshot below, it is set by default to Magnitude. However, you can change it to Severity, Credibility or any other value from the incident.

    tutorial-siem-field-severity.png
  • ScaleToSetSeverityFrom - QRadar uses a scale of 1-10 to determine the severity of an incident. However, in Cortex XSOAR there are only 5 values with a scale of 0-4:

    • Informational

    • Low

    • Medium

    • High

    • Critical

    By default, the QRadar values are mapped to Cortex XSOAR values in the QRadar Generic mapping provided out-of-the-box. These values are mapped as follows:

    1,1,1,2,2,2,2,3,3,3

    Magnitude 1-3 in QRadar will be given a Low severity in Cortex XSOAR.

    Magnitude 4-7 in QRadar will be given a Medium severity in Cortex XSOAR.

    Magnitude 8-10 in QRadar will be given a High severity in Cortex XSOAR.

    To change this mapping, you must enter a value from 0-4 that corresponds to the slot representing the value from QRadar. The following is an example of an alternate mapping:

    This would be translated as follows:

    • 1-2 - Informational

    • 3-4 - Low

    • 5-6 - Medium

    • 7-8 - High

    • 9-10 - Critical

Enrich Indicators

If you selected in the playbook inputs to enrich all of the indicators from the incident, the playbook extracts the indicators and uses whichever integrations you have enabled for the enrichment. Enriched indicators will provide the analyst with more information about each indicator.

The Entity Enrichment playbook checks for information about all kinds of indicators, including malicious URLs, domains, or IP addresses. It can also check against a list of VIP assets in your organization that might have been targeted, or detonate a file that was attached to the incident.

While these capabilities are available in the playbook, they are only available if you have enabled or installed integrations for these checks. For information about enabling additional integrations, refer to the integration documentation.

If you also configured the playbook to use the Calculate Severity playbook, the playbook takes the information from the enriched indicators and provides a severity based on the DBot score.

Manual Investigation

After the playbook gathers all of the necessary information, an SLA timer is triggered and the incident is assigned to an analyst for manual investigation. This includes optimizing the analysis by eliminating indicators and false positives.

Exclude Indicators

The analyst can optionally email the SIEM admin to select indicators from the list generated by the offense and QRadar logs to add to the XSOAR exclude list. The excluded indicators won't be created in XSOAR anymore, reducing the number of indicators the analyst needs to investigate and improving performance.

True/False Positive

The analyst also needs to determine if an incident is a true positive or false positive.

If the incident is a true positive, the analyst can tag malicious indicators to fine-tune the system for future cases, and then generate an incident report and take the necessary remediation steps.

If the incident is a false positive, the analyst can email the SIEM admin a list of indicators to exclude at the QRadar integration level and rules to label in order to fine-tune the system for future cases.

tutorial-siem-which-rules.png

The feedback from the analyst is sent to the SIEM admin so they can review the suggested changes and make any necessary adjustments.

4. Analyze Incident Data

After events are fetched and the playbook executes, the events appear as incidents in the Incidents page. All of the information that is displayed in the respective tabs is determined by the layout you designed for the QRadar Generic incident type.

The layout appearance, including which tabs, their order, names, and more is customizable.Customize Incident Layouts

The analyst can click on the ID of any incident to view additional incident data that you configured for the incident type.

Incident Info View

The Incident Info tab provides information about the incident itself. For example, when was the incident opened and updated, what’s the severity of the incident and from where did it originate. It gives you easy access to all of the indicators extracted from the incident, what is the current state of the incident, and more.

tutorial-siem-incident-info.png

For example, in the screenshot above, we see, in the Work Plan section of the page, that the incident is currently on hold waiting for the manual investigation to complete.

QRadar Offense View

The QRadar Offense tab bring the information that is available in QRadar and provides it to you in one screen within Cortex XSOAR. So, for example, you can see how many of the events associated with the offense were pulled (by default, it’s 20) and the magnitude that was assigned in QRadar.

tutorial-siem-offense-summary.png

The view includes all of the network data that is available from QRadar, as well as a link to the incident in QRadar itself.

QRadar Events View

The QRadar Events tab presents a table with the events that were pulled when the offense was ingested. By default, to improve performance, only the last 20 events are pulled, but you can override this in the Max number of events per incident field in the integration settings.

QRadar Assets

The QRadar Assets tab presents information about any critical assets that were involved in the offense, whether they were targeted or tangentially related.