Ingest Incidents from a SIEM Using QRadar
Step-by-step tutorial for ingesting and handling incidents and events from QRadar.
The information in this tutorial is intended for Cortex XSOAR (SOC) engineers or architects. The goal of this tutorial is to explain how an XSOAR architect should design the platform and incident lifecycle so that SOC analysts can easily identify and digest critical incident data and to quickly and confidently make decisions.
The tutorial walks you step-by-step through designing the incident lifecycle, starting with configuring your XSOAR instance through ingesting, processing, and creating the right configuration for investigating, and closing security events. The tutorial uses QRadar as the SIEM, but the general flows are applicable to any SIEM and other data sources.
By the end of this step-by-step tutorial, you will have configured your integration and set up a basic flow, as well as started to ingest incidents from the SIEM to Cortex XSOAR.
As you progress through configuring Cortex XSOAR, always have the analyst journey in mind. Your end goal is to make the SOC efficient, which means creating flows that require the analyst to spend the least amount of time on each incident, and ensuring they have all the tools and data to make accurate and confident decisions.
By the time an analyst picks up an incident for investigation, a series of actions will already have been taken on the incident, most of which you as the architect will design. Designing flows for an analyst in Cortex XSOAR is a balance between gathering and processing as much relevant data as possible while only displaying the data and suggesting actions that analysts require for resolving incidents.
Your goal should be to automate as much of the process as possible and leave the analyst to make accurate and confident decisions when needed.
This tutorial includes the following sections:
Depending on how you design the incident lifecycle, a lot of actions will be performed automatically before the analyst even sees an incident. In fact, incidents might be closed as duplicates, closed as false-positives, data will be enriched, and many more.
For most incidents, the analyst will:
- Pick up an open incident.
- View enough data to be able to make decisions including what steps have already been taken (Incident Summary, Work Plan).
- Easily collect additional data by running commands on 3rd-party tools.
- Collaborate with SOC team members in the War Room; ask questions, post comments, and more.
- Make an informed decision on how to handle the incident. A typical flow is to design the incident page to include a set of buttons that the analyst can select from (Escalate, Close, etc.)
- Activate one of several possible responses either within XSOAR or externally (e.g., open a ticket for IT).
- Close the incident and specify a close reason.
To explain the architect flow we will use QRadar to define and design the incident lifecycle for the analyst. The process will be similar for any security event source (e.g., Splunk).
Define the Incident Type
The first step in planning your incident identification and response posture is to create a basic outline of the flows that your SOC will handle. For example, grab a piece of scrap paper and jot down the main steps of an incident starting with the first alert all the way through to the response and closure.
Most SOCs handle several types of incidents generated from the SIEM that originate from SIEM alerts. For example:
- Alert on a high number of unsuccessful log-in attempts by a user (authentication incident)
- Alert on a suspicious file triggering endpoint security alerts (Malware incident)
- Alert on a user that is accessing data from different locations at the same time (Impossible Travel/Access incident)
Each of these incident types will require you to create targeted workflows and automated responses that generate accurate data sets from which you can design incident-type specific layouts. Your goal is to enable analysts to quickly digest event information.
The incident type determines how the incident progresses through and is presented in XSOAR:
- Playbook: which playbook will run for this incident type.
- Automation level: Is it possible to automate the entire process, in which case an analyst will never interact with the incident. You can also automate the majority of the incident response but require an analyst to review it before taking a final decision. All of this is determined by the playbook.
- Default playbook: it is also recommended to create default playbook for all incident types that do not have a specific playbooks associated (in time you will add more playbooks and the default playbook will be triggered less)
- Incident/Indicator fields: the incident and indicator fields will vary depending on the incident type as well. For example, a file hash field will be relevant for Malware incidents but not for an access incident. Cortex XSOAR comes with several pre-built playbooks and incident types. Each incident type has out-of-the-box fields that comprise the incident and indicator layouts. You can customize these to add required fields and remove fields that you think will not be necessary. When selecting fields to include in a layout it’s important to remember that the analyst will need to view, digest, and act upon the information you present. It’s a good practice to include everything that will accomplish this goal, but not more than is required.
- Incident assignment: Once the playbooks defines that an analyst intervention is required, you can also decide which analyst will be assigned (e.g. do you want your analysts to pick up the first incident they see, do you want to auto-assign to the most appropriate analyst that XSOAR suggests, etc.
Plan the workflow for each incident type
This section describes the aspects of the workflow that take place before the analyst is assigned.
- Assign incidents to the flowIn Cortex XSOAR, incident classification is the process by which an incoming security event/alert is assigned to an XSOAR incident type. When you fetch security events from a third-party service, in our example, QRadar, the event data is ingested in JSON format. You classify the security event based on the fields (keys) from the event JSON.In QRadar, thecategorieskey contains the offense (event) type in QRadar. The values from this key can be used to classify the incidents in XSOAR, although you can use any field (key). Most integrations that fetch security incidents include out-of-the-box (default) event classification for the most common security events. The default classifier for QRadar assigns the following values for thecategorieskey as Access incidents in XSOAR.
In addition to these default classifiers, we can add additional categories (security events) to an incident type in Cortex XSOAR.For more detailed information, learn about classifying incidents in Cortex XSOAR.
- SIM User Authentication
- User Login Failure
- Pre-process incoming incidentsIn many cases, you’ll want to eliminate false-positives and deduplicate incidents. This enables you to maintain a clean set of incidents for analysts to work on.Pre-processing rules enable you to perform certain actions on security events before they are ingested as incidents in XSOAR. You perform actions when the condition of a rule is met, for example, link the incoming incident to an existing incident, or under configured conditions, drop the incoming incident altogether. In addition, you can run a script on the incident when the condition(s) are met.For practical application, see the pre-processing tutorial for phishing incidents.
- Run a playbook on the incidentAfter the security event is ingested, classified to an XSOAR incident type, and filtered through the pre-processing rules, the default playbook for the incident type is run on the incident. In most cases, the default playbook will extract one or a combination of the following:
Depending on how you design the workflow for the incident type, when the playbook completes its run, will be the first time that the incident data is presented to analysts in the layouts and dashboards.
- Extract and enrich indicators
- Determine the incident severity
- Assign the incident to an analyst for manual review
- Plan the incident process per incident type (full-automated, manual, or hybrid).
- Steps an incident goes through.
- Analyst intervention in the incident (communication tasks, ad-hoc tasks, and manual tasks).
- How data is displayed in Cortex XSOAR.
Using Cortex XSOAR, you can define integrations with your security products, your security services and any other component that make up your security operations environments. You can then ingest events from these integrations and transform them into incidents in Cortex XSOAR. After the incidents are created, you can run playbooks on these incidents to enrich them with information from other products and services, which helps you complete the picture.In most cases, you can use automation to determine if an incident requires further investigation or can be closed based on the findings. This enables your analysts to focus on the minority of incidents that require further investigation.
- Analyze SOC’s current workflows and categorize the incident types you expect to handle.
- Plan what the process should be per each type of incident (fully automated or manual or hybrid).
- Then to put your plan in action, define the following per each incident type:
- What incidents are going to be included in this flow (called classification in XSOAR).
- In many cases you might want to eliminate false positives and duplicate incidents early (preprocessing).
- What steps each incident should go through (playbook).
- When analyst intervention required if at all (also in the playbook, and dashboards).
- How the incident data is going to be presented within XSOAR (mapping).
- How the incident details would be presented to the analyst (layouts, dashboards…).
- What happens when the incident is over (e.g. close incident and force or encourage analyst to specify close reason).
This tutorial assumes that you already have the following components installed and configured in your Cortex XSOAR instance. It’s important to have this done because the workflows in this guide require their functionality at some point. For example, a playbook task might send an email notification to users, therefore, if you don’t have an outgoing email integration configured, you won’t be able to complete the playbook workflow.
- Install the Cortex XSOAR server and verify that it’s up and running.
- Configure an outgoing email integration.
- Make sure you have your IBM QRadar credentials.
2. Set up your IBM QRadar integration instance
One of the most common use cases in Cortex XSOAR is integrating with a SIEM, such as QRadar or Splunk, to ingest events as XSOAR incidents.
- Install the IBM QRadar pack.
- Go to the Cortex XSOAR Marketplace.
- Search for IBM QRadar v3.
- Click on the pack and clickInstall.
- ClickInstallagain to confirm the installation.
- Configure the IBM QRadar instance.Before you configure the instance:
- Make sure you have your IBM QRadar API key or authentication details
- Define your email sender integration and the SIEM admin email address.
- Go to.SettingsIntegrationsServers & Services
- Search for IBM QRadar.
- ClickAdd Instance.
- UnderClassifier, selectN/A.
- UnderIncident Type, selectQRadar GenericAt this point we are using the generic QRadar incident type. As you become more familiar with Cortex XSOAR, you can create custom incident types as needed.
- UnderMapper Incoming, selectQRadar - Generic Incoming Mapper.
- Enter the URL of your QRadar server, as well as the username and password, or API key.
- Select theLong running instancecheckbox.This ensures that the connection with the QRadar server is always kept open so you have the most up-to-date information.
- In theQuery to fetch offensessection, define which offenses to fetch.Before defining your query to fetch incidents, Cortex XSOAR recommends that you check your QRadar offenses and fetch the most recent 5 offenses using the ID field. For example,id=483where 483 is the fifth most recent ID number in the QRadar offenses.
- Select theIncident IP EnrichmentandIncident Asset Enrichmentcheckboxes.This enrichment provides additional information about IP addresses and assets that are related to the offenses you’re ingesting.
- Add or remove fields from theEvents fields to return...box.This box lists all of the fields from this offense that Cortex XSOAR ingests out-of-the-box. This is a list of fields that exist in IBM QRadar.
- Under theMax number of events per incidentfield, enter the number of events for this offense that you want to ingest.Out-of-the-box, we ingest 20 events. When we investigate these incidents, you will see that these are 20 events out of a much larger number. However, ingesting all of the events for each offense would have adverse consequences on your system performance.
- Go to theIncidentspage and verify that incidents are being ingested.
3. Run a playbook
After you define the pack and start ingesting incidents, the default playbook assigned to the QRadar incident type automatically processes the incoming incidents. No additional steps are required from the analyst to initiate the playbook run/incident-handling process.
By default, the QRadar integration uses the
QRadar Genericplaybook, which is executed for the QRadar Generic incident type. It performs all the common parts of the investigation, including:
- Notifying the SOC
- Enriching data for indicators and users
- Calculating severity
- Excluding irrelevant indicators
- Assigning incidents
- Tagging malicious indicators
- Notifying the SIEM admin about false positives and adjusting indicators and rules
- Generating a report with the playbook findings
Every playbook has inputs that help determine how the playbook will flow. The inputs (and outputs) are configured at the very beginning of the playbook, under
For a full description of each of the inputs for this playbook, refer to the playbook documentation. However, for purposes of this tutorial, we will focus on the following inputs:
- Enrich- this input is used to determine whether you want the playbook to enrich all of the indicators in the incident. By default, it’s set to true. However, enrichment can be a very resource intensive operation, in which case, you might want to change this setting to false and enrich only specific indicators and only using certain integrations.Also, if you do want to enrich indicators, you should ensure that you have enabled at least some of the out-of-the-box enrichment integrations, such as AutoFocus, VirusTotal, etc.
- RunAdditionalSearch- determines whether to fetch more QRadar logs associated with an offense, in addition to the default 20 logs provided by the integration. To fetch more than the default number, set this input to true and in the QRadar - Get Offense Logs subplaybook, define how many and what kind of logs to fetch.
- UseCustomSeveritySettings- determines whether the severity mapping that's provided in the mapper is used or not. By default, this input is set to true, meaning that QRadar events are mapped as follows:1,1,1,2,2,2,2,3,3,3Magnitude 1-3 in QRadar will be given a Low severity in Cortex XSOAR.Magnitude 4-7 in QRadar will be given a Medium severity in Cortex XSOAR.
- Magnitude 8-10 in QRadar will be given a High severity in Cortex XSOAR.
- UseCalculateSeverity- this input determines if the severity of the incident is set based on the Calculate Severity playbook, or if you want to set the severity based on a different attribute, by default, incident magnitude. By default, this input is set to true. However, if you set this to false, you should configure the field by which you do want to determine severity in the FieldToSetSeverityFrom input, (by default, magnitude), as well as configure how magnitude values are translated in Cortex XSOAR using the ScaleToSetSeverityFrom input.
- FieldToSetSeverityFrom- as mentioned above, if you are not calculating the severity based on the Calculate Severity playbook, you must determine the severity by one of the fields from the QRadar incident. QRadar has several fields that can be used to determine severity, and this can be configured in Cortex XSOAR by changing the value in the Get field for this input. As you can see in the screenshot below, it is set by default to Magnitude. However, you can change it to Severity, Credibility or any other value from the incident.
- ScaleToSetSeverityFrom- QRadar uses a scale of 1-10 to determine the severity of an incident. However, in Cortex XSOAR there are only 5 values with a scale of 0-4:
By default, the QRadar values are mapped to Cortex XSOAR values in the QRadar Generic mapping provided out-of-the-box. These values are mapped as follows:1,1,1,2,2,2,2,3,3,3Magnitude 1-3 in QRadar will be given a Low severity in Cortex XSOAR.Magnitude 4-7 in QRadar will be given a Medium severity in Cortex XSOAR.Magnitude 8-10 in QRadar will be given a High severity in Cortex XSOAR.To change this mapping, you must enter a value from 0-4 that corresponds to the slot representing the value from QRadar. The following is an example of an alternate mapping:This would be translated as follows:
- 1-2 - Informational
- 3-4 - Low
- 5-6 - Medium
- 7-8 - High
- 9-10 - Critical
If you selected in the playbook inputs to enrich all of the indicators from the incident, the playbook extracts the indicators and uses whichever integrations you have enabled for the enrichment. Enriched indicators will provide the analyst with more information about each indicator.
The Entity Enrichment playbook checks for information about all kinds of indicators, including malicious URLs, domains, or IP addresses. It can also check against a list of VIP assets in your organization that might have been targeted, or detonate a file that was attached to the incident.
While these capabilities are available in the playbook, they are only available if you have enabled or installed integrations for these checks. For information about enabling additional integrations, refer to the integration documentation.
If you also configured the playbook to use the Calculate Severity playbook, the playbook takes the information from the enriched indicators and provides a severity based on the DBot score.
After the playbook gathers all of the necessary information, an SLA timer is triggered and the incident is assigned to an analyst for manual investigation. This includes optimizing the analysis by eliminating indicators and false positives.
The analyst can optionally email the SIEM admin to select indicators from the list generated by the offense and QRadar logs to add to the XSOAR exclude list. The excluded indicators won't be created in XSOAR anymore, reducing the number of indicators the analyst needs to investigate and improving performance.
The analyst also needs to determine if an incident is a true positive or false positive.
If the incident is a true positive, the analyst can tag malicious indicators to fine-tune the system for future cases, and then generate an incident report and take the necessary remediation steps.
If the incident is a false positive, the analyst can email the SIEM admin a list of indicators to exclude at the QRadar integration level and rules to label in order to fine-tune the system for future cases.
The feedback from the analyst is sent to the QRadar admin so they can review the suggested changes and make any necessary adjustments.
4. Analyze Incident Data
After events are fetched and the playbook executes, the events appear as incidents in the Incidents page. All of the information that is displayed in the respective tabs is determined by the layout you designed for the QRadar Generic incident type.
The layout appearance, including which tabs, their order, names, and more is customizable.
The analyst can click on the ID of any incident to view additional incident data that you configured for the incident type.
Incident Info View
The Incident Info tab provides information about the incident itself. For example, when was the incident opened and updated, what’s the severity of the incident and from where did it originate. It gives you easy access to all of the indicators extracted from the incident, what is the current state of the incident, and more.
For example, in the screenshot above, we see, in the Work Plan section of the page, that the incident is currently on hold waiting for the manual investigation to complete.
QRadar Offense View
The QRadar Offense tab bring the information that is available in QRadar and provides it to you in one screen within Cortex XSOAR. So, for example, you can see how many of the events associated with the offense were pulled (by default, it’s 20) and the magnitude that was assigned in QRadar.
The view includes all of the network data that is available from QRadar, as well as a link to the incident in QRadar itself.
QRadar Events View
The QRadar Events tab presents a table with the events that were pulled when the offense was ingested. By default, to improve performance, only the last 20 events are pulled, but you can override this in the
Max number of events per incidentfield in the integration settings.
The QRadar Assets tab presents information about any critical assets that were involved in the offense, whether they were targeted or tangentially related.
Recommended For You
Recommended videos not found.