Performance Benchmark

Details the Cortex XSOAR hardware specifications and requirements and benchmarking performance tests conducted in Cortex XSOAR labs.
Cortex XSOAR is designed to maximize performance and enable scalability, to provide the best experience and performance. A benchmarking process is conducted annually to ensure the best performance levels.
Cortex XSOAR performance is determined by compute, memory, and HD performance. Each component can impact a different part of the system, therefore it is important to ensure that you deploy Cortex XSOAR on an infrastructure that meets all requirements.
The amount of data each incident holds can have a significant impact on performance and disk space. To achieve optimal performance and disk usage, we recommend that an incident be no larger than 0.5mb.
Testing was performed with Cortex XSOAR v6.2 Build 1271082
Testing was performed for deployments with the Bolt database, with Elasticsearch, and with High Availability (Elasticsearch with multiple app servers). For all three environments, 60 events were ingested per minute from a SIEM.
Benchmark Process
The benchmarking process is executed using an integration that creates phishing events at a rate of 60 events per minute. Incidents are then ingested into Cortex XSOAR through the classification and mapping process, which creates phishing incidents in the system. Each incident automatically triggers the
Phishing Investigation - Generic v2
.
The
Phishing Investigation - Generic v2
playbook performs the following actions:
  • Parse and process the email
  • Auto-run IOC extraction and reputation checks for all indicators
  • Extract attachments
  • Calculate incident severity based on IOCs
  • Notify users (administrators and the email sender) about the progress of the incident
  • Close the incident

Specifications

Database
Type
Release
CPU Cores
RAM
Disk
Bolt DB
c5.4xlarge
4.14.231-173.418.amzn2.x86_64
16
30.41 GB
4.88 TB Maximum IOPS (16 KiB I/O) 20,000
Elasticsearch
c5.4xlarge
4.14.231-173.361.amzn2.x86_64
16
30.63 GB
0.98 TB Maximum IOPS (16 KiB I/O) 20,000
High Availability Elasticsearch - 4 app servers with identical specifications
c5.24xlarge
4.14.219-164.354.amzn2.x86_64
96
184.72 GB
0.49 TB Maximum IOPS (16 KiB I/O) 80,000NFS shared file system
Elasticsearch Cluster Architecture
The following is the standalone Elasticsearch cluster architecture used for performance benchmark testing:
Node
Node ES Roles
Node CPU
Node Memory
Node JVM
Node AWS Instance Type
Node Disk Size and Type
Node ES Version
Master 1
master-eligible, remote cluster client node
4 vCPUs
8.0 GiB
4 GB
c5.xlarge
1000 GiB gp2
7.9.0
Master 2
master-eligible, remote cluster client node
4 vCPUs
8.0 GiB
4 GB
c5.xlarge
1000 GiB gp2
7.9.0
Master 3
master-eligible, remote cluster client node
4 vCPUs
8.0 GiB
4 GB
c5.xlarge
1000 GiB gp2
7.9.0
Data 1
data, ingest, remote cluster client, transform
8 vCPUs
64.0 GiB
30 GB
r5.2xlarge
4000 GiB io1
7.9.0
Data 2
data, ingest, remote cluster client, transform
8 vCPUs
64.0 GiB
30 GB
r5.2xlarge
4000 GiB io1
7.9.0
Data 3
data, ingest, remote cluster client, transform
8 vCPUs
64.0 GiB
30 GB
r5.2xlarge
4000 GiB io1
7.9.0
Client 1
remote cluster client node
8 vCPUs
16.0 GiB
8 GB
c5.2xlarge
1000 GiB gp2
7.9.0
Client 2
remote cluster client node
8 vCPUs
16.0 GiB
8 GB
c5.2xlarge
1000 GiB gp2
7.9.0
High Availability Elasticsearch Cluster Architecture
The following is the high availability Elasticsearch cluster architecture used for performance benchmark testing:
Node
Node ES Roles
Node CPU
Node Memory
Node JVM
Node AWS Instance Type
Node Disk Size and Type
Node ES Version
Master 1
master-eligible, remote cluster client node
4 vCPUs
8.0 GiB
4 GB
c5.xlarge
1000 GiB gp2
7.11.0
Master 2
master-eligible, remote cluster client node
4 vCPUs
8.0 GiB
4 GB
c5.xlarge
1000 GiB gp2
7.9.0
Master 3
master-eligible, remote cluster client node
4 vCPUs
8.0 GiB
4 GB
c5.xlarge
1000 GiB gp2
7.9.0
Data 1
data, ingest, remote cluster client, transform
8 vCPUs
64.0 GiB
30 GB
r5.2xlarge
4000 GiB io1
7.9.0
Data 2
data, ingest, remote cluster client, transform
8 vCPUs
64.0 GiB
30 GB
r5.2xlarge
4000 GiB io1
7.9.0
Data 3
data, ingest, remote cluster client, transform
8 vCPUs
64.0 GiB
30 GB
r5.2xlarge
4000 GiB io1
7.9.0
Client 1
remote cluster client node
8 vCPUs
16.0 GiB
8 GB
c5.2xlarge
1000 GiB gp2
7.9.0
Client 2
remote cluster client node
8 vCPUs
16.0 GiB
8 GB
c5.2xlarge
1000 GiB gp2
7.9.0

Utilization

Average over 7 days.
Database
CPU
RAM
RAM XSOAR
RAM Docker
RAM Python
HDD
Bolt DB
45%
22.77 GB
17.47 GB
2.41 GB
2.01 GB
4.20 TB
Elasticsearch
60%
18.60 GB
3.14 GB
6.42 GB
4.32 GB
0.10 TB
High Availability - APP Server 1
9%
146.19 GB
2.16 GB
2.03 GB
1.54 GB
0.02 TB
High Availability - APP Server 2
8%
144.25 GB
1.77 GB
1.12 GB
1.00 GB
0.02 TB
High Availability - APP Server 3
8%
155.43 GB
1.72 GB
1.11 GB
0.99 GB
0.02 TB
High Availability - APP Server 4
7%
132.45 GB
1.26 GB
2.07 GB
1.56 GB
0.02 TB

Workers

Database
Total Workers
Busy (Average over 7 day period)
Bolt DB
600
300
Elasticsearch
1,000
167
High Availability Elasticsearch
1,000
65

Incidents

Database
Last 7 days closed incidents
Incidents per-hour rate
Bolt DB
217,020
1,240
Elasticsearch
531,743
3,120
High Availability Elasticsearch
862,743
5,039

Phishing Use Case

Based on the `Phishing Investigation - Generic v2` playbook. Average over 7 days.
Database
Fetch duration - how long it took for integration to fetch data - time
Ingestion duration - classification and mapping
Playbook duration - how long it took to run playbook on incident
Bolt DB
373ms
2m 12s 677ms
3m 33s 703ms
Elasticsearch
288ms
13s 923ms
44s 188ms
High Availability Elasticsearch
444ms
1m 6s 289ms
54s 515ms

Searches

Average over all time.
Database
All time active
All time pending
All time closed
Bolt DB
953ms
903ms
2s 183ms
Elasticsearch
408ms
400ms
975ms
High Availability Elasticsearch
312ms
212ms
896ms
Average over 7 days.
Database
Last 7 days active
Last 7 days pending
Last 7 days closed
Bolt DB
546ms
522ms
1s 198ms
Elasticsearch
417ms
393ms
488ms
High Availability Elasticsearch
262ms
206ms
569ms

Bolt Database Details

The following server configurations were used for the BoltDB testing environment:
Key
Value
content.unlock.scripts
CommonServerPython
create.related.indicators.entry
True
custom.transformer.override.convertkeystotablefieldformat
True
execution.demisto rest api.demisto-api-post
False
feedintegrationscript.timeout
60
investigation.task.partial.index
15
job.monitor.log
True
monitoring.pprof
True
playbook.willnotexecute.old.eval
False
relationships.enabled
False
tim.features.enabled
True
workers.count.tasks
600
Bolt DB - Web Client - JS Heap
Page
Total Size
Used Size
Automation
92.64 MB
73.13 MB
Incidents
104.46 MB
76.91 MB
Indicators
107.49 MB
69.17 MB
Integrations
93.08 MB
74.68 MB
Jobs
95.09 MB
75.37 MB
Login_page_load
52.58 MB
36.36 MB
Playbooks
104.05 MB
71.46 MB
Reports
67.78 MB
56.64 MB
Bolt DB - Visual
Page
First Visual Change
Fully Loaded
Largest Image
Last Visual Change
Speed Index
Time to First Byte
Automation
553ms
547ms
486ms
1s 112ms
774ms
20ms
Incidents
795ms
3s 990ms
824ms
8s 123ms
1s 200ms
20ms
Indicators
1s 39ms
145ms
1s 328ms
1s 466ms
1s 309ms
20ms
Integrations
978ms
636ms
1s 319ms
1s 462ms
1s 275ms
20ms
Jobs
477ms
22ms
484ms
922ms
493ms
20ms
Login_page_load
4s 359ms
7s 413ms
4s 361ms
4s 359ms
20ms
Playbooks
759ms
730ms
1s 56ms
3s 727ms
1s 44ms
20ms
Reports
535ms
636ms
539ms
1s 71ms
571ms
20ms

Elasticsearch Detalis

The following server configurations were used for the Elasticsearch testing environment:
Key
Value
containers.low.water.mark.demisto/python:1.3-alpine
40
create.related.indicators.entry
True
custom.transformer.override.convertkeystotablefieldformat
True
execution.demisto rest api.demisto-api-post
False
feedintegrationscript.timeout
60
investigation.task.partial.index
15
job.monitor.log
True
log.rolling.backups
20
monitoring.pprof
True
playbook.willnotexecute.old.eval
False
tim.features.enabled
True
workers.count.tasks
1,000

High Availability Details

The following server configurations were used for the high availability testing environment:
Key
Value
containers.low.water.mark.demisto/python:1.3-alpine
40
create.related.indicators.entry
True
custom.transformer.override.convertkeystotablefieldformat
True
disable.msgs.sending
True
execution.demisto rest api.demisto-api-post
False
feedintegrationscript.timeout
60
investigation.task.partial.index
15
job.monitor.log
True
monitoring.pprof
True
playbook.willnotexecute.old.eval
False
reputation.calc.algorithm
1
reputation.calc.algorithm.tasks
1
tim.features.enabled
True
workers.count.tasks
1,000

Recommended For You