Best Practices

Outline of how Elasticsearch should be configured for working with Cortex XSOAR high availability. Best practices
Elasticsearch comes with built-in support for data redundancy. Refer to the Elasticsearch documentation to ensure that you have configured your database properly for high availability.
In addition, there are certain best practices and configurations that will enable you to work best with Cortex XSOAR.

Performance

  • To achieve high availability, you will need to set up at least 3 nodes.
  • It is important to utilize Elasticsearch versatile behavior using shards and replicas. If you are using more than 1 node, Cortex XSOAR recommends that you set the
    elasticsearch.defaultShardsPerIndex
    and
    elasticsearch.defaultReplicasPerIndex
    to 1 and 2, respectively. You can also use
    elasticsearch.shards
    and
    elasticsearch.replicas
    to configure specific number of shards and replicas respectively on each index.
    For example, setting the following values in
    demisto.conf
    sets 2 replicas for each shard for the incidents index:
    "elasticsearch": { "shards": { "common-incident": 1 }, "replicas": { "common-incident": 2 } } }
    This configuration takes effect when the index is created.
  • Suggested configuration based on the number of data nodes in the Elasticsearch cluster:
    Configuration
    2 Data Nodes
    3 Data Nodes
    4 Data Nodes
    defaultShardsPerIndex
    1
    1
    1
    defaultReplicasPerIndex
    1
    2
    3
    Shards:
    - common entry
    - common-invplaybook
    2
    3
    4
    Replicas:
    - common entry
    - common-invplaybook
    1
    1
    1
  • Cortex XSOAR uses internal batch to Elasticsearch to avoid heavy requests and large indexing latency. You can increase the batch size using
    elasticsearch.innerBatchSize
    to increase performance and speed, or decrease the value if Elasticsearch is not optimized.
  • Due to Cortex XSOAR’s high usage of Elasticsearch, Cortex XSOAR recommends to have the lowest latency possible between XSOAR server and the Elasticsearch nodes configured. It is also recommended to have low latency between nodes to avoid slow indexing or possible data loss on outages. Any latency between Cortex XSOAR and Elasticsearch will greatly impact performance.
  • Elasticsearch uses an inner memory management implemented in JAVA that is called the heap size. It defaults to 1GB and Cortex XSOAR recommends you increase the heap size to 50% of machine's memory, or more if the machine is dedicated to Elasticsearch.
  • Elasticsearch uses a refreshing mechanism to flush updated documents to the disk every configured seconds. Cortex XSOAR recommends setting up the configuration index to 30 seconds to avoid unneeded refreshing and better performance on that index. You can use the Cortex XSOAR
    demisto.conf
    file to set a specific refreshing interval in seconds for each index using
    elasticsearch.refreshIntervals
    .
    For example, the following sets the configuration index to 30 seconds:
    "elasticsearch": { "refreshIntervals": { "common-configuration": 30 } }
    This configuration takes effect when the index is created.
  • Cortex XSOAR recommends using the most recent supported version of Elasticsearch, to ensure you have the latest features and all security updates.
  • Circuit breakers in Elasticsearch are enabled by default. Verify they are enabled and configured correctly, specifically the
    request circuit breaker
    and the
    parent circuit breaker
    . These circuit breakers will prevent data nodes from crashing on high memory consumption and will also drop certain requests that Cortex XSOAR will retry later.
  • Elasticsearch requires a large number of open file descriptors. Verify that the limit on the number of open file descriptors is sufficient to handle a high load of requests, to avoid possible data loss or service interruptions.

Scaling

To optimize an Elasticsearch deployment for scaling, we recommend using dedicated master node(s) (1 or 3 for full high availability) as well as coordinating nodes to handle requests, thus having all connections and requests go through the coordinating nodes. Using dedicated coordinating nodes improves Elasticsearch balancing when some data nodes might be unavailable or unresponsive due to high load. Dedicated master node(s) ease data nodes responsibility around shard management, snapshots and monitoring. Elasticsearch distributes the requests more efficiently through the use of master and coordinating nodes.
We also recommend using a load balancer or round-robin DNS server when configuring an elasticsearch.url that points to all/any coordinating nodes. A load balancer enables you to swap, add or remove data nodes, master nodes and coordinating nodes without having to update the Cortex XSOAR configuration file and restart the Cortex XSOAR service, thus significantly decreasing down time for any changes.

Mapping

  • Indexes with custom fields such as incidents, evidence, and indicators might exceed the max fields mapping limit in Elasticsearch. By default, Cortex XSOAR sets the limit to 2000. If you require more, you can increase the
    elasticsearch.totalFields
    field in the
    demisto.conf
    file.
    For example, the following sets the incidents index maximum to 3000 mapped fields:
    "elasticsearch": { "totalFields": { "common-incident": 3000 } }
    This configuration takes effect when the index is created.

Suggested Configuration for High Availability

The following is a recommended configuration for Cortex XSOAR indexing within Elasticsearch for 3 data nodes.
"shards": { "common-invplaybook": 3, "common-entry": 3 }, "replicas": { "common-invplaybook": 1, "common-entry": 1 }, "defaultShardsPerIndex": 1, "defaultReplicasPerIndex": 2, "refreshIntervals": { "*": "30s", "common-configuration": "1s", "common-incident": "1s" } }
In this configuration, the default shards and replicas as 1 and 2, respectively, to support the high availability functionality. In addition, we have set the playbook and entry index shards to 3 apiece. This is done as playbooks and entries are both resource intensive.
Also, the refresh interval is set to 30 seconds (the default in Elasticsearch is 1 second) and have defined specific indexes, such as the incidents and configurations, to 1 second. This allows for more efficient accessibility for these specific indexes.

Recommended For You