Elasticsearch Overview - Administrator Guide - 6.8 - Cortex XSOAR - Cortex - Security Operations

Cortex XSOAR Administrator Guide

Product
Cortex XSOAR
Version
6.8
Creation date
2022-09-28
Last date published
2024-03-21
End_of_Life
EoL
Category
Administrator Guide
Abstract

Migrating Cortex XSOAR data to Elasticsearch. Object information in an existing Cortex XSOAR instance is copied to a designated Elasticsearch index.

Elasticsearch is a distributed, open source search and analytics engine for all types of data. It enables processing and storing large amounts of data.

When you migrate the objects that exist in Cortex XSOAR to Elasticsearch, they are moved to a designated index used by a specific Elasticsearch instance. Every Elasticsearch index is composed of at least one primary shard where the data is stored. A replica shard is a copy of a primary shard. Replicas provide redundant copies of your data to protect against hardware failure and increase capacity to serve read requests like searching or retrieving data.

The following diagram depicts a Cortex XSOAR environment with Elasticsearch.

xsoar-elasticsearch-install.png

The number of primary shards in an index is fixed at the time that an index is created, but the number of replica shards can be changed at any time, without interrupting indexing or query operations.

To migrate your data to Elasticsearch, you must be running Cortex XSOAR v6.0 or later and have an Elasticsearch installation available. We recommend that you install Elasticsearch on a different server than Cortex XSOAR due to the high memory consumption for both services.

You perform the migration by running the migration tool, which is a standalone binary file. The binary file must be run with either sudo or admin permission. The migration tool uses the demisto.conf file to read the following information:

  • Database location

  • Partitions data

Warning

You must stop the Cortex XSOAR server before you run the migration tool. This enables the tool to safely access the database and required configurations.

The migration tool begins by reading the Cortex XSOAR database to identify existing partitions and custom fields. It then creates the index (if it does not already exist) based on the Elasticsearch configuration in the demisto.conf file, or based on the Elasticsearch default configurations set in the Elasticsearch cluster setting. After Elasticsearch is successfully configured, the tool reads each partition, from older to newer, and copies all objects to the index. Duplicate objects are overridden, taking only the latest version of the object.

When working with Elasticsearch, Cortex XSOAR does not maintain, nor is it responsible, for the following:

  • Redundancy

  • Backups

  • Security

  • Elasticsearch clusters

Note

Moving data from the Elasticsearch database back to the Cortex XSOAR Bolt database is not supported.