Migrate Objects to Elasticsearch for a Distributed Database

For the main database, copy the Cortex XSOAR database either by taking a snapshot OR manually create a copy of the
/var/lib/demisto/data
directory and the
demisto.conf
file. Then follow the same procedure for each node.
Download the migration tool by appending
downloadName=elasticsearch_migration_tool_6_5_0
to the end of the download link that you received, when installing Cortex XSOAR.
Copy your database and migrate data from the copy database to Elasticsearch.
It is recommended to copy your data up to the last 3 months, without any downtime. If you do not want to do this, go to step 4.
Copy the Cortex XSOAR database by doing one of the following:
Take a snapshot of the database.
Manually create a copy of the
/var/lib/demisto/data
directory.
Copy the
demisto.conf
file.
Edit the copy of the
demisto.conf
file, by adding your Elasticsearch configuration.
Ensure that the same Elasticsearch object exists in the
demisto.conf
on the app server and the main database and hat
elasticsearch
is the top-level object in the
demisto.config
file (within the main curly brackets).
Edit the copies of
demisto.conf
for the main database and for each node to add your Elasticsearch configuration.
For the main database and for each node, using
demisto
or
sudo
permissions, run the following command:
sudo ./elasticMigrator -config-path
<file path-of-copy-of-demisto.conf>
-db-path
<path-of-the-copy-of-the-demisto-database>
-
<flags>
For a full list of the flags, see Migration Tool Flags. For example, to exclude the last 3 partitions from the migration, add the
-only-old-partitions
and
-partitions-back
flags:
sudo ./elasticMigrator -config-path /usr/local/dev/copy_of_demisto.conf -db-path /usr/local/dev/lib_demisto_copy/data -only-old-partitions -partitions-back 3
When you run the migration tool, parameter values specified in the
demisto.conf
file override values supplied for tool flags and default values. If no value exists in the
demisto.conf
file, values supplied in the tool flags override default values, but do not write the values to the
demisto.config
file.
Complete steps 1 to 3 in Validate the migration.
After the migration of the data is complete and validated, migrate your data from the active database to Elasticsearch.
Create a backup copy of the
demisto.conf
file for your active database.
Edit the original
demisto.conf
file (not a copy) for the main database and for each node to add your Elasticsearch configuration. Ensure that the same Elasticsearch object exists in the
demisto.conf
on the app server and the main database.
For the main database and for each node, stop the Cortex XSOAR server.
CentOS:
sudo systemctl stop demisto
Ubuntu:
sudo service demisto stop
For the main database and for each node, run the
sudo ./elasticMigrator
command with either
demisto
or
sudo
permissions. Use the
partitions-back
flag to specify the remaining partitions.
For example,
sudo ./elasticMigrator -partitions-back 3
migrates the last three partitions, which would include the current month and the previous two months, as well as the main partition.
Validate the migration (all steps).