Migrate an Existing Elasticsearch Deployment
Migrate a Cortex XSOAR deployment that already uses Elasticsearch for managing indicators.
To use Cortex XSOAR version 6.2 with Elasticsearch, you must migrate all of your existing content. It is no longer possible to use Elasticsearch to manage only indicators.
In the BoltDB, data related to incidents and indicators is stored in partitions by month. To minimize downtime during the migration, we recommend you create a copy of the database, then migrate data that is older than three months from the copy, while continuing to work in your current environment. Once the initial migration is completed, you should then migrate the last three months.
If you are working in an environment with remote repositories, you must run the migration separately on each environment. For example, if both your development and production environments are going to be used with an Elasticsearch database, you must migrate each of those environments, and each environment must use a different index prefix.
Run all commands from the Cortex XSOAR server machine. To migrate your data, you need to use the migration tool.
You cannot run more than one migration tool process at a time.
Always migrate older data before newer data. Migrating partitions out of order can cause duplicate incident ids.
- Upgrade the Cortex XSOAR server to 6.1, but do not start the server.
- Download the migration tool by appendingdownloadName=elasticsearch_migration_tool_6_1_0to the end of the download link that you received, when installing Cortex XSOAR.
- Copy your database and migrate data from the copy database to Elasticsearch.
- Copy the Cortex XSOAR database by doing one of the following:
- Copy thedemisto.conffile.
- Edit the copy of thedemisto.conffile, by adding your Elasticsearch configuration.Ensure thatelasticsearchis the top-level object in thedemisto.configfile (within the main curly brackets).
- Usingdemistoorsudopermissions, run the following command:sudo ./elasticMigrator -config-path<file path-of-copy-of-demisto.conf>-db-path<path-of-the-copy-of-the-demisto-database>-<flags>For a full list of the flags, see Migration Tool Flags. For example, to exclude the last 3 partitions from the migration, add the-partitions-to-ignoreflag and value to the command by typing the following:sudo ./elasticMigrator -config-path /usr/local/dev/copy_of_demisto.conf -db-path /usr/local/dev/lib_demisto_copy/data -partitions-to-ignore '042021,052021,062021'When you run the migration tool, parameter values specified in thedemisto.conffile override values supplied for tool flags and default values. If no value exists in thedemisto.conffile, values supplied in the tool flags override default values, but do not write the values to thedemisto.configfile.To avoid overwriting indicators that might already exist in Elasticsearch, you must run the migration with the-objects-to-ignore "newInsights"flag. If you already migrated audits in a previous version, you must run the migration with the-objects-to-ignore "newInsights, audits"flag.Example:sudo ./elasticMigrator -config-path /usr/local/dev/copy_of_demisto.conf -db-path /usr/local/dev/lib_demisto_copy/data -objects-to-ignore “newInsights” -partitions-to-ignore '042021,052021,062021'
- Complete steps 1 to 3 in Validate the migration.
- After the migration of the data is complete and validated, migrate your data from the active database to Elasticsearch.
- Create a backup copy of thedemisto.conffile for your active database.
- Edit thedemisto.confas needed.
- Stop the Cortex XSOAR server.
- CentOS:sudo systemctl stop demisto
- Ubuntu:sudo service demisto stop
- Migrate the remaining months from the active database to Elasticsearch. Use the-partitionsflag to specify the remaining partitions.To avoid overwriting indicators that might already exist in Elasticsearch, you must run the migration with the-objects-to-ignore "newInsights"flag. If your already migrated audits in a previous version, you must run the migration with the-objects-to-ignore "newInsights, audits"flag.Example:sudo ./elasticMigrator -partitions '042021,052021,062021' -objects-to-ignore “newInsights”
- Validate the migration (all steps).
Recommended For You
Recommended videos not found.