Validate the Migration to Elasticsearch

Verify that the Cortex XSOAR migration to Elasticsearch was successful.
After completing the migration process, you should verify that the migration completed successfully. Part of this validation should be done before you start the system, and part can only be done once you start the system.
  1. After completing the migration, navigate to the directory in which your logs are stored. By default, this is /var/log/demisto.
  2. Open the elastic.migration.log file.
  3. At the end of each object migration, there is a summary of the migration.
    To view the migration results in a user-friendly format, run the migration with
    previous-results
    flag.
    For example, we see here that 14725 objects out 14725 object in the main database for the tenant were migrated, meaning, everything was migrated successfully.
    info === file /var/lib/demisto/tenants/acc_<tenantName>/data/demisto.db migrated 14725 out of 14725 objects in 6.311s
    In contrast, if something was not migrated properly, you will see something similar to:
    info successfully migrated: insight: 3024 | 1 failed to migrate, took 1.884s
    In the event that you receive an error like this, contact your support representative.
  4. (
    Optional
    ) - If you are using Kibana, validate the migration in Kibana.
    Verify cluster status is green.
    Verify Cortex XSOAR indexes are green.
  5. Query the Elasticsearch API to verify incident ID offset, cluster health, number of shards, etc.
    Description
    Example
    Retrieves the Incident ID offset, which should match the latest Incident ID in your system prior to the migration. The ID is in the ‘offset’ key, and it is critical that this ID be equal to the ID of the last Incident in BoltDB. If data was not migrated in the correct order, the ID will not be equal. Do not restart the Cortex XSOAR server if it is not equal.
    curl -k -u username:password https://readyxsoarone:9200/dmst-common-incidentidtrack/_doc/thisistheid
    Get cluster health
    curl -k -u username:password https://readyxsoarone:9200/_cluster/health
    GET XSOAR indexes
    curl -k -u username:password https://readyxsoarone:9200/_cat/indices | grep dsmt-
    GET only the health, status, and index
    curl -k -u username:password https://readyxsoarone:9200/_cat/indices?h=health,status,index | grep dsmt-
  6. If no errors were found, start the Cortex XSOAR server.
  7. Perform a basic sanity check on your system. For example, ensure that your content and incidents were migrated.
  8. In the event that there are any errors when starting the server, or if you see that data was not migrated, contact your support representative.

Recommended For You