Free up Disk Space with Data Archiving
Free up disk space by archiving Cortex XSOAR folders
to condense the unused data within them.
Cortex XSOAR supports full archiving of incidents,
entries and indicators by month. Data from incidents, insights (indicators),
and entries are stored in folders on a monthly basis.
If
disk space needs to be freed up, you can archive folders to condense
unused data within them. It is recommended to archive folders and
not delete them permanently.
For Elasticsearch deployments,
see Archive Data with Elasticsearch. For Hosted
Service deployments, contact Customer Support.
Locate
the folders that reside in the following location (where Cortex
XSOAR is installed),
/var/lib/demisto/data/
. Although
the folders reside in
/var/lib/demisto/data/
, Do
Not
save the backup folders under /var/lib/demisto/
. The
instructions below refer to paths for a single server deployment.
For multi-tenant, you archive data on a per tenant basis. The following
are the equivalent paths for multi-tenant:
Single Server | Multi-Tenant |
|---|---|
/var/lib/demisto/ | /var/lib/demisto/tenants/acc_{ TENANT_NAME }/ |
/var/lib/demisto/data/ | /var/lib/demisto/tenants/acc_{ TENANT_NAME }/data/ |
/var/lib/demisto-archive/ | /var/lib/demisto-archive/acc_{ TENANT_NAME }/ |
The following data folder and files can
be found in this folder:
- demisto.db: The database for all playbooks and automation (not anything related to incidents and insights).
- demistoidx: Indexing of the system.
- partitionsData: Data of incidents, insights, and entries separated by month resolution.If you archive indexes you need to archive the matching partitions, so they are not rebuilt.
The following is an example
of how the folders and filenames will appear in your system.
$ tree /var/lib/demisto/data ├── demisto.db ├── demistoidx │ ├── accounts │ │ ├── index_meta.json │ │ └── store ... │ ├── entries_082017 │ │ ├── index_meta.json │ │ └── store │ ├── entries_092017 │ │ ├── index_meta.json │ │ └── store │ ├── entries_102017 │ │ ├── index_meta.json │ │ └── store │ ├── evidences │ │ ├── index_meta.json │ │ └── store │ ├── incidents_082017 │ │ ├── index_meta.json │ │ └── store │ ├── incidents_092017 │ │ ├── index_meta.json │ │ └── store │ ├── incidents_102017 │ │ ├── index_meta.json │ │ └── store │ ├── investigations_082017 │ │ ├── index_meta.json │ │ └── store │ ├── investigations_092017 │ │ ├── index_meta.json │ │ └── store │ ├── investigations_102017 │ │ ├── index_meta.json │ │ └── store ... │ ├── newInsights_082017 │ │ ├── index_meta.json │ │ └── store │ ├── newInsights_092017 │ │ ├── index_meta.json │ │ └── store │ ├── newInsights_102017 │ │ ├── index_meta.json │ │ └── store │ ├── playbooks │ │ ├── index_meta.json │ │ └── store ... └── partitionsData ├── demisto_082017.db ├── demisto_092017.db └── demisto_102017.db
Follow these steps to free up disk space by
archiving folders.
In a distributed database deployment,
first stop the app server and then the databases. Then run this
procedure on each database that contains incidents. By default,
you only need to run this procedure on the secondary nodes. However,
in distributed database deployments that were converted from a single server
deployment, there is incident data in the main database. Therefore,
you also need to run this procedure on the main database.
- Stop the Cortex XSOAR service using the following command.$ sudo service demisto stop
- Create the following directories:mkdir /var/lib/demisto-archivemkdir /var/lib/demisto-archive/archived-2019
- Navigate to the/var/lib/demisto-archive/filepath using the following command.cd /var/lib/demisto-archive/
- Move the data you want to archive to the archive directory using the following command. The following command moves all folders that have ammyyyysuffix.mv /var/lib/demisto/data/**/*_<date_to_archive>* /var/lib/demisto-archive/archived-2019For example:mv /var/lib/demisto/data/**/*_092019* /var/lib/demisto-archive/Multi-tenant example:mv /var/lib/demisto/tenants/acc_{TENANT_NAME}/data/**/*_<date_to_archive>* var/lib/demisto-archive/archived-2019If themv /var/lib/demisto/data/**/*_<date_to_archive>* /var/lib/demisto-archive/archived-2019command does not work in your environment, archive the indicies and partition separately. First create the following directories:mkdir /var/lib/demisto-archive/archived-2019/demistoidx/mkdir /var/lib/demisto-archive/archived-2019/partitionsData/Then, archive each index and the partition that has a mmyyyy suffix. The following are the commands to archive the current indices and the partition:sudo mv /var/lib/demisto/data/demistoidx/entries_082019 /var/lib/demistoarchive/archived-2019/demistoidx/sudo mv /var/lib/demisto/data/demistoidx/evidences_082019 /var/lib/demistoarchive/archived-2019/demistoidx/sudo mv /var/lib/demisto/data/demistoidx/incidents_082019 /var/lib/demistoarchive/archived-2019/demistoidx/sudo mv /var/lib/demisto/data/demistoidx/invTaskIdx_082019 /var/lib/demistoarchive/archived-2019/demistoidx/sudo mv /var/lib/demisto/data/demistoidx/investigations_082019 /var/lib/demistoarchive/archived-2019/demistoidx/sudo mv /var/lib/demisto/data/demistoidx/newInsights_082019 /var/lib/demistoarchive/archived-2019/demistoidx/sudo mv /var/lib/demisto/data/demistoidx/todosTask_082019 /var/lib/demistoarchive/archived-2019/demistoidx/sudo mv /var/lib/demisto/data/partitionsData/demisto_082019.db /var/lib/demisto-archive/archived-2019/partitionsData/demisto_082019.db
- Create the compressed archive of your selected files and folders using the following tarball command.$ tar -cvzf demisto-2019-archive.tar.gz /var/lib/demisto-archive/archived-2019
- Start the Cortex XSOAR service using the following command.$ sudo service demisto start
Recommended For You
Recommended Videos
Recommended videos not found.
