Elasticsearch Configurations

Improve Elasticsearch performance. List of all of the supported configuration elements for Elasticsearch in the demisto.conf file. Cortex XSOAR
Cortex XSOAR uses the
demisto.conf
file to store the Elasticsearch configuration details. When setting up Elasticsearch for a new Cortex XSOAR installation or migrating data from BoltDB to Elasticsearch, you need to add connection information for the Elasticsearch database to the demisto.conf file. You can also add optional parameters including shards, replicas, and refresh intervals for the indexes that will be created within Elasticsearch
For more information about Elasticsearch configuration for Cortex XSOAR, we recommend reviewing Best Practices before editing the configuration file.
Edit the
/etc/demisto.conf
file to add the new Elasticsearch configuration. At a minimum, you must provide the URL and a username/password or API key. You can also provide an indexPrefix if you want the indexes to be created with a different prefix.
Example
{ "elasticsearch": { "username": "elastic", "password": "changeme", "apiKey": "", "url": "https://readyelasticone:9200", "enabled": true, "insecure": true, "indexPrefix": "", "responseHeaderTimeoutSeconds": 120, "shards": { "common-invplaybook": 3, "common-entry": 3 }, "replicas": { "common-invplaybook": 1, "common-entry": 1 }, "defaultShardsPerIndex": 1, "defaultReplicasPerIndex": 2, "refreshIntervals": { "*": "30s", "common-configuration": "1s", "common-incident": "1s" } } }
If you move to Elasticsearch without migrating data or do a migration that is only partially successful, you need to add the externalEntities key to the Server key. If you successfully migrate your data, the key is added automatically.
"Server": { "HttpsPort": "443", "externalEntities": "incident,indicator,audit,configuration" // store all types of data objects in Elasticsearch },
The following table lists all the possible Elasticsearch configurations supported in the
demisto.conf
file. Use these elements to tweak Elasticsearch performance.
Name
Type
Description
url
string
The url, or urls, comma-separated to an Elasticsearch node. The url includes the schema and port.
Default:
http://localhost:9200
Example:
“url”: "http://localhost:9200"
enabled
boolean
Set to true to enable the use of Elasticsearch as the database.
Default:
false
Example:
“enabled”: true
indexPrefix
string
The index prefix to address when creating, writing, and reading from Cortex XSOAR indexes.
Default:
N/a
Example:
“indexPrefix”: “xsoar”
proxy
boolean
Set to true if Elasticsearch is used behind a proxy service.
Default:
false
Example:
“proxy”: true
username
string
The Elasticsearch username to establish a connection.
* required (unless API key is provided)
Default:
N/a
Example:
“username”: “elastic”
password
string
A plain, tommed, or encrypted password for Elasticsearch to establish connection.
* required (unless API key is used)
Default:
N/a
Example:
“password”: “123”
apiKey
string
The Elasticsearch API key to establish connection.
* required (unless a username and password are used)
Default:
N/a
Example:
“apiKey”: “ani1”
insecure
boolean
Allow an insecure connection to Elasticsearch for unsigned certificates.
Default:
false
Example:
“insecure”: true
responseHeaderTimeoutSeconds
number
The number of seconds for timeout while awaiting response headers from Elasticsearch.
Default:
0
Example:
“responseHeaderTimeoutSeconds”: 60
dialerTimeoutSeconds
number
The number of seconds for timeout while awaiting to establish connection to Elasticsearch.
Default:
30
Example:
“dialerTimeoutSeconds”: 60
maxIdleConnections
number
The maximum number of idle connections to Elasticsearch that remain alive.
Default:
100
Example:
“maxIdleConnections”: 400
idleConnectionTimeout
number
The number of seconds for an idle connection to be closed, if not used.
Default:
100
Example:
“idleConnectionTimeout”: 200
maxRetriesOnConnectivity
number
The amount of maximum retries when connection is refused or not found from Elasticsearch.
Default:
10
Example:
“maxRetriesOnConnectivity”: 50
shards
object
An index name for the number of shards for an object to determine shards allocation on each index.
Default:
N/a
Example:
“shards”: { “common-incident”: 1 }
replicas
object
An index name for the number of replica objects to determine the replicas allocation on each index.
Default:
N/a
Example:
“replicas”: { “common- incident”: 1 }
defaultShardsPerIndex
number
The default number of shards for each index created using Cortex XSOAR.
Default:
N/a
Example:
“defaultShardsPerIndex”: 1
defaultReplicasPerIndex
number
The default number of replicas for each index created using Cortex XSOAR.
Default:
N/a
Example:
“defaultReplicasPerIndex”: 2
totalFields
object
An index name for the number of maximum fields allowed in the index mapping.
Default:
{ “common-incident”: 2000, “common-indicator”: 2000, “common-evidence”: 2000 }
Example:
{ “common- incident”: 5000 }
refreshIntervals
object
An index name to number of seconds to perform refresh on the index. Use
*
for all indexes.
Default:
N/a
Example:
{ “common-configuration”: 30, “common-incident”: 1 }
innerBatchSize
number
The amount of maximum objects to send to Elasticsearch when using bulk updates.
Default:
250
Example:
“innerBatchSize”: 500
maxRetriesOnVersionConflicts
number
Amount of retries to run on version conflict errors received from Elasticsearch.
Default:
10
Example:
“maxRetriesOnVersionConflicts”: 30
maxResultWindow
number
Maximum allowed results to return from Elasticsearch searches.
Default:
10000
Example:
“maxResultWindow”: 50000
aggregationMaxSize
number
Maximum allowed results to return from Elasticsearch aggregations.
Default:
1000
Example:
“aggregationMaxSize”: 5000
lastFeedFetchLimit
number
Maximum allowed objects to store on the last feed fetch limit.
Default:
300000
Example:
“lastFeedFetchLimit”: 100000
disableCreateIncidentForceIndex
boolean
Disable force refresh on new incidents to allow back-to-back searches using pre-processing scripts.
Default:
false
Example:
“disableCreateIncidentForceIndex”: true
template.patternPrefix
string
Uses a custom template name prefix.
Default:
indexPrefix
Example:
“template.patternPrefix”: “template-name”
role.roles
array
Default cluster roles to provide new users in Elasticsearch created in Cortex XSOAR.
Default:
[“manage”, “monitor”, “manage_own_api_key”]
Example:
“role.roles”: [“manage”]
api.roles
array
Default cluster roles to provide new API keys in Elasticsearch created in Cortex XSOAR.
Default:
[“manage”, “monitor”, “manage_own_api_key”]
Example:
“api.roles”: [“manage”]
debug.enableQuery
boolean
Show Elasticsearch built queries before sending when using the log level debug.
Default:
false
Example:
“debug.enableQuery”: true

Recommended For You