Focus

Panorama Administrator's Guide

Optimize Log Collector Scaling

Table of Contents

Optimize Log Collector Scaling

Address the challenges of managing large numbers of Log Collectors within a Collector Group and streamline the management node configuration to optimize overall system performance.

With (PAN-OS 12.1.2 and later versions), you can manage large numbers of Log Collectors within a Collector Group and streamline the master node configuration to optimize overall system performance.

Within each Collector Group, a single Log Collector is designated as the master node. This master node is responsible for managing and coordinating the entire log collection process. In the event of a master node failure, the system automatically initiates an internal election process to select a new master node. This election mechanism relies on master-eligible nodes participating in a consensus-based process, where a specific number of nodes must agree on the election outcome. Ideally, more than half of the master-eligible nodes must agree.

In PAN-OS versions 12.1.2 and earlier releases, every Log Collector in a Collector Group was automatically considered master-eligible. This meant all nodes participated in election processes, regardless of their primary function.

With the new configurable feature, you can now explicitly select master-eligible nodes. This enhancement allows you to select a reduced number of master-eligible nodes. The reduction in master-eligible nodes results in improved performance for the entire Collector Group by minimizing the overhead associated with constant communication between nodes for the election processes. By default, all the master-eligible nodes are selected. The system recommends you to select up to 4 master-eligible nodes for best performance.

Scalability

Log collector scaling is supported across all platforms allowing a significantly higher logging rate, which is crucial for large-scale deployments. With a Collector Group utilizing up to 16 M-700 appliances, you can now scale log ingestion rates to over 1 million Logs Per Second (LPS). This level of scaling is currently supported exclusively on M-700 appliances.

Configuration

You can use the Panorama web interface or the command-line interface (CLI) to configure master-eligible nodes. For more information about configuring master-eligible nodes, see Collector Group Configuration.

You can choose the master-eligible management nodes based on the resiliency and availability of the appliances in your environment. A strategic selection process helps in optimal performance while maintaining the stability of the Log Collector. For more information about selecting management nodes, see Configure Collector Groups.

You must configure at least one master node to save your configuration. While a single master node is sufficient, it is not the ideal setup, as logging is disrupted if that node goes down. When you add new Log Collectors to a Collector Group, the default master node setting is pre-checked.

Manage Collector Groups

Configure a Collector Group