Focus

Panorama Administrator's Guide

Optimize Log Collector Scaling

Table of Contents

Optimize Log Collector Scaling

Address the challenges of managing large numbers of Log Collectors within a Collector Group and streamline the management node configuration to optimize overall system performance.

Within each Collector Group, a single Log Collector is designated as the master node. This master node is responsible for managing and coordinating the entire log collection process. In the event of a master node failure, the system automatically initiates an internal election process to select a new master node.

This election mechanism relies on master-eligible nodes participating in a consensus-based process, where a specific number of nodes must agree on the election outcome. To ensure a robust election, ideally, more than half of the master-eligible nodes must agree.

In PAN-OS versions 12.1.2 and earlier releases, every Log Collector in a Collector Group was automatically considered master-eligible. This meant all nodes participated in election processes, regardless of their primary function.

With the new configurable feature, you can now explicitly select up to a maximum of four master-eligible nodes, irrespective of the total number of Log Collectors available in the Collector Group. This change reduces the number of nodes actively involved in the election processes. The reduction in management-eligible nodes results in improved performance for the entire Collector Group by minimizing the overhead associated with constant communication between nodes for the election processes.

Log collector scaling is supported across all platforms allowing a significantly higher logging rate, which is crucial for large-scale deployments. With a Collector Group utilizing up to 16 M-700 appliances, you can now scale log ingestion rates to over 1 million Logs Per Second (LPS). This level of scaling is currently supported exclusively on M-700 appliances.

You can use the Panorama web Interface or the Panorama command-line interface (CLI) to configure master-eligible nodes. For more information about configuring master-eligible nodes, see Collector Group Configuration.

You can choose the master-eligible management nodes based on the resiliency and availability of the appliances in your environment. A strategic selection process helps in optimal performance while maintaining the stability of the Log Collector.

Manage Collector Groups

Configure a Collector Group