Log Collector Scaling Optimization
Focus
Focus
What's New in the NetSec Platform

Log Collector Scaling Optimization

Table of Contents

Log Collector Scaling Optimization

Enhance log collection performance and stability by limiting the number of master-eligible nodes in large Collector Groups, achieving throughput up to 1 million logs per second.
PAN-OSĀ® 12.1 introduces support for Log Collector Scaling. This feature allows you to explicitly select master-eligible nodes to address performance bottlenecks in large-scale log collection environments. This optimization provides a more predictable failover behavior and more efficient resource utilization across your Collector Group.
To achieve the best performance, select a maximum of four Log Collectors per Collector Group to be master-eligible. Previously, all Log Collectors within a Collector Group were eligible to become the master node. When the active master failed, the system dynamically elected a new one. This election process involved continuous communication among numerous nodes, creating significant overhead, particularly in larger deployments.
This feature supports all platforms, enabling a significantly higher logging rate. For example, with a Collector Group using up to 16 M-700 appliances, you can scale log ingestion rates to over 1 million Logs Per Second (lps). This level of scaling is currently supported only on M-700 appliances.
You can designate specific Log Collectors as master-eligible nodes based on strategic criteria such as hardware capacity, network resiliency, or geographic distribution. You can configure master-eligible nodes through either the Panorama web interface or the command-line interface.
When you implement this feature, consider selecting nodes with the best hardware specifications, network connectivity, and geographic placement to ensure optimal performance and availability. By strategically designating your master-eligible nodes, you can create a more resilient logging infrastructure that maintains high performance even under demanding conditions.