: Configure a Collector Group
Focus
Focus

Configure a Collector Group

Table of Contents
End-of-Life (EoL)

Configure a Collector Group

Before configuring Collector Groups, decide whether each one will have a single Log Collector or multiple Log Collectors (up to 16). A Collector Group with multiple Log Collectors supports higher logging rates and log redundancy but has the following requirements:
  • In any single Collector Group, all the Log Collectors must run on the same Panorama model: all M-700 appliances, all M-600 appliances, all M-500 appliances, all M-300 appliances, all M-200 appliances, or all Panorama virtual appliances.
  • Log redundancy is available only if each Log Collector has the same number of logging disks. To add disks to a Log Collector, see Increase Storage on the M-Series Appliance.
  • (Best Practice) All Log Collectors in the same Collector Group should be in the same local area network (LAN). Avoid adding Log Collectors in the same or different wide area networks (WAN) to the same Collector Group as network disruption are much more common and may result in log data loss. Additionally, it is recommended that Log Collectors in the same Collector Group be in close physical proximity to each other to allow Panorama to quickly query the Log Collectors when needed.
You must add the Log Collector to a Collector Group and push the Collector Group configuration to the Log Collector before it can start ingesting firewall logs. Otherwise, the ElasticSearch health status displays as degraded and the Log Collector cannot ingest logs until added to a Collector Group.
  1. Perform the following tasks before configuring the Collector Group.
    1. Add a Firewall as a Managed Device for each firewall that you will assign to the Collector Group.
    2. Configure a Managed Collector for each Log Collector that you will assign to the Collector Group.
  2. Add the Collector Group.
    1. Access the Panorama web interface, select PanoramaCollector Groups, and Add a Collector Group or edit an existing one.
    2. Enter a Name for the Collector Group if you are adding one.
      You cannot rename an existing Collector Group.
    3. Enter the Minimum Retention Period in days (1 to 2,000) for which the Collector Group will retain firewall logs.
      By default, the field is blank, which means the Collector Group retains logs indefinitely.
    4. Add Log Collectors (1 to 16) to the Collector Group Members list.
    5. (Recommended) Enable log redundancy across collectors if you are adding multiple Log Collectors to a single Collector group.
      Redundancy ensures that no logs are lost if any one Log Collector becomes unavailable. Each log will have two copies and each copy will reside on a different Log Collector. For example, if you have two Log Collectors in the collector group the log is written to both Log Collectors.
      Enabling redundancy creates more logs and therefore requires more storage capacity, reducing storage capability in half. When a Collector Group runs out of space, it deletes older logs. Redundancy also doubles the log processing traffic in a Collector Group, which reduces its maximum logging rate by half, as each Log Collector must distribute a copy of each log it receives.
  3. Assign Log Collectors and firewalls to the Collector Group.
    1. Select Device Log Forwarding and Add log forwarding preference lists for the firewalls.
      Log data is forwarded over a separate TCP channel. By adding a log forwarding preference, list you enable the creation of separate TCP connections for forwarding log data.
      A preference list determines the order in which Log Collectors receive logs from a firewall. If a log forwarding preference list is not assigned, you may encounter one of the following scenarios:
      • If Panorama is in Management Only mode, Panorama drops all incoming logs.
      • If the local Log Collector is not configured as a managed collector when Panorama is in Panorama mode, Panorama drops all incoming logs.
      • If the local Log Collector is configured as a managed collector when Panorama is in Panorama mode, incoming logs are received but the Panorama may act as a bottleneck because all managed firewalls are forwarding logs to the local Log Collector first before being redistributed to other available Log Collectors.
      1. In the Devices section, Modify the list of firewalls and click OK.
      2. In the Collectors section, Add Log Collectors to the preference list.
        If you enabled redundancy in Step 2, it is recommended to add at least two Log Collectors. If you assign multiple Log Collectors, the first one will be the primary; if the primary becomes unavailable, the firewalls send logs to the next Log Collector in the list. To change the priority of a Log Collector, select it and Move Up (higher priority) or Move Down (lower priority).
      3. Click OK.
  4. Define the storage capacity (log quotas) and expiration period for each log type.
    1. Return to the General tab and click the Log Storage value.
      If the field displays 0MB, verify that you enabled the disk pairs for logging and committed the changes (see Configure a Managed Collector, Disks tab).
    2. Enter the log storage Quota(%) for each log type.
    3. Enter the Max Days (expiration period) for each log type (1 to 2,000).
      By default, the fields are blank, which means the logs never expire.
  5. Commit and verify your changes.
    1. Select CommitCommit and Push and then Commit and Push your changes to Panorama and the Collector Group you configured.
    2. Select PanoramaManaged Collectors to verify the Log Collectors in the Collector Group are:
      • Connected to Panorama—The Connected column displays a check mark icon to indicate that a Log Collector is connected to Panorama.
      • Synchronized with Panorama—The Configuration Status column indicates whether a Log Collector is In Sync (green icon) or Out of Sync (red icon) with Panorama.
  6. Troubleshoot Connectivity to Network Resources to verify your firewalls successfully connected to the Log Collector.
  7. Next steps...
    1. Configure Log Forwarding to Panorama.
      The Collector Group won’t receive firewall logs until you configure the firewalls to forward to Panorama.
    2. (Optional) Configure Log Forwarding from Panorama to External Destinations.
      You can configure each Collector Group to forward logs to separate destinations (such as a syslog server).