Recover From a Split-Brain Condition

To resolve a split-brain condition, debug your network issues and then restore connectivity between the WildFire HA pair. WildFire appliance clusters automatically attempt to recover from split-brain conditions, but if those measures fail, you must manually initiate the recovery process.
  1. Verify that your network is operating normally and that the WildFire appliance is transmitting and receiving traffic.
    1. Enable the ability to ping on a WildFire appliance interface.
      • Enable ping on a specific appliance interface—set deviceconfig system <interface_number> service disable-icmp no
      • Enable ping on all appliance interfaces—set deviceconfig system service disable-icmp no
    2. Generate ping traffic from a WildFire interface to an external device. Verify that the received and transmitted counters increment.
      ping source <wildfire-interface-ip> host <destination-ip-address>
  2. Determine which WildFire appliance is unhealthy. Refer to View WildFire Cluster Status Using the CLI or View WildFire Cluster Status Using Panorama to view the status of the appliance.
  3. Gracefully restart the unhealthy node using the following command:
    request cluster reboot-local-node
    The WildFire appliance that is rebooted should auto-enroll into the WildFire cluster it was configured for.
    The remaining controller node that is in split-brain mode must be in a healthy state.
  4. Wait for the Data Migration to complete. Run show cluster membership and refer to Application status to view the status of the database merge. After the data merge is complete, it displays readyleader:
    Application status:
                      wildfire-apps-service: Stopped
                      global-db-service: DataMigration
    The duration of a data merge depends on the amount of data stored on the WildFire appliance. Be sure to allot at least several hours for recovery as the data merge can be a lengthy process.
  5. Verify the status of the cluster on Panorama or through the WildFire appliance CLI.

Related Documentation