Focus

CN-Series

Test Case: CN-DB Failure Handling

Table of Contents

Test Case: CN-DB Failure Handling

Where Can I Use This?	What Do I Need?
CN-Series HSF Firewall deployment	CN-Series 11.0.x or above Container Images Panorama running PAN-OS 11.0.x or above version

This test evaluates CN-DB failure handling. The preferred number of CN-DB pods for a CN-Series HSF deployment is two. Both CN-DBs have the same configuration.

When CN-DB 1 is down for extended period of time, CN-DB 2 takes care of the existing sessions and setting up new sessions. When CN-DB 1 is UP again, it checks for session sync, lookup, and teardown for existing sessions and set up new sessions.

View cluster membership details of CN-MGMT pod using the command

show cluster-membership show-slot-info slot all

MP leader status: Leader
        Slot-id Type            CI-IP           TI-IP           State   CI-State        TI-State
        ========================================================================================
        1       CN-GW           192.168.23.100  192.168.24.80   UP      UP              UP
        10      CN-NGFW         192.168.23.81   192.168.24.82   UP      UP              UP
        2       CN-GW           192.168.23.101  192.168.24.100  UP      UP              UP
        5       CN-DB           192.168.23.102  ::              UP      UP              NA
        6       CN-DB           192.168.23.104  ::              UP      UP              NA
        7       CN-NGFW         192.168.23.103  192.168.24.86   UP      UP              UP
        8       CN-NGFW         192.168.23.105  192.168.24.84   UP      UP              UP
        9       CN-NGFW         192.168.23.82   192.168.24.81   UP      UP              UP

Delete the CN-DB pod in Slot 6.

Get the CN-DB pod name on Slot 6 using the command

show clusters name cluster-001

from the Panorama CLI.

Cluster: cluster-001
Creation time: 2022/11/22 05:11:09
CN-MGMT pods:
8FF0233D36BD57D (active, pan-mgmt-sts-1.cluster-001, connected, In Sync)
8F846238B0740D2 (pan-mgmt-sts-0.cluster-001, connected, In Sync)


Slot-ID         PodName                             Type                      Version
----------------------------------------------------------------------------------------
5               pan-db-dep-7b6f6c5458-5fgnr         CN-DB                11.0.1-c156.dev_e_rel
1               pan-gw-dep-748cdb856d-4f66g         CN-GW                11.0.1-c156.dev_e_rel
2               pan-gw-dep-748cdb856d-p5qdd         CN-GW                11.0.1-c156.dev_e_rel
7               pan-ngfw-dep-56cdfdd656-srmdt       CN-NGFW              11.0.1-c156.dev_e_rel
8               pan-ngfw-dep-56cdfdd656-hvcw2       CN-NGFW              11.0.1-c156.dev_e_rel
9               pan-ngfw-dep-56cdfdd656-bjtmd       CN-NGFW              11.0.1-c156.dev_e_rel
10              pan-ngfw-dep-56cdfdd656-6jq2f       CN-NGFW              11.0.1-c156.dev_e_rel
6               pan-db-dep-7b6f6c5458-4tvpq         CN-DB                11.0.1-c156.dev_e_rel

From the controller CLI, enter the command

kubectl delete pod pan-db-dep-7b6f6c5458-4tvpq -n kube-system

to delete the CN-DB pod in Slot 6.

The CN-DB pod in Slot 6 is now deleted.

admin@pan-mgmt-sts-1.cluster-001> show cluster-membership show-slot-info slot all

        MP leader status: Leader
        Slot-id Type            CI-IP           TI-IP           State   CI-State        TI-State
        ========================================================================================
        1       CN-GW           192.168.23.100  192.168.24.80   UP      UP              UP
        10      CN-NGFW         192.168.23.81   192.168.24.82   UP      UP              UP
        2       CN-GW           192.168.23.101  192.168.24.100  UP      UP              UP
        5       CN-DB           192.168.23.102  ::              UP      UP              NA
        7       CN-NGFW         192.168.23.103  192.168.24.86   UP      UP              UP
        8       CN-NGFW         192.168.23.105  192.168.24.84   UP      UP              UP
        9       CN-NGFW         192.168.23.82   192.168.24.81   UP      UP              UP

Check the cluster traffic flow using the command

show cluster-flow all

Slot 5
--------------------------------------------------------------------------------
Id      State  Type Src[Sport]/Proto       Dst[Dport]
--------------------------------------------------------------------------------
536870953 ACTIVE  FLOW 192.168.101.100[3784]/17 192.168.101.6[49156]
536870958 ACTIVE  FLOW 192.168.200.100[48706]/6 192.168.250.100[22]
536870954 ACTIVE  FLOW 192.168.100.6[49153]/17 192.168.100.100[3784]
536870955 ACTIVE  FLOW 192.168.100.100[3784]/17 192.168.100.6[49153]
536870952 ACTIVE  FLOW 192.168.101.6[49156]/17 192.168.101.100[3784]
536870951 ACTIVE  FLOW 192.168.100.101[3784]/17 192.168.100.6[49154]
536870960 OPENING FLOW fe80:0:0:0:20c:29ff:fe85:3442[133]/58 ff02:0:0:0:0:0:0:2[0]
536870957 ACTIVE  FLOW 192.168.101.101[3784]/17 192.168.101.6[49155]
536870959 ACTIVE  FLOW 192.168.250.100[22]/6 192.168.200.100[48706]
536870950 ACTIVE  FLOW 192.168.100.6[49154]/17 192.168.100.101[3784]
536870956 ACTIVE  FLOW 192.168.101.6[49155]/17 192.168.101.101[3784]
--------------------------------------------------------------------------------
Slot 6
--------------------------------------------------------------------------------
No Active Flows

Slot 6 with CN-DB pod in now in PREPARE state and CI link is down.

MP leader status: Leader
        Slot-id Type            CI-IP           TI-IP           State   CI-State        TI-State
        ========================================================================================
        1       CN-GW           192.168.23.100  192.168.24.80   UP      IMPACTED        UP
        10      CN-NGFW         192.168.23.81   192.168.24.82   UP      IMPACTED        UP
        2       CN-GW           192.168.23.101  192.168.24.100  UP      IMPACTED        UP
        5       CN-DB           192.168.23.102  ::              UP      IMPACTED        NA
        6       CN-DB           192.168.23.104  ::              PREPARE DOWN            NA
        7       CN-NGFW         192.168.23.103  192.168.24.86   UP      IMPACTED        UP
        8       CN-NGFW         192.168.23.105  192.168.24.84   UP      IMPACTED        UP
        9       CN-NGFW         192.168.23.82   192.168.24.81   UP      IMPACTED        UP

Enter

show cluster-membership show-slot-info slot all

until the CN-DB pod becomes active again.

MP leader status: Leader
        Slot-id Type            CI-IP           TI-IP           State   CI-State        TI-State
        ========================================================================================
        1       CN-GW           192.168.23.100  192.168.24.80   UP      UP              UP
        10      CN-NGFW         192.168.23.81   192.168.24.82   UP      UP              UP
        2       CN-GW           192.168.23.101  192.168.24.100  UP      UP              UP
        5       CN-DB           192.168.23.102  ::              UP      UP              NA
        6       CN-DB           192.168.23.104  ::              PROBE   UP              NA
        7       CN-NGFW         192.168.23.103  192.168.24.86   UP      UP              UP
        8       CN-NGFW         192.168.23.105  192.168.24.84   UP      UP              UP
        9       CN-NGFW         192.168.23.82   192.168.24.81   UP      UP              UP

Check the cluster traffic flow again using the command

show cluster-flow all

--------------------------------------------------------------------------------
Slot 5
--------------------------------------------------------------------------------
Id      State  Type Src[Sport]/Proto       Dst[Dport]
--------------------------------------------------------------------------------
536870953 ACTIVE  FLOW 192.168.101.100[3784]/17 192.168.101.6[49156]
536870958 ACTIVE  FLOW 192.168.200.100[48706]/6 192.168.250.100[22]
536870954 ACTIVE  FLOW 192.168.100.6[49153]/17 192.168.100.100[3784]
536870955 ACTIVE  FLOW 192.168.100.100[3784]/17 192.168.100.6[49153]
536870952 ACTIVE  FLOW 192.168.101.6[49156]/17 192.168.101.100[3784]
536870951 ACTIVE  FLOW 192.168.100.101[3784]/17 192.168.100.6[49154]
536870960 OPENING FLOW fe80:0:0:0:20c:29ff:fe85:3442[133]/58 ff02:0:0:0:0:0:0:2[0]
536870957 ACTIVE  FLOW 192.168.101.101[3784]/17 192.168.101.6[49155]
536870959 ACTIVE  FLOW 192.168.250.100[22]/6 192.168.200.100[48706]
536870950 ACTIVE  FLOW 192.168.100.6[49154]/17 192.168.100.101[3784]
536870956 ACTIVE  FLOW 192.168.101.6[49155]/17 192.168.101.101[3784]
--------------------------------------------------------------------------------
Slot 6
--------------------------------------------------------------------------------
Id      State  Type Src[Sport]/Proto       Dst[Dport]
--------------------------------------------------------------------------------
671088642 ACTIVE  FLOW 192.168.101.100[3784]/17 192.168.101.6[49156]
671088641 ACTIVE  FLOW 192.168.200.100[48706]/6 192.168.250.100[22]
671088643 ACTIVE  FLOW 192.168.100.6[49153]/17 192.168.100.100[3784]
671088645 ACTIVE  FLOW 192.168.100.100[3784]/17 192.168.100.6[49153]
671088644 ACTIVE  FLOW 192.168.101.6[49156]/17 192.168.101.100[3784]
671088646 ACTIVE  FLOW 192.168.100.101[3784]/17 192.168.100.6[49154]
671088647 ACTIVE  FLOW fe80:0:0:0:20c:29ff:fe85:3442[133]/58 ff02:0:0:0:0:0:0:2[0]
671088648 ACTIVE  FLOW 192.168.101.101[3784]/17 192.168.101.6[49155]
671088649 ACTIVE  FLOW 192.168.250.100[22]/6 192.168.200.100[48706]
671088650 ACTIVE  FLOW 192.168.100.6[49154]/17 192.168.100.101[3784]
671088651 ACTIVE  FLOW 192.168.101.6[49155]/17 192.168.101.101[3784]

show cluster-flow all filter count yes

--------------------------------------------------------------------------------
Slot 5
--------------------------------------------------------------------------------
Number of sessions that match filter: 11
--------------------------------------------------------------------------------
Slot 6
--------------------------------------------------------------------------------
Number of sessions that match filter: 11

show cluster-membership show-slot-info slot all

MP leader status: Leader
        Slot-id Type            CI-IP           TI-IP           State   CI-State        TI-State
        ========================================================================================
        1       CN-GW           192.168.23.100  192.168.24.80   UP      UP              UP
        10      CN-NGFW         192.168.23.81   192.168.24.82   UP      UP              UP
        2       CN-GW           192.168.23.101  192.168.24.100  UP      UP              UP
        5       CN-DB           192.168.23.102  ::              UP      UP              NA
        6       CN-DB           192.168.23.104  ::              UP      UP              NA
        7       CN-NGFW         192.168.23.103  192.168.24.86   UP      UP              UP
        8       CN-NGFW         192.168.23.105  192.168.24.84   UP      UP              UP
        9       CN-NGFW         192.168.23.82   192.168.24.81   UP      UP              UP

From Panorama CLI

show clusters name cluster-001

Cluster: cluster-001
Creation time: 2022/11/22 05:11:09
CN-MGMT pods:
8FF0233D36BD57D (active, pan-mgmt-sts-1.cluster-001, connected, In Sync)
8F846238B0740D2 (pan-mgmt-sts-0.cluster-001, connected, In Sync)


Slot-ID         PodName                             Type                      Version
----------------------------------------------------------------------------------------
5               pan-db-dep-7b6f6c5458-5fgnr         CN-DB                11.0.1-c156.dev_e_rel
1               pan-gw-dep-748cdb856d-4f66g         CN-GW                11.0.1-c156.dev_e_rel
2               pan-gw-dep-748cdb856d-p5qdd         CN-GW                11.0.1-c156.dev_e_rel
7               pan-ngfw-dep-56cdfdd656-srmdt       CN-NGFW              11.0.1-c156.dev_e_rel
8               pan-ngfw-dep-56cdfdd656-hvcw2       CN-NGFW              11.0.1-c156.dev_e_rel
9               pan-ngfw-dep-56cdfdd656-bjtmd       CN-NGFW              11.0.1-c156.dev_e_rel
10              pan-ngfw-dep-56cdfdd656-6jq2f       CN-NGFW              11.0.1-c156.dev_e_rel
6               pan-db-dep-7b6f6c5458-r449b         CN-DB                11.0.1-c156.dev_e_rel

You can view the CN-DB changes in the Panorama web interface under

Monitor

Logs

System

Results:

No impact to existing or new sessions. Cluster membership updated on Panorama.

Test Case: CN-NGFW Failure Handling

Features Not Supported on the CN-Series

CN-Series Firewall Docs

Test Case: CN-DB Failure Handling

CN-Series Firewall Docs

Test Case: CN-DB Failure Handling

Recommended For You

Activation & Onboarding

Next-Generation Firewalls

Cloud-Delivered Security Services

Best Practices

Experts Corner