Focus

CN-Series Deployment Guide

Test Case: CN-MGMT Failure Handling

Table of Contents

Test Case: CN-MGMT Failure Handling

This test evaluates CN-NGMT failure handling.

The desired minimum number of CN-MGMT pods for a CN-Series HSF deployment is two to ensure failure handling. After deployment, the CN-MGMT pod that becomes active first becomes the Leader and the second CN-MGMT becomes the Follower. Both CN-MGMT pods have the same configuration. At any instance, one CN-MGMT pod is in READY state. CN-DB, CN-GW, and CN-NGFW pods connect to the CN-MGMT pod in READY state through Traffic Interconnect (TI) links.

The two CN-MGMT pods are not in HA Active-Passive or HA Active-Active mode. Both pods have the same configuration, and is configured using Panorama.

The CN-MGMT pod failure happens due to one of the following conditions.

Liveness check fails

slotd

is down

ipsec

strongswan

is down

CN-MGMT pod crashes and restarts

From the Panorama CLI, enter

show clusters name <cluster-name>

to view the Leader and Follower CN-MGMT pod.

The following output shows that the

pan-mgmt-sts-1

pod is active.

Cluster: cluster-001
Creation time: 2022/11/30 03:23:50
CN-MGMT pods:
88C00D31E1FC86B (pan-mgmt-sts-0.cluster-001, connected, In Sync)
84CC9A394B3E196 (active, pan-mgmt-sts-1.cluster-001, connected, In Sync)


Slot-ID         PodName                             Type                      Version
----------------------------------------------------------------------------------------
5               pan-db-dep-6774cd774d-k49cm         CN-DB                11.0.1-c183.dev_e_rel
1               pan-gw-dep-d849c7df8-4sk54          CN-GW                11.0.1-c183.dev_e_rel
6               pan-ngfw-dep-668965d598-pnthb       CN-NGFW              11.0.1-c183.dev_e_rel
8               pan-ngfw-dep-668965d598-s2zcc       CN-NGFW              11.0.1-c183.dev_e_rel
7               pan-ngfw-dep-668965d598-vf9l4       CN-NGFW              11.0.1-c183.dev_e_rel
9               pan-ngfw-dep-668965d598-pmmjd       CN-NGFW              11.0.1-c183.dev_e_rel
10              pan-db-dep-6774cd774d-gjpkr         CN-DB                11.0.1-c183.dev_e_rel
2               pan-gw-dep-d849c7df8-ct6wk          CN-GW                11.0.1-c183.dev_e_rel

View cluster membership for the

pan-mgmt-sts-1

pod and state of CN-DB, CN-GW, and CN-NGFW pods from the Kubernetes controller CLI.

Enter

kubectl get pods -n kube-system

to view the state of all pods.

Output:

The

pan-mgmt-sts-1

is Active. All CN-DB, CN-GW, and CN-NGFW pods are connected to

pan-mgmt-sts-1

NAME                            READY   STATUS    RESTARTS   AGE
pan-db-dep-6774cd774d-gjpkr     1/1     Running   0          69m
pan-db-dep-6774cd774d-k49cm     1/1     Running   0          69m
pan-gw-dep-d849c7df8-4sk54      1/1     Running   0          69m
pan-gw-dep-d849c7df8-ct6wk      1/1     Running   0          69m
pan-mgmt-sts-0                  0/1     Running   0          83m
pan-mgmt-sts-1                  1/1     Running   0          83m
pan-ngfw-dep-668965d598-pmmjd   1/1     Running   0          69m
pan-ngfw-dep-668965d598-pnthb   1/1     Running   0          69m
pan-ngfw-dep-668965d598-s2zcc   1/1     Running   0          69m
pan-ngfw-dep-668965d598-vf9l4   1/1     Running   0          69m

Check cluster membership from

pan-mgmt-sts-1

Get in to the

pan-mgmt-sts-1

pod.

kubectl -n kube-system exec -it pan-mgmt-sts-1 -- bash

su - admin

Check if all CN-DB, CN-GW, and CN-NGFW pods are connected to the Leader CN-MGMT pod using the following command.

show cluster-membership show-slot-info slot all

Output:

MP leader status: Leader
        Slot-id Type            CI-IP           TI-IP           State   CI-State        TI-State
        ========================================================================================
        1       CN-GW           192.168.23.101  192.168.24.100  UP      UP              UP
        10      CN-DB           192.168.23.104  ::              UP      UP              NA
        2       CN-GW           192.168.23.100  192.168.24.98   UP      UP              UP
        5       CN-DB           192.168.23.102  ::              UP      UP              NA
        6       CN-NGFW         192.168.23.89   192.168.24.83   UP      UP              UP
        7       CN-NGFW         192.168.23.105  192.168.24.86   UP      UP              UP
        8       CN-NGFW         192.168.23.103  192.168.24.84   UP      UP              UP
        9       CN-NGFW         192.168.23.82   192.168.24.81   UP      UP              UP

Check cluster membership from

pan-mgmt-sts-0

Get in to the

pan-mgmt-sts-0

pod.

kubectl -n kube-system exec -it pan-mgmt-sts-0 -- bash

su - admin

Check if any CN-DB, CN-GW, and CN-NGFW pods are connected to the Follower CN-MGMT pod using the following command.

show cluster-membership show-slot-info slot all

Output:

No members info present

Test CN-MGMT pod failure handling.

From the Kubernetes controller CLI, enter the following command to delete the Leader

pan-mgmt-sts-1

pod.

kubectl -n kube-system delete pod pan-mgmt-sts-1

From the Panorama CLI, enter

show clusters name <cluster-name>

to view the new Leader and Follower CN-MGMT pod.

The following output shows that the

pan-mgmt-sts-0

pod is now active.

Cluster: cluster-001
Creation time: 2022/11/30 03:23:50
CN-MGMT pods:
88C00D31E1FC86B (active, pan-mgmt-sts-0.cluster-001, connected, In Sync)
84CC9A394B3E196 (pan-mgmt-sts-1.cluster-001, connected, In Sync)


Slot-ID         PodName                             Type                      Version
----------------------------------------------------------------------------------------
5               pan-db-dep-6774cd774d-k49cm         CN-DB                11.0.1-c183.dev_e_rel
1               pan-gw-dep-d849c7df8-4sk54          CN-GW                11.0.1-c183.dev_e_rel
6               pan-ngfw-dep-668965d598-pnthb       CN-NGFW              11.0.1-c183.dev_e_rel
8               pan-ngfw-dep-668965d598-s2zcc       CN-NGFW              11.0.1-c183.dev_e_rel
7               pan-ngfw-dep-668965d598-vf9l4       CN-NGFW              11.0.1-c183.dev_e_rel
9               pan-ngfw-dep-668965d598-pmmjd       CN-NGFW              11.0.1-c183.dev_e_rel
10              pan-db-dep-6774cd774d-gjpkr         CN-DB                11.0.1-c183.dev_e_rel
2               pan-gw-dep-d849c7df8-ct6wk          CN-GW                11.0.1-c183.dev_e_rel

View cluster membership for the

pan-mgmt-sts-0

pod and state of CN-DB, CN-GW, and CN-NGFW pods from the Kubernetes controller CLI.

Enter

kubectl get pods -n kube-system

to view the state of all pods.

Output:

The

pan-mgmt-sts-0

is Active. All CN-DB, CN-GW, and CN-NGFW pods are connected to

pan-mgmt-sts-1

NAME                            READY   STATUS    RESTARTS   AGE
pan-db-dep-6774cd774d-gjpkr     1/1     Running   0          76m
pan-db-dep-6774cd774d-k49cm     1/1     Running   0          76m
pan-gw-dep-d849c7df8-4sk54      1/1     Running   0          76m
pan-gw-dep-d849c7df8-ct6wk      1/1     Running   0          76m
pan-mgmt-sts-0                  1/1     Running   0          90m
pan-mgmt-sts-1                  0/1     Running   0          90m
pan-ngfw-dep-668965d598-pmmjd   1/1     Running   0          76m
pan-ngfw-dep-668965d598-pnthb   1/1     Running   0          76m
pan-ngfw-dep-668965d598-s2zcc   1/1     Running   0          76m
pan-ngfw-dep-668965d598-vf9l4   1/1     Running   0          76m

Check cluster membership from

pan-mgmt-sts-0

Get in to the

pan-mgmt-sts-0

pod.

kubectl -n kube-system exec -it pan-mgmt-sts-0 -- bash

su - admin

Check if all CN-DB, CN-GW, and CN-NGFW pods are connected to the Leader CN-MGMT pod using the following command.

show cluster-membership show-slot-info slot all

Output:

MP leader status: Leader
        Slot-id Type            CI-IP           TI-IP           State   CI-State        TI-State
        ========================================================================================
        1       CN-GW           192.168.23.101  192.168.24.100  UP      UP              UP
        10      CN-DB           192.168.23.104  ::              UP      UP              NA
        2       CN-GW           192.168.23.100  192.168.24.98   UP      UP              UP
        5       CN-DB           192.168.23.102  ::              UP      UP              NA
        6       CN-NGFW         192.168.23.89   192.168.24.83   UP      UP              UP
        7       CN-NGFW         192.168.23.105  192.168.24.86   UP      UP              UP
        8       CN-NGFW         192.168.23.103  192.168.24.84   UP      UP              UP
        9       CN-NGFW         192.168.23.82   192.168.24.81   UP      UP              UP

Check cluster membership from

pan-mgmt-sts-1

Get in to the

pan-mgmt-sts-1

pod.

kubectl -n kube-system exec -it pan-mgmt-sts-1 -- bash

su - admin

Check if any CN-DB, CN-GW, and CN-NGFW pods are connected to the Follower CN-MGMT pod using the following command.

show cluster-membership show-slot-info slot all

Output:

No members info present

Test Result:
When the Leader pod

pan-mgmt-sts-1

fails, the Follower pod

pan-mgmt-sts-0

becomes the new Leader. This CN-MGMT failure handling mechanism ensures that traffic flow is uninterrupted. No impact to existing or new sessions.

Scale Out Firewalls Based on Custom Metrics Supported

Test Case: CN-NGFW Failure Handling

CN-Series Deployment Guide

Test Case: CN-MGMT Failure Handling

Test Case: CN-MGMT Failure Handling

Recommended For You