Upgrade the CN-Series Firewall—Rolling Update
Focus
Focus
CN-Series

Upgrade the CN-Series Firewall—Rolling Update

Table of Contents

Upgrade the CN-Series Firewall—Rolling Update

Deploy an additional CN-MGMT statefulset to complete a rolling update of the CN-NGFW pods.
Where Can I Use This?What Do I Need?
  • CN-Series upgrade
  • CN-Series deployment
  • CN-Series 10.1.x or above Container Images
  • Panorama running PAN-OS 10.1.x or above version
  • Helm 3.6 or above version client
Use one of the following options to perform a rolling update, upgrade or downgrade to a supported PAN-OS version.
Before you begin, ensure the CN-Series YAML file version is compatible with the PAN-OS version.
  • PAN-OS 10.1.2 or later requires YAML 2.0.2
  • PAN-OS 10.1.0 and 10.1.1 require YAML 2.0.0 or 2.0.1

Rolling Update

This process enables you to first upgrade the CN-MGMT StatefulSet and then upgrade the CN-NGFW pods. The disruption to application traffic is minimal because the CN-NGFW pods are functioning during the CN-MGMT StatefulSet upgrade, and the rolling update for the CN-NGFW pods occurs one instance of the CN-NGFW pod at a time.
If you have a large Kubernetes cluster with a significant number of CN-NGFW pods and want a faster upgrade, you can schedule a maintenance window to delete the CN-NGFW yaml and upgrade all CN-NGFW pods at once.
During the CN-MGMT upgrade, logging is impacted. Additionally, both kubectl logs and System log messages are generated for temporary version mismatch and connection restarts between the CN-NGFW and the CN-MGMT pods.
  1. Upgrade the CN-MGMT StatefulSet.
    1. Use one of the following options.
      • Option 1— Update the image name in the pan-cn-mgmt.yaml and apply the changes.
        containers: - name: pan-mgmt image:<your-private-registry-image-path-new-image>
        kubectl apply -f pan-cn-mgmt.yaml
      • Option 2—Use kubectl. When using kubectl, you are not updating the yaml files and therefore must keep track of the image used for the upgrade.
        kubectl -n kube-system set image sts/pan-mgmt-sts pan-mgmt=<your-private-registry-image-path-new-image>
    2. Verify that the CN-MGMT StatefulSet is deployed.
      1. Use kubectl -n kube-system get sts/pan-mgmt-sts -o wide
      2. Check the status of the upgrade.
        kubectl exec -it pan-mgmt-sts-0 -n kube-system -- su admin
        admin@pan-mgmt-sts-0> show jobs all
        admin@pan-mgmt-sts-0.Basc-cluster-180> show jobs all Enqueued Dequeued ID PositionInQ Type Status Result Completed ------------------------------------------------------------------------------------------------------------------------------------------ 2020/08/25 14:11:11 14:11:11 2 AutoCom FIN OK 14:11:44
  2. Upgrade the CN-NGFW pods.
    1. Use one of the following options.
      • Option1— Update the image name in the pan-cn-ngfw.yaml and apply the changes.
        containers: - name: pan-ngfw-container image:<your-private-registry-image-path-new-image>
        kubectl apply -f pan-cn-ngfw.yaml
      • Option 2—Use kubectl. When using kubectl, you are not updating the yaml files and therefore must keep track of the image used for the upgrade.
        • In CN-Series as a DaemonSet deployment, use:
          kubectl -n kube-system set image ds/pan-ngfw-ds pan-ngfw-container=<your-private-registry-image-path-new-image>
        • In CN-Series as a Kubernetes Service deployment, use:
          kubectl -n kube-system set image deployment/pan-ngfw-dep pan-ngfw-container=<your-private-registry-image-path-new-image>
    2. Check the status of the upgrade.
      Use kubectl -n kube-system get ds/pan-ngfw-ds -o wide
      In a CN-Series as a Kubernetes Service deployment, use kubectl -n kube-system get deployment/pan-ngfw-dep -o wide
  3. Required only if the images are updated for the PAN-OS version Update the init container and pan-cni images.
    1. Modify the Init container image in the pan-cn-mgmt.yaml for the CN-MGMT firewall.
      initContainers: - name: pan-mgmt-init image:<your-private-registry-image-path>
    2. Edit the image path for the PAN-CNI container image in the pan-cni.yaml.
      containers: name: install-pan-cni image:<your-private-registry-image-path>

Rolling Update with Additional CN-MGMT StatefulSet

  1. Before you begin.
    1. Verify that the nodes in your cluster have the memory and CPU resources required for the additional CN-MGMT StatefulSet.
    2. (Required for statically provisioned PVs only) Verify that you have PVs available for the additional CN-MGMT StatefulSet.
      The pan-cn-pv-local.yaml creates the directories required to deploy the CN-MGMT.
  2. Set up the new pan-cn-mgmt-configmap.yaml.
    Edit the PAN_SERVICE_NAME: value to match what you added above in the new pan-cn-mgmt.yaml.
    apiVersion: v1
    kind: ConfigMap
    metadata:
    name: pan-ngfw-config
    namespace: kube-system
    data:
    PAN_SERVICE_NAME: pan-mgmt-svc2
    Retain the same values for the # Panorama settings and # Intended License Bundle type in the new file to reduce any updates on Panorama.
  3. Set up the new pan-cn-mgmt.yaml file.
    There are multiple places you need to replace the service names, apps, and labels. See Compare the Old and New PAN-CN-MGMT.yaml.
  4. Required only if the images are updated for the PAN-OS version Update the Init container and pan-cni images
    Image path for the Init container image in the pan-cn-mgmt.yaml for the CN-MGMT firewall
    initContainers: - name: pan-mgmt-init image: <your-private-registry-image-path>
    Image path for the PAN-CNI container image that has the CNI binaries and the CNI network config file on each node.
    containers: name: install-pan-cni image: <your-private-registry-image-path>
  5. Apply the new CN-MGMT yaml files.
    kubectl apply -f pan-cn-mgmt-configmap-new.yaml
    kubectl apply -f pan-cn-mgmt-new.yaml
  6. Verify that the new CN-MGMT StatefulSet is deployed.
    kubectl -n kube-system get sts -o wide
    NAME READY AGE CONTAINERS IMAGES
    pan-mgmt-sts 2/2 16h pan-mgmt 018147215560.dkr.ecr.ap-southeast-1.amazonaws.com/test/panos_ctnr/10.0.0/b/mp:63
    pan-mgmt-sts-new 2/2 50m pan-mgmt-new 018147215560.dkr.ecr.ap-southeast-1.amazonaws.com/test/panos_ctnr/10.0.1/b/mp:64
  7. Edit the CN-NGFW pod yaml files with the new service name.
    1. Update the pan-cn-ngfw-configmap.yaml.
      When you modify the PAN_SERVICE_NAME: value referenced in the pan-cn-ngfw-configmap.yaml to match the value you defined in the pan-cn-mgmt.yaml Service name, the pods that use the new image will connect to the new StatefulSet.
      apiVersion: v1
      kind: ConfigMap
      metadata:name: pan-ngfw-config
      namespace: kube-system
      data:
      PAN_SERVICE_NAME: pan-mgmt-svc2
    2. Deploy the pan-cn-ngfw-configmap.yaml
      kubectl apply -f pan-cn-ngfw-configmap.yaml
    3. Edit the image path referenced in the pan-cn-ngfw.yaml.
      For example, you can use kubectl set image ds/pan-cn-ngfw-ds -n kube-system pan-ngfw-container=018147215560.dkr.ecr.ap-southeast-1.amazonaws.com/test/panos_ctnr/10.0.2/b/dp:62
    4. Check the status of the rolling update.
      UP-TO-DATE column displays the number of replicas that have been updated successfully.
      kubectl get ds/pan-ngfw-ds -n kube-system -o wide
      NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE CONTAINERS IMAGES SELECTOR
      pan-ngfw 4 4 3 1 3 <none> 16h pan-ngfw-container 018147215560.dkr.ecr.ap-southeast-1.amazonaws.com/test/panos_ctnr/10.0.0/b/dp:22 app=pan-ngfw
  8. Verify that the CN-NGFW pods are deployed.
    kubectl -n kube-system get pods -l app=pan-ngfw
    NAME               READY STATUS RESTARTS AGE
    pan-ngfw-ds-8b5gp 1/1 Running 0       40m
    pan-ngfw-ds-h8xc6 1/1 Running 0        40m
    pan-ngfw-ds-sn62b 1/1 Running 0       40m
    pan-ngfw-ds-vxfqp 1/1 Running 0       40m
  9. Get the Serial Number for the CN-MGMT pods.
    kubectl exec -it pan-mgmt-sts-0 -n kube-system -- su admin
    Warning: Your device is still configured with the default admin account credentials. Please change your password prior to deployment.admin@pan-mgmt-sts-0>
  10. Install the dynamic content updates for the subscriptions you have purchased.
    You can either install it manually or set up a schedule. Verify the serial numbers of the CN-MGMT pods when selecting them for the dynamic updates.
    or on a recurring schedule.

Compare the Old and New PAN-CN-MGMT.yaml

Review the different places where you need to update the service names, apps, and labels within the yaml file when you deploy a new CN-MGMT StatefulSet.
OldNew
apiVersion: v1
kind: Service
metadata:
name: pan-mgmt-svc
apiVersion: v1
kind: Service
metadata:
name: pan-mgmt-svc2
namespace: kube-system
labels:
app: pan-mgmt-svc
namespace: kube-system
labels:
app: pan-mgmt-svc2
spec:
ports:
- protocol: UDP
port: 4500
name: ipsec
selector:
appname: pan-mgmt-sts
spec:
ports:
- protocol: UDP
port: 4500
name: ipsec
selector:
appname: pan-mgmt-sts-new
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: pan-mgmt-sts
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: pan-mgmt-sts-new
namespace: kube-system
spec:
selector:
matchLabels:
appname: pan-mgmt-sts
serviceName: pan-mgmt-svc
# Replicas are for fault-tolerance. Max 2 replicas supported.
replicas: 2
updateStrategy:
type: RollingUpdate
podManagementPolicy: Parallel
template:
metadata:
labels:
app: pan-mgmt
appname: pan-mgmt-sts
namespace: kube-system
spec:
selector:
matchLabels:
appname: pan-mgmt-sts-new
serviceName: pan-mgmt-svc2
# Replicas are for fault-tolerance. Max 2 replicas supported.
replicas: 2
updateStrategy:
type: RollingUpdate
podManagementPolicy: Parallel
template:
metadata:
labels:
app: pan-mgmt
appname: pan-mgmt-sts-new
labelSelector:
matchExpressions:- key: "appname"
operator: In
values:
pan-mgmt-sts
labelSelector:
matchExpressions:- key: "appname"
operator: In
values:
pan-mgmt-sts-new
topologyKey: "kubernetes.io/hostname"
initContainers:
- name: pan-mgmt-init
mountPath: /var/log/pan/
envFrom:
configMapRef:
name: pan-mgmt-config
topologyKey: "kubernetes.io/hostname"
initContainers:
- name: pan-mgmt-init
mountPath: /var/log/pan/
envFrom:
configMapRef:
name: pan-mgmt-new-config
# sw-secret in pan-cn-ngfw.yaml and hard-coded in ipsec.conf
value: pan-fw
containers:
name: pan-mgmt
image: 018147215560.dkr.ecr.ap-southeast-1.amazonaws.com/test/panos_ctnr/10.0.0/b/mp:63
# sw-secret in pan-cn-ngfw.yaml and hard-coded in ipsec.conf
value: pan-fw
containers:
name: pan-mgmt-new
image: 018147215560.dkr.ecr.ap-southeast-1.amazonaws.com/test/panos_ctnr/10.0.1/b/mp:64
volumes
name: dshm
envFrom:
configMapRef:
name: pan-mgmt-config
volumes
name: dshm
envFrom:
configMapRef:
name: pan-mgmt-new-config