System Requirements

• Kernel Configuration—Enable the HUGETLBFS option in the host OS kernel.
• KNI and UIO/VFIO—Insert KNI and UIO/VFIO in the host OS kernel.
• Hugepages
  1. Reserve hugepages
     • Reserve hugepages during runtime, before the pods starts. Add the number of hugepages required to the nr_hugepages file in the /sys/kernel/ directory corresponding to a specific page size (in KB). For example, use the following command for a single-node system if 1024 of 2M pages are required.

       echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages

     • Reserve hugepages during boot. For example, to reserve hugepage of memory 4G as four 1G pages, the following options should be passed to the kernel.

       default_hugepagesz=1G hugepagesz=1G hugepages=4

  2. Use hugepages with DPDK—Create mount points for hugepages because PanOS 10.2 uses DPDK secondary processes.
     The following is a sample command to create hugepage of size 1 GB for DPDK use.

     mkdir /mnt/huge
     mount -t hugetlbfs pagesize=1GB /mnt/huge

  3. Restart the kubelet service on the host after enabling hugepages using the following command.

     sudo systemctl restart kubelet

  4. Check /sys/fs/cgroup/hugetlb/kubepods.slice/hugetlb.2MB.limit_in_bytes to make sure the size matches the hugepage size. If the size does not match the hugepage size, update the size using the following command.

     echo 2147483648 > /sys/fs/cgroup/hugetlb/kubepods.slice/hugetlb.2MB.limit_in_bytes
     

     In a pod, applications can allocate and consume pre-allocated hugepages of multiple sizes. Applications consume hugepages through container level resource requirements using resource name hugepages-<size>. For example, hugepages-2Mi or hugepages-1Gi.

     Unlike CPU or memory, hugepages do not support over commit.

     Privileged mode is enabled to access host device space. To list and bind network devices to a container, mount /sys to the container so that DPDK can access files within the directory.

     Following is a code snippet to enable hugepages on DPDK.

     requests:
         cpu: "1"
         memory: "4Gi"
         hugepages-2Mi: 4Gi
     limits:
         cpu: "1"
         memory: "4Gi"
         hugepages-2Mi: 4Gi
     volumeMounts:
         - mountPath: /sys
           name: sys
         - mountPath: /dev
           name: dev
         - mountPath: /dev/shm
           name: dshm
         - mountPath: /run/tmp
           name: hosttmp
         - mountPath: /etc/pan-fw-sw
           name: sw-secret
         envFrom:
         - configMapRef:
             name: pan-ngfw-config-0
         env:
             - name: CPU_REQUEST
               valueFrom:
                 resourceFieldRef:
                   containerName: pan-ngfw-container
                   resource: requests.cpu
             - name: CPU_LIMIT
               valueFrom:
                 resourceFieldRef:
                   containerName: pan-ngfw-container
                   resource: limits.cpu
             - name: MEMORY_REQUEST
               valueFrom:
                 resourceFieldRef:
                   containerName: pan-ngfw-container
                   resource: requests.memory
             - name: MEMORY_LIMIT
               valueFrom:
                 resourceFieldRef:
                   containerName: pan-ngfw-container
                   resource: limits.memory
             - name: MY_POD_UUID
               valueFrom:
                 fieldRef:
                   fieldPath: metadata.uid
             - name: MY_NODE_NAME
               valueFrom:
                 fieldRef:
                   fieldPath: spec.nodeName
             - name: MY_POD_NAME
               valueFrom:
                 fieldRef:
                   fieldPath: metadata.name
             - name: MY_POD_NAMESPACE
               valueFrom:
                 fieldRef:
                   fieldPath: metadata.namespace
             - name: MY_POD_SERVICE_ACCOUNT
               valueFrom:
                 fieldRef:
                   fieldPath: spec.serviceAccountName
             - name: MY_POD_IP
               valueFrom:
                 fieldRef:
                   fieldPath: status.podIP
     volumes:
         - name: sys
           hostPath:
             path: /sys
         - name: dev
           hostPath:
             path: /dev
         - name: hosttmp
           hostPath:
             path: /tmp/pan
         - name: dshm
           emptyDir:
             medium: Memory
         - name: sw-secret
           secret:
             secretName: pan-fw-sw

• NUMA and CPU Pinning—Multiple DPDK process cannot run on the same core because it causes memory pool cache corruption among other issues. The secondary processes are pinned to a different core. Use CPU pinning options in configmap to control secondary processes.
• Config and Pod Changes
  • Enable PAN_DATA_MODE: "dpdk" in pan-cn-ngfw-configmap-0.yaml and pan-cn-ngfw-configmap-1.yaml.
    DPDK is not the default mode for CN-Series-as-a-kubernetes-CNF.
  • Match the #HUGEPAGE_MEMORY_REQUEST parameter with the hugepage memory request in pan-cn-ngfw-configmap-0.yaml and pan-cn-ngfw-configmap-1.yaml.
    If the hugepage memory is not available, it defaults to MMAP.

• Set up DPDK on On-Premises Worker Nodes
• Set up DPDK on AWS EKS