Kubernetes/OpenShift clusters
Overview
Microsegmentation provides a close integration with Kubernetes and OpenShift to make it easy to control and monitor clusters composed of Linux hosts.
You can use either of the following methods to deploy the enforcer DaemonSet.
Either procedure deploys the enforcer to a cluster in approximately five minutes.
Using a YAML file
Prerequisites
- Logged into apoctl with the namespace.administrator role
- Unintercepted TLS connection from the nodes to the Microsegmentation API (enforcer pods use an app credential to authenticate)
Deploying the enforcers
- The enforcer requires a CNI plugin.Most Kubernetes and OpenShift clusters use CNI plugins by default, but GKE and AKS do not. GKE and AKS default to kubenet. Before deploying the enforcer to a GKE or AKS cluster, configure them to use CNI as follows.
- GKE: pass the --enable-network-policy flag to use CNI.Update existing clustergcloud container clusters update $CLUSTER-NAME --update-addons=NetworkPolicy=ENABLED gcloud container clusters update $CLUSTER-NAME --enable-network-policyCreate new clustergcloud container clusters create $CLUSTER-NAME --enable-network-policyAKS: pass the --network-plugin azure flag at creation to use CNI.az aks create --name $CLUSTER-NAME --resource-group $RESOURCE_GROUP --generate-ssh-keys --network-plugin azure
- Set a CLUSTER_NS environment variable identifying the Microsegmentation namespace for this cluster. This should be a grandchild namespace.export CLUSTER_NS=/803920923337065472/aws-dev-826088932159/k8sTo generate a YAML file that deploys the enforcers to your current context, use one of the following commands.EKSapoctl enforcer install kubernetes --installation-mode yaml \ --cluster-type eks \ --namespace $CLUSTER_NS \ --api $MICROSEG_API \ --repo https://charts.aporeto.com{version}/clients \ --set imageRegistry=gcr.io/prismacloud-cnsGKEapoctl enforcer install kubernetes --installation-mode yaml \ --cluster-type gke \ --namespace $CLUSTER_NS \ --api $MICROSEG_API \ --repo https://charts.aporeto.com{version}/clients \ --set imageRegistry=gcr.io/prismacloud-cnsAKSapoctl enforcer install kubernetes --installation-mode yaml \ --cluster-type aks \ --namespace $CLUSTER_NS \ --api $MICROSEG_API \ --repo https://charts.aporeto.com{version}/clients \ --set imageRegistry=gcr.io/prismacloud-cnskopsapoctl enforcer install kubernetes --installation-mode yaml \ --cluster-type custom \ --custom-cni-chained \ --custom-cni-bin-dir /opt/cni/bin \ --namespace $CLUSTER_NS \ --api $MICROSEG_API \ --repo https://charts.aporeto.com{version}/clients \ --set imageRegistry=gcr.io/prismacloud-cnsOther Kubernetesapoctl enforcer install kubernetes --installation-mode yaml \ --cluster-type custom \ --namespace $CLUSTER_NS \ --api $MICROSEG_API \ --repo https://charts.aporeto.com{version}/clients \ --set imageRegistry=gcr.io/prismacloud-cnsOpenShiftapoctl enforcer install kubernetes --installation-mode yaml \ --cluster-type ocp4 \ --enable-openshift \ --namespace $CLUSTER_NS \ --api $MICROSEG_API \ --repo https://charts.aporeto.com{version}/clients \ --set imageRegistry=gcr.io/prismacloud-cnsWe detail the apoctl enforcer install kubernetes command further in the reference documentation. You can also run apoctl enforcer install kubernetes -h to review its flags. To learn how to customize your deployment, refer to Enforcer configuration options at the bottom of this page.Review the generated YAML.cat enforcerd.yamlApply the YAML to your cluster.kubectl apply -f enforcerd.yamlTo confirm your deployment, issue the following command.watch kubectl get pods --all-namespacesapoctl api list enforcers --namespace $CLUSTER_NS \ -o table \ -c ID \ -c name \ -c namespace \ -c operationalStatusapoctl should return a list of the enforcers deployed. You should see an enforcer instance on each agent node. An example for a three-node GKE cluster follows.ID | name | namespace | operationalStatus ---------------------------+-------------------------------------------+----------------------------------+-------------------- 5f74d837f0fe170703c10d6b | gke-aws-dev-01-default-pool-cf284cf1-5bqn | /803920923337065472/aws-dev-826088932159/k8s | Connected 5f74d836f0fe170703c10d6a | gke-aws-dev-01-default-pool-cf284cf1-5pjs | /803920923337065472/aws-dev-826088932159/k8s | Connected 5f74d836f0fe170703c10d69 | gke-aws-dev-01-default-pool-cf284cf1-cqrd | /803920923337065472/aws-dev-826088932159/k8s | ConnectedAll enforcer instances should have an operationalStatus of Connected.Open theNetwork Securitysection of the Prisma Cloud web interface, navigate to the enforcer’s namespace, and selectunderAgentManage. You should find your enforcers listed with the statusconnected. Click the enforcers to review their Microsegmentation tags.SelectApp Dependency Mapin the side navigation menu. If your cluster contains pods outside of the kube-system namespace, you should see them with dashed green lines to a Somewhere external network. Your cluster is in discovery mode. Refer to Securing a Kubernetes namespace to learn how to allow the desired traffic and disable discovery mode.To see the pods and their traffic in theApp Dependency Mappane, you may need to toggleRecursiveto on.Using a Helm chartPrerequisites
- Logged into apoctl with the namespace.administrator role
- Unintercepted TLS connection from the nodes to the Microsegmentation API (enforcer pods use an app credential to authenticate)
Deploying the enforcers- The enforcer requires a CNI plugin.Most Kubernetes and OpenShift clusters use CNI plugins by default, but GKE and AKS do not. GKE and AKS default to kubenet. Before deploying the enforcer to a GKE or AKS cluster, configure them to use CNI as follows.
- GKE: pass the --enable-network-policy flag to use CNI.Update existing clustergcloud container clusters update $CLUSTER-NAME --update-addons=NetworkPolicy=ENABLED gcloud container clusters update $CLUSTER-NAME --enable-network-policyCreate new clustergcloud container clusters create $CLUSTER-NAME --enable-network-policyAKS: pass the --network-plugin azure flag at creation to use CNI.az aks create --name $CLUSTER-NAME --resource-group $RESOURCE_GROUP --generate-ssh-keys --network-plugin azure
- Set a CLUSTER_NS environment variable identifying the Microsegmentation namespace for this cluster. This should be a grandchild namespace.export CLUSTER_NS=/803920923337065472/aws-dev-826088932159/k8sTo generate a Helm chart that deploys the enforcers to your current context, use one of the following commands.EKSapoctl enforcer install kubernetes --installation-mode helm \ --cluster-type eks \ --namespace $CLUSTER_NS \ --api $MICROSEG_API \ --repo https://charts.aporeto.com{version}/clients \ --set imageRegistry=gcr.io/prismacloud-cnsGKEapoctl enforcer install kubernetes --installation-mode helm \ --cluster-type gke \ --namespace $CLUSTER_NS \ --api $MICROSEG_API \ --repo https://charts.aporeto.com{version}/clients \ --set imageRegistry=gcr.io/prismacloud-cnsAKSapoctl enforcer install kubernetes --installation-mode helm \ --cluster-type aks \ --namespace $CLUSTER_NS \ --api $MICROSEG_API \ --repo https://charts.aporeto.com{version}/clients \ --set imageRegistry=gcr.io/prismacloud-cnskopsapoctl enforcer install kubernetes --installation-mode helm \ --cluster-type custom \ --custom-cni-chained \ --custom-cni-bin-dir /opt/cni/bin \ --namespace $CLUSTER_NS \ --api $MICROSEG_API \ --repo https://charts.aporeto.com{version}/clients \ --set imageRegistry=gcr.io/prismacloud-cnsOther Kubernetesapoctl enforcer install kubernetes --installation-mode helm \ --cluster-type custom \ --namespace $CLUSTER_NS \ --api $MICROSEG_API \ --repo https://charts.aporeto.com{version}/clients \ --set imageRegistry=gcr.io/prismacloud-cnsOpenShiftapoctl enforcer install kubernetes --installation-mode helm \ --cluster-type ocp4 \ --enable-openshift \ --namespace $CLUSTER_NS \ --api $MICROSEG_API \ --repo https://charts.aporeto.com{version}/clients \ --set imageRegistry=gcr.io/prismacloud-cnsWe detail the apoctl enforcer install kubernetes command further in the reference documentation. You can also run apoctl enforcer install kubernetes -h to review its flags. To learn how to customize your deployment, refer to Enforcer configuration options at the bottom of this page.Confirm the Helm chart creation.ls enforcerdCreate an aporeto namespace.kubectl create namespace aporetoUse the Helm chart to deploy the enforcers to your cluster.helm install enforcerd ./enforcerd --namespace aporetoTo confirm your deployment, issue the following command.watch kubectl get pods --all-namespacesapoctl api list enforcers --namespace $CLUSTER_NS \ -o table \ -c ID \ -c name \ -c namespace \ -c operationalStatusapoctl should return a list of the enforcers deployed. You should see an enforcer instance on each agent node. An example for a three-node GKE cluster follows.ID | name | namespace | operationalStatus ---------------------------+-------------------------------------------+----------------------------------+-------------------- 5f74d837f0fe170703c10d6b | gke-aws-dev-01-default-pool-cf284cf1-5bqn | /803920923337065472/aws-dev-826088932159/k8s | Connected 5f74d836f0fe170703c10d6a | gke-aws-dev-01-default-pool-cf284cf1-5pjs | /803920923337065472/aws-dev-826088932159/k8s | Connected 5f74d836f0fe170703c10d69 | gke-aws-dev-01-default-pool-cf284cf1-cqrd | /803920923337065472/aws-dev-826088932159/k8s | ConnectedAll enforcer instances should have an operationalStatus of Connected.Open theNetwork Securitysection of the Prisma Cloud web interface, navigate to the enforcer’s namespace, and selectunderAgentManage. You should find your enforcers listed with the statusconnected. Click the enforcers to review their Microsegmentation tags.SelectApp Dependency Mapin the side navigation menu. If your cluster contains pods outside of the kube-system namespace, you should see them with dashed green lines to a Somewhere external network. Your cluster is in discovery mode. Refer to Securing a Kubernetes namespace to learn how to allow the desired traffic and disable discovery mode.To see the pods and their traffic in theApp Dependency Mappane, you may need to toggleRecursiveto on.Enforcer configuration optionsThe enforcer exposes the following configuration options. You can pass these to the apoctl enforcer install command using the --raw-flags flag. Example: --raw-flags "--log-level=debug --log-format=human --log-to-console=true" You can also modify the enforcer’s configuration after install via kubectl edit daemonset enforcerd -n aporeto, adding the flags as arguments to the container. The enforcer pods will restart. An example follows.... spec: containers: - args: - --log-level=debug - --log-format=human ...FlagDescriptionPass this flag if you wish to recognize the Microsegmentation Console as a processing unit, allowing its communications to be monitored and controlled. By default, the enforcer ignores them.Pass this flag if you wish to recognize containers in the kube-system namespace as processing units, allowing their communications to be monitored and controlled. By default, the enforcer ignores them.Pass this flag if you wish to recognize containers in Kubernetes namespaces starting with openshift- as processing units, allowing their communications to be monitored and controlled. By default, the enforcer ignores them.The URL of the Microsegmentation Console API.Path to CA certificate.Disables check on certificate signature as trusted.Path to application credentials.Start of the port range for ports used by the enforcer application proxy. Defaults to 20992. You may adjust this if you experience conflicts.The enforcer can determine if it is running in a cloud environment, such as AWS, GCP, or Azure. This is the maximum amount of time to wait for these internal probes to complete. Default is two seconds.Pass this flag to disable the enforcer DNS proxy, which allows policies to be written based on FQDN, in cases where an exact IP address may be unpredictable.DNS server address or CIDR that is observed by the enforcer DNS proxy. Defaults to 0.0.0.0/0.(Beta) Pass this flag to gain performance improvements by using extended Berkeley Packet Filter (eBPF) on systems that support it.(Beta) The enforcer ignores IPv6 communications by default. If you have IPv6 enabled and wish to monitor and control these connections, pass this flag.Enable the Kubernetes API server to access the Microsegmentation API using kubectl.Controls whether the enforcer’s logs are written to stdout. Boolean that defaults to false.The Microsegmentation namespace the enforcer should register in.Microsegmentation tag for this enforcer.Note: to modify after the enforcer has started, you must shut down the enforcer, delete the enforcer object in the Microsegmentation Console, and perform a fresh install.Microsegmentation token for the enforcer to use to register to the Microsegmentation Console.A persistent working directory with write, read, and execute permissions. Files such as logs are stored here. Defaults to /var/lib/enforcerd
