Kubernetes/OpenShift clusters

Overview

Microsegmentation provides a close integration with Kubernetes and OpenShift to make it easy to control and monitor clusters composed of Linux hosts. You can use either of the following methods to deploy the enforcer DaemonSet.
Either procedure deploys the enforcer to a cluster in approximately five minutes.

Using a YAML file

Prerequisites

Before you begin

The enforcer [requires a CNI plugin](../reqs/#clusters). Most Kubernetes and OpenShift clusters use CNI plugins by default, but GKE and AKS do not. GKE and AKS default to kubenet. Before deploying the enforcer to a GKE or AKS cluster, configure them to use CNI as follows.

Deploying the enforcers

  1. Set a CLUSTER_NS environment variable identifying the Microsegmentation namespace for this cluster.
    This should be a grandchild namespace.
    export CLUSTER_NS=/acme/aws-dev/k8s
  2. To generate a YAML file that deploys the enforcers to your current context, use one of the following commands.
    EKS
    apoctl enforcer install kubernetes --installation-mode yaml \ --cluster-type eks \ --namespace $CLUSTER_NS \ --api $MICROSEG_API \ --repo https://charts.aporeto.com/releases/release-5.0.12/clients \ --set imageRegistry=gcr.io/prismacloud-cns
    GKE
    apoctl enforcer install kubernetes --installation-mode yaml \ --cluster-type gke \ --namespace $CLUSTER_NS \ --api $MICROSEG_API \ --repo https://charts.aporeto.com/releases/release-5.0.12/clients \ --set imageRegistry=gcr.io/prismacloud-cns
    AKS
    apoctl enforcer install kubernetes --installation-mode yaml \ --cluster-type aks \ --namespace $CLUSTER_NS \ --api $MICROSEG_API \ --repo https://charts.aporeto.com/releases/release-5.0.12/clients \ --set imageRegistry=gcr.io/prismacloud-cns
    kops
    apoctl enforcer install kubernetes --installation-mode yaml \ --cluster-type custom \ --custom-cni-chained \ --custom-cni-bin-dir /opt/cni/bin \ --namespace $CLUSTER_NS \ --api $MICROSEG_API \ --repo https://charts.aporeto.com/releases/release-5.0.12/clients \ --set imageRegistry=gcr.io/prismacloud-cns
    Other Kubernetes
    apoctl enforcer install kubernetes --installation-mode yaml \ --cluster-type custom \ --namespace $CLUSTER_NS \ --api $MICROSEG_API \ --repo https://charts.aporeto.com/releases/release-5.0.12/clients \ --set imageRegistry=gcr.io/prismacloud-cns
    OpenShift
    apoctl enforcer install kubernetes --installation-mode yaml \ --cluster-type ocp4 \ --enable-openshift \ --namespace $CLUSTER_NS \ --api $MICROSEG_API \ --repo https://charts.aporeto.com/releases/release-5.0.12/clients \ --set imageRegistry=gcr.io/prismacloud-cns
    We detail the apoctl enforcer install kubernetes command further in the reference documentation. You can also run apoctl enforcer install kubernetes -h to review its flags. To learn how to customize your deployment, refer to Enforcer configuration options at the bottom of this page.
  3. Review the generated YAML.
    cat enforcerd.yaml
  4. Apply the YAML to your cluster.
    kubectl apply -f enforcerd.yaml
  5. To confirm your deployment, issue the following command.
    watch kubectl get pods --all-namespaces
    Wait until all of the pods have a status of Running or Completed.
    The above command uses watch, which is not installed by default on macOS. While we recommend installing it, you can also omit the watch portion of the command and repeatedly issue the command until the enforcer pods achieve the necessary status.
  6. Press CTRL+C to exit the watch command.
    Issue the following apoctl command to check the enforcers.
    apoctl api list enforcers --namespace $CLUSTER_NS \ -o table \ -c ID \ -c name \ -c namespace \ -c operationalStatus
  7. apoctl should return a list of the enforcers deployed.
    You should see an enforcer instance on each agent node. An example for a three-node GKE cluster follows.
    ID | name | namespace | operationalStatus ---------------------------+-------------------------------------------+----------------------------------+-------------------- 5f74d837f0fe170703c10d6b | gke-aws-dev-01-default-pool-cf284cf1-5bqn | /acme/aws-dev/k8s | Connected 5f74d836f0fe170703c10d6a | gke-aws-dev-01-default-pool-cf284cf1-5pjs | /acme/aws-dev/k8s | Connected 5f74d836f0fe170703c10d69 | gke-aws-dev-01-default-pool-cf284cf1-cqrd | /acme/aws-dev/k8s | Connected
    All enforcer instances should have an operationalStatus of Connected.
  8. Open the Microsegmentation Console web interface, navigate to the enforcer’s namespace, and select
    Enforcers
    under
    Manage
    .
    You should find your enforcers listed with the status
    connected
    . Click the enforcers to review their Microsegmentation tags.
  9. Select
    Platform
    in the side navigation menu.
    If your cluster contains pods outside of the kube-system namespace, you should see them with dashed green lines to a Somewhere external network. Your cluster is in discovery mode. Refer to Securing a Kubernetes namespace to learn how to allow the desired traffic and disable discovery mode.
    To see the pods and their traffic in the
    Platform
    pane, you may need to toggle
    Recursive
    to on.

Using a Helm chart

Prerequisites

Before you begin

The enforcer [requires a CNI plugin](../reqs/#clusters). Most Kubernetes and OpenShift clusters use CNI plugins by default, but GKE and AKS do not. GKE and AKS default to kubenet. Before deploying the enforcer to a GKE or AKS cluster, configure them to use CNI as follows.

Deploying the enforcers

  1. Set a CLUSTER_NS environment variable identifying the Microsegmentation namespace for this cluster.
    This should be a grandchild namespace.
    export CLUSTER_NS=/acme/aws-dev/k8s
  2. To generate a Helm chart that deploys the enforcers to your current context, use one of the following commands.
    EKS
    apoctl enforcer install kubernetes --installation-mode helm \ --cluster-type eks \ --namespace $CLUSTER_NS \ --api $MICROSEG_API \ --repo https://charts.aporeto.com/releases/release-5.0.12/clients \ --set imageRegistry=gcr.io/prismacloud-cns
    GKE
    apoctl enforcer install kubernetes --installation-mode helm \ --cluster-type gke \ --namespace $CLUSTER_NS \ --api $MICROSEG_API \ --repo https://charts.aporeto.com/releases/release-5.0.12/clients \ --set imageRegistry=gcr.io/prismacloud-cns
    AKS
    apoctl enforcer install kubernetes --installation-mode helm \ --cluster-type aks \ --namespace $CLUSTER_NS \ --api $MICROSEG_API \ --repo https://charts.aporeto.com/releases/release-5.0.12/clients \ --set imageRegistry=gcr.io/prismacloud-cns
    kops
    apoctl enforcer install kubernetes --installation-mode helm \ --cluster-type custom \ --custom-cni-chained \ --custom-cni-bin-dir /opt/cni/bin \ --namespace $CLUSTER_NS \ --api $MICROSEG_API \ --repo https://charts.aporeto.com/releases/release-5.0.12/clients \ --set imageRegistry=gcr.io/prismacloud-cns
    Other Kubernetes
    apoctl enforcer install kubernetes --installation-mode helm \ --cluster-type custom \ --namespace $CLUSTER_NS \ --api $MICROSEG_API \ --repo https://charts.aporeto.com/releases/release-5.0.12/clients \ --set imageRegistry=gcr.io/prismacloud-cns
    OpenShift
    apoctl enforcer install kubernetes --installation-mode helm \ --cluster-type ocp4 \ --enable-openshift \ --namespace $CLUSTER_NS \ --api $MICROSEG_API \ --repo https://charts.aporeto.com/releases/release-5.0.12/clients \ --set imageRegistry=gcr.io/prismacloud-cns
    We detail the apoctl enforcer install kubernetes command further in the reference documentation. You can also run apoctl enforcer install kubernetes -h to review its flags. To learn how to customize your deployment, refer to Enforcer configuration options at the bottom of this page.
  3. Confirm the Helm chart creation.
    ls enforcerd
  4. Create an aporeto namespace.
    kubectl create namespace aporeto
  5. Use the Helm chart to deploy the enforcers to your cluster.
    helm install enforcerd ./enforcerd --namespace aporeto
  6. To confirm your deployment, issue the following command.
    watch kubectl get pods --all-namespaces
    Wait until all of the pods have a status of Running or Completed.
    The above command uses watch, which is not installed by default on macOS. While we recommend installing it, you can also omit the watch portion of the command and repeatedly issue the command until the enforcer pods achieve the necessary status.
  7. Press CTRL+C to exit the watch command.
    Issue the following apoctl command to check the enforcers.
    apoctl api list enforcers --namespace $CLUSTER_NS \ -o table \ -c ID \ -c name \ -c namespace \ -c operationalStatus
  8. apoctl should return a list of the enforcers deployed.
    You should see an enforcer instance on each agent node. An example for a three-node GKE cluster follows.
    ID | name | namespace | operationalStatus ---------------------------+-------------------------------------------+----------------------------------+-------------------- 5f74d837f0fe170703c10d6b | gke-aws-dev-01-default-pool-cf284cf1-5bqn | /acme/aws-dev/k8s | Connected 5f74d836f0fe170703c10d6a | gke-aws-dev-01-default-pool-cf284cf1-5pjs | /acme/aws-dev/k8s | Connected 5f74d836f0fe170703c10d69 | gke-aws-dev-01-default-pool-cf284cf1-cqrd | /acme/aws-dev/k8s | Connected
    All enforcer instances should have an operationalStatus of Connected.
  9. Open the Microsegmentation Console web interface, navigate to the enforcer’s namespace, and select
    Enforcers
    under
    Manage
    .
    You should find your enforcers listed with the status
    connected
    . Click the enforcers to review their Microsegmentation tags.
  10. Select
    Platform
    in the side navigation menu.
    If your cluster contains pods outside of the kube-system namespace, you should see them with dashed green lines to a Somewhere external network. Your cluster is in discovery mode. Refer to Securing a Kubernetes namespace to learn how to allow the desired traffic and disable discovery mode.
    To see the pods and their traffic in the
    Platform
    pane, you may need to toggle
    Recursive
    to on.

Enforcer configuration options

The enforcer exposes the following configuration options. You can pass these to the apoctl enforcer install command using the --raw-flags flag. Example: --raw-flags "--log-level=debug --log-format=human --log-to-console=true" You can also modify the enforcer’s configuration after install via kubectl edit daemonset enforcerd -n aporeto, adding the flags as arguments to the container. The enforcer pods will restart. An example follows.
... spec: containers: - args: - --log-level=debug - --log-format=human ...
Flag
Description
Pass this flag if you wish to recognize the Microsegmentation Console as a processing unit, allowing its communications to be monitored and controlled. By default, the enforcer ignores them.
Pass this flag if you wish to recognize containers in the kube-system namespace as processing units, allowing their communications to be monitored and controlled. By default, the enforcer ignores them.
Pass this flag if you wish to recognize containers in Kubernetes namespaces starting with openshift- as processing units, allowing their communications to be monitored and controlled. By default, the enforcer ignores them.
The URL of the Microsegmentation Console API.
Path to CA certificate.
Disables check on certificate signature as trusted.
Path to application credentials.
Start of the port range for ports used by the enforcer application proxy. Defaults to 20992. You may adjust this if you experience conflicts.
The enforcer can determine if it is running in a cloud environment, such as AWS, GCP, or Azure. This is the maximum amount of time to wait for these internal probes to complete. Default is two seconds.
Pass this flag to disable the enforcer DNS proxy, which allows policies to be written based on FQDN, in cases where an exact IP address may be unpredictable.
DNS server address or CIDR that is observed by the enforcer DNS proxy. Defaults to 0.0.0.0/0.
(
Beta
) Pass this flag to gain performance improvements by using extended Berkeley Packet Filter (eBPF) on systems that support it.
The enforcer ignores IPv6 communications by default. If you have IPv6 enabled and wish to monitor and control these connections, pass this flag.
Quantity of logs that the enforcer should generate. Defaults to info. Alternatively, you can set it to debug, trace, or warn.
Controls whether the enforcer’s logs are written to stdout. Boolean that defaults to false.
The Microsegmentation namespace the enforcer should register in.
Microsegmentation tag for this enforcer.
Note
: to modify after the enforcer has started, you must shut down the enforcer, delete the enforcer object in the Microsegmentation Console, and perform a fresh install.
Microsegmentation token for the enforcer to use to register to the Microsegmentation Console.
A persistent working directory with write, read, and execute permissions. Files such as logs are stored here. Defaults to /var/lib/enforcerd

Recommended For You