Secure Kubernetes Services on Azure
Learn how the Azure plugin for Panorama works to secure
Azure Kubernetes services.
| Where Can I Use This? | What Do I Need? |
- Microsoft Azure
- Microsoft Azure Stack
- Azure® Marketplace
- Azure Government Marketplace
|
- VM-Series License (PAYG or BYOL)
- VM-Series plugin
- Panorama
- Panorama plugin for Azure
|
To secure Azure Kubernetes services, you must first install the Azure plugin on Panorama
and configure an Azure Secure Kubernetes Services on Azure deployment. The Azure plugin
for Panorama supports tag-based VM monitoring and Secure Kubernetes Services on Azure,
secures inbound traffic for Azure Kubernetes Services (AKS) clusters, and monitors
outbound traffic from AKS clusters. The
Panorama orchestrated deployment allows you to
use the Azure auto scale metrics and the scale-in and scale-out thresholds to manage
surges in demand for application workload resources by independently scaling the
VM-Series firewalls.
To secure inbound traffic for your AKS cluster, you must first Secure Kubernetes Services
on Azure. The
Panorama orchestrated deployment works with
Secure Kubernetes Services on Azure to gather information about your network and
resources, then create an auto-scaling tier of VM-Series firewalls for either Secure
Kubernetes Services on Azure deployments. See the
Palo Alto Networks Compatibility Matrix, to verify the
minimum OS, plugin, and template versions required to
secure AKS clusters.
Palo Alto Networks provides a
AKS template that deploys an Azure Kubernetes
Service (AKS) cluster in a new Azure VNet. The Azure plugin on Panorama helps you set up
a connection, which can monitor Azure Kubernetes cluster workloads, harvesting services
you have annotated as “internal load balancer” and creating tags you can use in
dynamic address groups. Use the Panorama
Dynamic Address Groups to apply a security policy on inbound traffic routed to services
running on your AKS cluster.
How Does the Panorama Plugin for Azure Secure Kubernetes Services?
Use VM-Series firewalls to secure inbound traffic for Azure Kubernetes
Service (
AKS) clusters. The VM-Series firewall can
only secure services exposed by a
load balancer (such as an Azure load
balancer). You can monitor only outbound traffic using this service.
This chapter reviews the different components that enable the Azure plugin for
Panorama to connect to an AKS cluster.
Requirements
This solution requires the following components. See the
Panorama plugin information in the
Compatibility Matrix for the minimum version requirements.
A Sample Hub-and-Spoke Topology to Secure AKS Clusters
The following diagram illustrates a sample auto scale deployment that
secures inbound traffic for Azure AKS clusters. Let’s review some of the
components.
Auto scaling Infrastructure—The
Azure Auto Scaling templates
create the messaging infrastructure and the basic hub and spoke
architecture.
AKS Clusters—The
Palo Alto Networks AKS template creates an
AKS cluster in a new VNet. Given the name of the Spoke resource group, the
template tags the VNet and AKS cluster with the Spoke resource group name,
so that the resource group is discovered by the Azure auto scaling plugin
for Panorama. The Azure plugin for Panorama queries service IP addresses on
the Staging ILB to learn about AKS cluster services.
Only one Spoke firewall scale set is to be associated with an AKS
cluster; if you expose multiple services in a single AKS cluster, they
must be protected by the same Spoke.
For each resource group, create a subnet-based address group. In the
above diagram, for example, create an address group for 10.240.0.0/24
(AKS Cluster 1).
VNet Peering—You must manually configure
VNET peering to communicate
with other VNets in the same region.
Cross-region peering isn't supported.
You can use other automation tools to deploy AKS clusters. If you deploy
in an existing VNet (the hub firewall VNet, for example) you must
manually configure VNet peering to the inbound and outbound hub and
spoke resource groups, and manually tag the VNet and AKS cluster with
the resource group name.
User-Defined Routes and Rules—You must manually configure
user-defined routes and rules (see
User-Defined Routing). In the diagram above, incoming traffic
can be redirected, according to UDR rules, to the firewall ILB for
inspection. Outbound traffic exiting an AKS Cluster is redirected to the hub
firewall ILB with Azure user-defined routing (
UDR) rules. The solution assumes
Allow All as a default policy for Kubernetes orchestration to function
as-is, but to apply the policy you can use an allowlist or a denylist to
allow or deny outbound traffic.
User-Defined Routing
Manually create
user-defined routing and routing rules to
govern inbound or outbound traffic.
Inbound
In the above diagram, inbound traffic from the Application gateway is driven to the
backend pool, and based on UDR rules, redirected to the firewall ILB. For example,
create a UDR pointing to the VNet subnet so that the traffic for Kubernetes services
is pointed to the firewall ILB.
Outbound
On the hub firewall set, for each AKS cluster being protected. Create static routes
for the cluster subnet CIDR, with the next hop being the gateway address of the hub
VNet trust subnet.
All outbound traffic for an AKS cluster is directed to the hub firewall set with a
single UDR rule.
AKS Cluster Communication
The Panorama plugin for Azure can only communicate with the AKS controller node for a
given AKS cluster. For Outbound AKS traffic, the next hop is the hub firewall ILB.
Because Outbound traffic is monitored, you must Allow All traffic. The following
topics emphasize common practices that help you establish connectivity. Keep them in
mind when you plan your networks and subnets.
Create AKS Cluster Authentication
- Create a ClusterRole.
- Create a ClusterRoleBinding.
- Create a .yaml file for the ClusterRoleBinding.
For example, create a text file named
crb.yaml.
apiVersion: rbac.authorization.k8s.io
kind: ClusterRoleBinding
metadata:
name: default-view
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: view
subjects:
- kind: ServiceAccount
name: default
namespace: default
- Use Azure Cloud Shell to apply
the crb.yaml role binding.
kubectl
apply -f crb.yaml
- View the service account you created.
kubectl get
serviceaccounts
- Save the service account credential to a .json file.
- On your local machine, change to the directory in which you want to
save the credential.
- Use kubectl command to create the
token.
MY_SA_TOKEN=‘kubectl get serviceaccounts
default -o
jsonpath=’{.secrets[0].name}’‘
- View the token name.
$ echo
$MY_SA_TOKEN
- Display the credential.
kubectl get secret $MY_SA_TOKEN
-o json
Use an Address Group to Identify Traffic
To create some granularity for monitored Outbound traffic, create an address
group specifically for the AKS cluster VNet subnet (for example, 10.240.0.97/32
in the above diagram). You can then write rules that allow incoming or returning
traffic rather than using Allow All.
If communication is interrupted, application traffic can be lost or your
application deployment might have problems.
Add the Subnet Address Group to the Top-Level Policy
To maintain connectivity, the address group must be part of the top-level policy
in Panorama. You can configure the cluster address group, or bootstrap the
cluster to configure the cluster address group.
Add the address group to the top-level policy
before you
configure VNet peering or
User-Defined Routing.
Prevent Application Disruption When Workload and AKS Cluster VNets Are
Peered
If an AKS cluster coexists with VM workloads that run in separate VNets, and the
VNet is peered with both the workload spoke (Inbound) and the hub (Outbound),
you must create address groups to differentiate the workloads and the AKS
traffic, and add the address group to Top-Level Policy as described above.
Dynamic Address Groups with Kubernetes Labels
When monitoring an AKS cluster resource, the Azure plugin automatically generates the
following IP tags for AKS services.
aks.<aks cluster name>.<aks service name>
Tags are not generated for nodes, pods, or other resources.
If the AKS service has any labels, the tag is as follows (one per label):
aks.<aks cluster name>.svc.<label>.<value>
If a labelSelector tag is defined for a cluster, the plugin generates the following
IP tag:
aks_<labelSelector>.<aks cluster name>.<aks service name>