Prisma Cloud enables you to automate the compression
of flow logs using the Google Cloud Dataflow service.
This additional automation on Prisma cloud addresses the
lack of native compression support for flow logs sink setup on GCP,
and helps reduce the egress costs associated with transferring large
volume of logs to the Prisma Cloud infrastructure. Therefore, Prisma
Cloud recommends that you enable flowlog compression.
When you enable Dataflow compression on Prisma Cloud, the Dataflow
pipeline resources are created in the same GCP project associated
with the Google Cloud Storage bucket to which your VPC Flow logs are
sent, and it saves the compressed logs also to the Cloud Storage
bucket. Therefore, if you are onboarding a GCP Organization and
enabling Dataflow compression to it or enabling Dataflow compression
to an existing GCP Organization that has been added to Prisma cloud,
make sure that the Dataflow-enabled Project ID is the same Google
Cloud Storage bucket to which you send VPC flow logs.
In order to launch the Dataflow job and create and stage the
compressed files, the following permissions are required:
Create a network, subnetwork, and firewall rules within your
VPC —
compute.networks.create
,
compute.subnetworks.create
,
compute.firewalls.create
,
compute.networks.updatepolicy
To
enable connectivity with the Dataflow pipeline resources and the
compute instances that perform log compression within your VPC,
Prisma Cloud creates a network, subnetwork, and firewall rules in
your VPC. You can view the compute instance that are spun up with
the RQL
config where api.name='gcloud-compute-instances-list' AND json.rule = name starts with "prisma-compress"
The Cloud Dataflow service spins up short lived compute instances
to handle the compression jobs and you may have associated costs
with the service. Palo Alto Networks recommends keeping your Cloud Storage
bucket in the same project in which you have enabled the Dataflow
service. Based on the location of your Cloud Storage bucket, Prisma
Cloud launches the Cloud Dataflow jobs in the following regions:
