Forward Logs to Cortex Data Lake
After you activate Cortex Data Lake, you can configure firewalls to start forwarding logs. This includes enabling the firewalls to communicate with Cortex Data Lake, and specifying the log types that you want to send.
The steps here describe how to start forwarding logs from firewalls that are not managed by Panorama. This direct onboarding to Cortex Data Lake—where you can provision the certificates firewalls require to connect to Cortex Data Lake without using Panorama—is supported for firewalls running PAN-OS 9.0.3 and later. When you activated Cortex Data Lake, you should have also generated a key that you’ll now use to authenticate the firewall to Cortex Data Lake. If you haven’t generated this key yet, do this now on the hub, and then come back to these steps.
If you’re using Panorama, Traps, or GlobalProtect cloud service:
- GlobalProtect cloud service—To enable firewalls deployed within your GlobalProtect cloud service infrastructure to log to the Cortex Data Lake, see the GlobalProtect Cloud Service Getting Started Guide.
- If you haven’t done so already, Activate Cortex Data Lake.Onboarding includes activating Cortex Data Lake, and generating the key that enables the firewall to securely connect to Cortex Data Lake. You’ll need to use the key in the coming steps.After you’ve activated Cortex Data Lake, log in to the firewall to continue.
- Selectand confirm that the Logging Service (now called Cortex Data Lake) license is active.DeviceLicensesWhen you purchased Cortex Data Lake, all firewalls registered to your support account received a Cortex Data Lake license. If you don’t see the Logging Service license, selectRetrieve license keys from license serverto manually refresh the firewall licenses.
- Enable Cortex Data lake on the firewall.
- Selectand find theDeviceSetupManagementLogging Servicesettings (Cortex Data Lake used to be called the Logging Service).
- Enable Logging Serviceto allow the firewall to send and store logs in Cortex Data Lake.Note that logging to Cortex Data Lake only begins after you set up Log Forwarding, which you’ll do in the coming steps.Do not select the option toEnable Duplicate Logging. This option applies only to Panorama-managed firewalls (in this case, you can choose if you’d like Panorama to also store firewall logs, in addition to Cortex Data Lake).
- Select the geographicRegionof the Cortex Data Lake instance to which you’ll be forwarding logs. This is the region you chose when you were activating the Cortex Data Lake (either Europe or Americas).
- ToOnboard Without Panorama, selectConnectand enter thePSKyou generated in the Cortex Data Lake app. Then, clickConnectagain.
- Show Statusto checkLogging Service Status(Cortex Data Lake). All status indicators—License, Certificate, Customer Info, and Device Connectivity—should show as green.
- On the firewall, configure NTP so that the firewall can stay in sync with the Cortex Data Lake.On the firewall, selectand set it to the sameDeviceSetupServicesNTPNTP Server Addressyou configured on Panorama, for examplepool.ntp.org.
- (Optional)If you do not want to use the management interface to forward logs to the Cortex Data Lake, enable the firewall to send traffic through a different interface.Beginning with content release version 8067, you can use the paloalto-shared-services and paloalto-logging-service App-IDs to safely enable traffic between the firewalls and the Cortex Data Lake. You will also need to create a security policy rule to allow this traffic on any firewalls between the firewalls sending the logs and the internet. If the upstream firewalls are not Palo Alto Networks firewalls, you must enable access to the TCP Ports and FQDNs Required for Cortex Data Lake. Keep in mind that the firewalls and the Cortex Data Lake use mutual certificate authentication and therefore cannot be decrypted and you cannot connect through a proxy server.
- Configure a service route for Palo Alto Networks Services.
- Create a security policy rule that enables the firewalls to communicate with the Cortex Data Lake.This is required if you are using the Palo Alto Networks Services service route instead of the management interface to forward logs to the Cortex Data Lake. To create this rule, set theApplicationtopaloalto-shared-services(requires content release version 8066 or later) andpaloalto-logging-service(requires content release version 8033 or later). The paloalto-shared-services covers the common traffic for different Palo Alto Networks services and is a dependency for the paloalto-logging-service.Make sure you place this rule above any rule that allows the web-browsing and SSL traffic to the internet. In addition, if you have a firewall between Panorama and the internet, you must also add a rule that allows paloalto-shared-services and paloalto-logging service traffic on that firewall. The paloalto-logging-service app enables the firewalls and Panorama to connect to the Cortex Data Lake on ports 444 and 3978, the defaults ports for this communication.If that firewall is not a Palo Alto Networks firewall, create a security policy rule on that firewall that allows outbound SSL traffic to the internet to allow the TCP Ports and FQDNs Required for Cortex Data Lake so that the internet gateway firewall does not block traffic between Panorama and the Cortex Data Lake.The firewalls and Panorama need access to the domain 8.0.0 on port 3978 in order to forward logs to the Cortex Data Lake. This is true even if you are using the paloalto-logging-service App-ID to safely enable Cortex Data Lake traffic.
- Specify the log types to forward to the Cortex Data Lake.The way you enable forwarding depends on the log type. For logs that are generated based on a policy match, use a log forwarding profile within a device group. For other log types, use the Log Settings configuration within a template.
- To configure forwarding of System, Configuration, User-ID, and HIP Match logs:
- Select.DeviceLog Settings
- For each log type that you to forward to the Cortex Data Lake,Adda match list filter. Give it aName, optionally define aFilter, select theLogging Servicecheck box, and clickOK.
- To configure forwarding of all other log types that are generated when a policy match occurs—Traffic, Threat, WildFire Submission, URL Filtering, Data Filtering, and Authentication logs—create and attach a Log Forwarding profile to each policy rule for which you want to forward logs.
- SelecttoObjectsLog ForwardingAdda profile. In the log forwarding profile match list, add each log type that you want to forward.If you have already turned on Enhanced Application Logs, fully enable the firewall to forward these log types by selectingEnable enhanced application logging to Cortex Data Lake. Notice that when you select this option, match lists that specify the logs types required for enhanced application logging are automatically added to the profile.
- SelectLogging Serviceas the Forward Method to enable the firewalls in the device group to forward the logs to the Cortex Data Lake. You will be able to monitor the logs and generate reports from Panorama.
- If you haven’t already done so, Create basic security policy rules now.Until the firewall has interfaces and zones and a basic security policy, it will not let any traffic through, and only traffic that matches a security policy rule will be logged (by default).
- For each rule you create, selectActionsand select the Log Forwarding profile that allows the firewall to send logs to the Cortex Data Lake.
- Commityour changes.
- Verify that the firewall logs are being forwarded to the Cortex Data Lake.
- On a firewall, enter the CLI commandshow logging-status:
Look for the----------------------------------------------------------------------------------------------------------------------------- Type Last Log Created Last Log Fwded Last Seq Num Fwded Last Seq Num Acked Total Logs Fwded ----------------------------------------------------------------------------------------------------------------------------- > CMS 0 Not Sending to CMS 0 > CMS 1 Not Sending to CMS 1 >Log Collection Service 'Log Collection log forwarding agent' is active and connected to xx.xxx.xxx.xx config 2017/07/26 16:33:20 2017/07/26 16:34:09 323 321 2 system 2017/07/31 12:23:10 2017/07/31 12:23:18 13634645 13634637 84831 threat 2014/12/01 14:47:52 2017/07/26 16:34:24 557404252 557404169 93 traffic 2017/07/28 18:03:39 2017/07/28 18:03:50 3619306590 3619306590 1740 hipmatch Not Available Not Available 0 0 0 gtp-tunnel Not Available Not Available 0 0 0 userid Not Available Not Available 0 0 0 auth Not Available Not Available 0 0 0‘Log collection log forwardingagent’ is active and connected to <IP_address>line. You can also see that CMS 0 and CMS (the Log Collectors) are not receiving logs.On firewall running PAN-OS 8.1.7 or later, select theShow Statuslink onto verify that the firewall is connected and sending logs to the Cortex Data Lake.DeviceSetupManagementCortex Data Lake
- You can use the Log Forwarding app to archive the logs you send to Cortex Data Lake.If you want to be able to archive the logs you send to the Cortex Data Lake for long-term storage, SOC, or internal audit directly from the Cortex Data Lake, you can use the Log Forwarding app, which is included with your Cortex Data Lake (formerly called Logging Service) license. This app enables log forwarding from the Cortex Data Lake to an external destination such as a Syslog server or an email server. Refer to the Log Forwarding App Getting Started Guide for more information. Alternatively, you continue to forward logs directly from the firewalls to your Syslog receiver.
Forward Logs to Cortex Data Lake (Panorama-Managed Firewalls)
Forward Logs to Cortex Data Lake (Panorama-Managed Firewalls) For Panorama-managed firewalls to send logs to Cortex Data Lake, the firewalls and Panorama need to be ...
Cortex Data Lake Logging for Firewalls without Panorama
Cortex Data Lake Logging for Firewalls without Panorama Palo Alto Networks® Cortex Data Lake provides cloud-based, centralized log storage and aggregation for firewalls and certain ...
Get Started with the Cortex Data Lake
Get up and running with the Palo Alto Networks Logging service quickly and easily. ...
Forward Logs to the Logging Service
Forward Logs to Cortex Data Lake Cortex Data Lake is Palo Alto Networks’ cloud-based logging infrastructure. Before you can configure your managed firewalls to send ...
Plan Your Cortex Data Lake Deployment
Plan Your Cortex Data Lake Deployment Here are some requirements and tips to consider as you plan your Cortex Data Lake deployment: Use the Cortex ...
Cortex Data Lake Log Sources
See the products and services that can send logs to Cortex Data Lake. ...
Configure Firewalls and Panorama to Support Cortex XDR – Analytics
Configure Firewalls and Panorama to Support Cortex XDR – Analytics Follow these required steps to configure Palo Alto Networks firewalls and Panorama to support Cortex ...
Activate Cortex Data Lake on the Cortex Hub
Activate Cortex Data Lake After purchasing Cortex Data Lake, you received an auth code that you’ll use to activate Cortex Data Lake. The steps here ...