Start Sending Logs to Cortex Data Lake (Panorama-Managed)
Table of Contents
Expand all | Collapse all
-
- Cortex Data Lake for Panorama-Managed Firewalls
- Start Sending Logs to a New Cortex Data Lake Instance
- Configure Panorama in High Availability for Cortex Data Lake
- Allocate Storage Based on Log Type
- View Cortex Data Lake Status
- View Logs in Cortex Data Lake
- TCP Ports and FQDNs Required for Cortex Data Lake
- Sizing for Cortex Data Lake Storage
-
- Forward Logs from Cortex Data Lake to a Syslog Server
- Forward Logs from Cortex Data Lake to an HTTPS Server
- Forward Logs from Cortex Data Lake to an Email Server
- Log Record Formats
- Create Log Filters
- Server Certificate Validation
- List of Trusted Certificates for Syslog and HTTPS Forwarding
- Log Forwarding Errors
Start Sending Logs to Cortex
Data Lake (Panorama-Managed)
Cortex
Data Lake
(Panorama-Managed)Learn how to send logs to
Cortex
Data Lake
from your Panorama-managed
firewalls.To send logs from Panorama™-managed firewalls to
Cortex
Data Lake
, you
must:- Install a supported PAN-OS® version on your Panorama and firewalls.
Activating
Cortex
Data Lake
includes provisioning the certificate that the
firewalls need to securely connect to Cortex
Data Lake
. Only after
you activate Cortex
Data Lake
can you enable Panorama-managed
firewalls to send logs.The following task describes how to start sending logs. First, you’ll enable firewalls to
communicate with
Cortex
Data Lake
and then you can specify the log
types that you want to send. You can then use Panorama device groups and templates
to push these settings to managed firewalls.If you’re using:
- Firewalls without Panorama—To send logs toCortex Data Lakefrom firewalls that are not managed by Panorama, follow these steps instead.
How you activate and implement
Cortex
Data Lake
varies depending on the
products and services you’re using. Learn more about how to get started with .- Specify the log types to send toCortex Data Lake.The way you enable sending depends on the log type. For logs that are generated based on a policy match, use a log forwarding profile within a device group. For other log types, use the Log Settings configuration within a template.
- To configure sending of System, Configuration, User-ID, and HIP Match logs:
- Select.DeviceLog Settings
- Select theTemplatethat contains the firewalls from which you want to send logs toCortex Data Lake.
- For each log type that you want to send toCortex Data Lake,Adda match list filter. Give it aName, optionally define aFilter, selectPanorama/Logging Service, and clickOK.
- To configure sending of all other log types that are generated when a policy match occurs, such as Traffic or Threat logs, create and attach a Log Forwarding profile to each policy rule for which you want to send logs.
- Select theDevice Groupand then selecttoObjectsLog ForwardingAdda profile. In the log forwarding profile match list, add each log type that you want to send.If you enabled the Enhanced Application Logs feature, then fullyEnable enhanced application logging to Cortex Data Lakeon the firewall to send these log types. When you select this option, match lists that specify the log types required for enhanced application logging are automatically added to the profile.
- SelectPanorama/as the Forward Method to enable the firewalls in the device group to send logs so you can monitor the logs and generate reports from Panorama.Cortex Data Lake
- Create basic Security policy rules in the device group.Until the firewall has interfaces and zones and a basic Security policy, it will not let any traffic through and, by default, will log only traffic that matches a Security policy rule.
- For each rule you create, selectActionsand select the Log Forwarding profile that allows the firewall to send logs toCortex Data Lake.
- (PA-7000 Series firewalls only) Configure a log card interface to perform log forwarding.As of PAN-OS 10.1, you can no longer forward system logs using the Management interface or using service routes through the Data Plane interfaces. The only way to forward system logs from a PA-7000 Series firewall running PAN-OS 10.1 or later is by configuring a Log Forwarding Card (LFC).
- Selectand clickNetworkInterfacesEthernetAdd Interface.
- Select theSlotandInterface Name.
- Set theInterface TypetoLog Card.
- Enter theIP Address,Default Gateway, and (for IPv4 only)Netmask.
- SelectAdvancedand specify theLink Speed,Link Duplex, andLink State.These fields default toauto, which specifies that the firewall automatically determines the values based on the connection. However, the minimum recommendedLink Speedfor any connection is1000(Mbps).
- ClickOKto save your changes.
- Commit your changes to Panorama and push them to the template and device group you created.
- Verify that the firewall logs are sent toCortex Data Lake.
- On Panorama 8.1.7 and later releases, selectand review the From Logging Service column to identify whether the logs that you view on Panorama are stored onMonitorLogsCortex Data Lake—yesindicates that the logs are saved toCortex Data Lake.Use the CLI commandrequest logging-service-forwarding statusfor detailed information on the connectivity status toCortex Data Lakeand to verify whether you enabled Duplicate Log Forwarding or Enhanced Application Logs.
- On a firewall, enter the CLI commandshow logging-status:
Look for the----------------------------------------------------------------------------------------------------------------------------- Type Last Log Created Last Log Fwded Last Seq Num Fwded Last Seq Num Acked Total Logs Fwded ----------------------------------------------------------------------------------------------------------------------------- > CMS 0 Not Sending to CMS 0 > CMS 1 Not Sending to CMS 1 >Log Collection Service 'Log Collection log forwarding agent' is active and connected to xx.xxx.xxx.xx config 2017/07/26 16:33:20 2017/07/26 16:34:09 323 321 2 system 2017/07/31 12:23:10 2017/07/31 12:23:18 13634645 13634637 84831 threat 2014/12/01 14:47:52 2017/07/26 16:34:24 557404252 557404169 93 traffic 2017/07/28 18:03:39 2017/07/28 18:03:50 3619306590 3619306590 1740 hipmatch Not Available Not Available 0 0 0 gtp-tunnel Not Available Not Available 0 0 0 userid Not Available Not Available 0 0 0 auth Not Available Not Available 0 0 0‘Log collection log forwarding agent’ is active and connected to <IP_address>line. You can also see that CMS 0 and CMS (the Log Collectors) are not receiving logs.On firewalls running PAN-OS 8.1.7 and later releases, you canShow Status) to verify that the firewall is connected and sending logs toDeviceSetupManagementCortex Data LakeCortex Data Lake.
- Use theACCon Panorama to monitor network activity.You can also selectandMonitorManage Custom ReportsRun Nowto generate reports on summary logs.
- (PAN-OS 10.0.2 or later and Cloud Services Plugin 1.8 or later) Generate scheduled reports onCortex Data Lakedata.
- ArchiveCortex Data Lakelogs by forwarding logs from to a Syslog server or email server for long-term storage, SOC, or internal audit.