Forward Logs from Cortex Data Lake to an HTTPS Server

Learn how to forward logs from Cortex Data Lake to an HTTPS server.
To meet your long-term storage, reporting and monitoring, or legal and compliance needs, you can configure Cortex Data Lake to forward logs to an HTTPS server or to services that receive events through HTTPS, such as Splunk HTTP Event Collector (HEC) or Microsoft Sentinel.
For successful log transmission to an HTTPS receiver, ensure that the receiver
  • Accepts and parses stacked or array JSON format. Cortex Data Lake forwards individual logs through HTTPS in JSON. However, the complete payload is either in stacked JSON, such as for Splunk HEC, or a JSON array, such as for Microsoft Sentinel.
    Stacked JSON
    JSON Array
    { "time": 1437522387, "host": "stream-logfwd20", "source": "Palo Alto Networks FLS LF", "event": { "LogType": "THREAT", "Severity": "Critical", ... } } { "time": 1437522387, "host": "stream-logfwd20", "source": "Palo Alto Networks FLS LF", "event": { "LogType": "THREAT", "Severity": "Info", ... } }
    [ { "time": 1437522387, "host": "stream-logfwd20", "source": "Palo Alto Networks FLS LF", "event": { "message": "Something happened", "severity": "INFO" } }, { "time": 1437522387, "host": "stream-logfwd20", "source": "testapp", "event": { "message": "Something happened", "severity": "INFO" } } ]
    For more information about HTTPS log format, see the Log Forwarding Schema Reference.
  • Accepts and decompresses GZIP HTTPS payloads. Cortex Data Lake compresses the JSON data using GZIP when forwarding through HTTPS.
    Microsoft Sentinel does not accept GZIP, so you must deploy a web app to decompress the data.
  • Accepts a batch size of 500 logs, or 2.25 MB (500 logs x 4500B/log = 2.25MB).
For each instance of Cortex Data Lake, you can forward logs to up to 200 destinations.
Cortex Data Lake communicates with the receiver using TLS 1.2 and the default cipher suites that Java uses. Upon connection Cortex Data Lake validates that the receiver has a certificate signed by a trusted root CA or a private CA. To complete the TLS handshake and establish the connection, the receiver must present all the certificates from the chain of trust.
  1. Enable communication between Cortex Data Lake and your HTTPS receiver. 
    Ensure that your HTTPS receiver can connect to Cortex Data Lake and can present a valid certificate to complete the connection request.
    • Allow an inbound TLS feed to your HTTPS receiver from the following IP address ranges:
      United States - Americas
      Netherlands - Europe
      United Kingdom
      DE (Germany)
      IN (India)
      SG (Singapore)
      CA (Canada)
      JP (Japan)
      AU (Australia)
      United States - Government
      If you have allowed specific IP addresses for inbound traffic, you must also allow the above IP address ranges to forward logs to your HTTPS receiver.
    • Obtain either a certificate from a well-known, public CA or a self-signed certificate and install it on your receiver. Please make sure that if you are using a certificate signed by a private CA, it contains CRL or OCSP information needed for certificate revocation checks.
      Because Cortex Data Lake validates the server certificate to establish a connection, you must verify that the receiver is configured to properly send the TLS certificate chain to Cortex Data Lake. If the app cannot verify that the certificate of the receiver and all CAs in the chain are trustworthy, the connection cannot be established. See the list of trusted certificates.
  2. Sign In
    to the hub at
  3. Select the Cortex Data Lake instance that you want to configure for HTTPS forwarding.
    If you have multiple Cortex Data Lake instances, click the Cortex Data Lake tile and select an instance from the list of those available.
  4. Select
    Log Forwarding
    to add a new HTTPS forwarding profile.
  5. Enter a descriptive
    for the profile.
  6. Enter the
    for the HTTPS receiver.
    The URL entered must begin with
    . At the end of the top-level domain, you can specify a port number. If no port is specified, the default is 443.
  7. (
    ) Enable Client Authorization.
    Do this if you are forwarding logs to Splunk HEC, Microsoft Sentinel, or if your HTTPS receiver requires a username and password.
    • For Splunk HEC, leave
      Splunk Basic Authorization
      selected and enter your HEC token.
    • For Microsoft Sentinel, select
      Sentinel Basic Authorization
      and enter your Workspace ID and Primary Key.
    • For client authorization on an HTTPS server, select
      Basic Authorization
      and specify a username and password.
  8. (
    the private Root CA and intermediate CAs (If an intermediate CA exists). Do not upload the certificate issued for the syslog server—only CA certificates are needed to verify the chain from the syslog server.
    Only do this if you installed a private CA-signed or self-signed certificate on your receiver. The file containing the certificates must be in PEM format.
  9. Test Connection
    to ensure that Cortex Data Lake can communicate with the receiver.
    This sends an empty log to the configured destination to verify that transmission is possible.
    If the test fails, you will not be able to proceed.
  10. Click
  11. In STATUS NOTIFICATION, enter an email address where you will be notified of any changes in log transmission status, such as if log forwarding stops.
  12. Enter a unique PROFILE TOKEN if your receiver needs to distinguish logs coming from different tenants.
  13. Select the logs you want to forward.
    1. Add
      a new log filter.
    2. Select the log type.
      The Threat log type does not include URL logs or Data logs. If you wish to forward these log types, you must add them individually.
    3. (Optional)
      Create a log filter to forward only the logs that are most critical to you.
      You can either write your own queries from scratch or use the query builder. You can also select the query field to choose from among a set of common predefined queries.
      Log filters function like queries in Explore. However, double quotes (
      ) are not supported.
      If you want to forward all logs of the type you selected, do not enter a query. Instead, proceed to the next step.
    4. Save
      your changes.
  14. Save
    your changes.
  15. Verify that the
    of your HTTPS forwarding profile is
    ( ).
    Immediately after creating or editing your profile, the
    may be
    for up to 10 minutes.
  16. Verify that you can view logs on the HTTPS receiver.
    For Splunk Common Information Model (CIM) fields and Enterprise Security, follow the guide at to install the PANW Splunk Addon and create a Splunk HEC input.

Recommended For You