HTTP Header Logging
Focus
Advanced URL Filtering

HTTP Header Logging

Table of Contents

HTTP Header Logging

Capture HTTP request and response headers in URL filtering logs to verify security policy enforcement and investigate network traffic.
Where can I use this?What do I need?
  • NGFW (Managed by PAN-OS or Panorama)
    • Decryption policy to decrypt HTTPS traffic
    • Legacy URL filtering licenses are discontinued, but active legacy licenses are still supported.
URL filtering provides visibility and control over web traffic on your network. For improved visibility into web content, you can configure the URL Filtering profile to log HTTP header attributes included in a web request. When a client requests a web page, the HTTP header includes the user agent, referer, and x-forwarded-for fields as attribute-value pairs and forwards them to the web server. When enabled for logging HTTP headers, the firewall logs the following attribute-value pairs in the URL Filtering logs.
You can also use HTTP headers to manage access to SaaS applications. You don’t need a URL Filtering license to do this, but you must use a URL Filtering profile to turn this feature on.
Attribute
Description
User-Agent
The web browser that the user used to access the URL, for example, Internet Explorer. This information is sent in the HTTP request to the server.
The HTTP header does not contain the full string for the User Agent. The maximum logged bytes from the packet preceding the packet containing the header-end is 36 bytes.
Referer
The URL of the web page that linked the user to another web page; it is the source that redirected (referred) the user to the web page that is being requested.
X-Forwarded-For (XFF)
The option in the HTTP request header field that preserves the IP address of the user who requested the web page. If you have a proxy server on your network, the XFF allows you to identify the IP address of the user who requested the content, instead of only recording the proxy server’s IP address as source IP address that requested the web page.
Headers Inserted
The type of header and the text of the header that the firewall inserts.
HTTP Header Logging gives you granular visibility into web traffic by capturing any HTTP request or response headers—including custom headers—directly in URL filtering logs. Without this visibility, verifying that security policies are enforcing correctly and investigating suspicious traffic requires guesswork.
You configure HTTP Header Logging in a URL filtering profile by selecting which headers to capture. You can add individual request headers, individual response headers, or enable Log all HTTP headers to capture every header. The HTTP response status code is captured in every response log entry. Enable Enable logging of response code in HTTP response to force a response log entry to be generated even when response headers are not logged.
When you log both request and response sides, the firewall generates two URL filtering log entries per resource. A URL Index field and a Direction field correlate the pair so you can reconstruct the full exchange.
Use Max Header Length to limit how much data is captured per header value (1–4,092 bytes). The limit applies to the header value only and excludes the header name, colon, and whitespace. Enable Enable truncation of headers that exceed the max header length limit to capture headers up to the limit rather than skipping them entirely. If you set Max Header Length to 1 and enable truncation, only the header name is logged.
The following headers are always excluded from logging, even if explicitly configured:
  • HTTP/1.1 and HTTP/2: host, referrer, user-agent, x-forwarded-for, content-type. These populate the legacy URL filtering log fields and are not duplicated in the new header fields.
  • HTTP/2 only: path, method, status, authority.
Request headers can contain personally identifiable information (PII) such as cookies and authorization tokens. A privacy notice appears the first time you enable request header logging. Review your data privacy requirements before enabling logging for sensitive headers.