PAN-DB—the URL Filtering cloud database—classifies websites based on site content, features, and safety. A URL can have up to four URL Categories, including risk categories (high, medium, and low) that indicate how likely it is that the site will expose you to threats. As PAN-DB categorizes sites, firewalls with URL Filtering enabled can leverage that knowledge in real-time to enforce security policy.
The following table describes the PAN-DB components in detail. The BrightCloud system works similarly, but does not use an initial seed database.
URL Filtering Seed Database
The initial seed database downloaded to the firewall is a small subset of the database that is maintained on the Palo Alto Networks URL cloud servers. The reason this is done is because the full database contains millions of URLs and many of these URLs may never be accessed by your users. When downloading the initial seed database, you select a region (North America, Europe, APAC, Japan). Each region contains a subset of URLs most accessed for the given region. This allows the firewall to store a much smaller URL database for better URL lookup performance. If a user accesses a website that is not in the local URL database, the firewall queries the full cloud database and then adds the new URL to the local database. This way the local database on the firewall is continually populated/customized based on actual user activity.
Re-downloading the PAN-DB seed database or switching the URL database vendor from PAN-DB to BrightCloud will clear the local database.
See Differences Between the PAN-DB Public Cloud and PAN-DB Private Cloud, for information on the private cloud.
The PAN-DB cloud service is implemented using Amazon Web Services (AWS). AWS provides a distributed, high-performance, and stable environment for seed database downloads and URL lookups for Palo Alto Networks firewalls and communication is performed over SSL. The AWS cloud systems hold the entire PAN-DB and is updated as new URLs are identified. The PAN-DB cloud service supports an automated mechanism to update the local URL database on the firewall if the version does not match. Each time the firewall queries the cloud servers for URL lookups, it will also check for critical updates. If there have been no queries to the cloud servers for more than 30 minutes, the firewall will check for updates on the cloud systems.
The cloud system also provides a mechanism to submit URL category change requests. This is performed through the test-a-site service and is available directly from the firewall (URL filtering profile setup) and from the Palo Alto Networks Test A Site website. You can also submit a URL categorization change request directly from the URL filtering log on the firewall in the log details section.
Management Plane (MP) URL Cache
When you activate PAN-DB on the firewall, the firewall downloads a seed database from one of the PAN-DB cloud servers to initially populate the local cache for improved lookup performance. Each regional seed database contains the top URLs for the region and the size of the seed database (number of URL entries) also depends on the platform. The URL MP cache is automatically written to the local drive on the firewall every eight hours, before the firewall is rebooted, or when the cloud upgrades the URL database version on the firewall. After rebooting the firewall, the file that was saved to the local drive will be loaded to the MP cache. A least recently used (LRU) mechanism is also implemented in the URL MP cache in case the cache is full. If the cache becomes full, the URLs that have been accessed the least will be replaced by the newer URLs.
Dataplane (DP) URL Cache
This is a subset of the MP cache and is a customized, dynamic URL database that is stored in the dataplane (DP) and is used to improve URL lookup performance. The URL DP cache is cleared at each firewall reboot. The number of URLs that are stored in the URL DP cache varies by hardware platform and the current URLs stored in the TRIE (data structure). A least recently used (LRU) mechanism is implemented in the DP cache in case the cache is full. If the cache becomes full, the URLs that have been accessed the least will be replaced by the newer URLs. Entries in the URL DP cache expire after a specified period of time; this expiration period is not configurable.
URL Database Out of Date
URL Database Out of Date If you have observed through the syslog or the CLI that PAN-DB is out-of-date, it means that the connection from ...
Enable PAN-DB URL Filtering
Enable PAN-DB URL Filtering Obtain and install a PAN-DB URL filtering license and confirm that it is installed. If the license expires, the firewall ceases ...
Troubleshoot URL Filtering
Troubleshoot URL Filtering The following topics provide troubleshooting guidelines for diagnosing and resolving common URL filtering problems. Problems Activating PAN-DB PAN-DB Cloud Connectivity Issues URLs ...
PAN-DB Cloud Connectivity Issues
PAN-DB Cloud Connectivity Issues To check connectivity between the firewall and the PAN-DB cloud: show url-cloud status If the cloud is accessible, the expected response ...
Problems Activating PAN-DB
Problems Activating PAN-DB Use the following workflow to troubleshoot PAN-DB activation issues. Access the PAN-OS CLI . Verify whether PAN-DB has been activated by running ...
URL Filtering Vendors
URL Filtering Vendors Palo Alto Networks firewalls support two URL filtering vendors: PAN-DB —The Palo Alto Networks-developed URL filtering database. PAN-DB provides high-performance local caching ...
URLs Classified as Not-Resolved
URLs Classified as Not-Resolved Use the following workflow to troubleshoot why some or all of the URLs being identified by PAN-DB are classified as Not-resolved: ...
Select a URL Filtering Vendor on Panorama
Select a URL Filtering Vendor on Panorama URL filtering enables firewalls to monitor and control web access for your users. The policy rules that you ...
M-500 Appliance for PAN-DB Private Cloud
M-500 Appliance for PAN-DB Private Cloud To deploy a PAN-DB private cloud, you need one or more M-500 appliances. The M-500 appliance ships in Panorama ...