Configure URL Filtering

Follow these steps to configure custom URL Filtering profiles that meet your organization’s business and security needs.
After you determine URL Filtering policy requirements, you should have a basic understanding of the types of websites and website categories your users are accessing. Use this information to create custom URL Filtering profiles and attach them to the Security policy rules that allow web access. In addition to managing web access with a URL Filtering profile, if you configure User-ID™, you can manage the sites to which users can submit corporate credentials.
  1. Create a URL Filtering profile.
    If you didn’t already, configure a best practice URL Filtering profile to ensure protection against URLs hosting malware or exploitive content.
    Select
    Objects
    Security Profiles
    URL Filtering
    and
    Add
    or modify a URL Filtering profile.
  2. Define site access for each URL category.
    Select
    Categories
    and set the Site Access for each URL category:
    • allow
      traffic destined for that URL category; allowed traffic is not logged.
    • Select
      alert
      to have visibility into sites that users are accessing. Traffic matching that category is allowed but a URL Filtering log is generated to record when a user accesses a site in that category.
    • Select
      block
      to deny access to traffic that matches that category and to enable logging of the blocked traffic.
    • Select
      continue
      to display a page to users with a warning and require them to click
      Continue
      to proceed to a site in that category.
    • To only allow access if users provide a configured password, select
      override
      . For more details, see Allow Password Access to Certain Sites.
  3. Configure the URL Filtering profile to detect corporate credential submissions to websites that are in allowed URL categories.
    To ensure the best performance and a low false positive rate, the firewall automatically skips checking the credential submissions for any App-ID™ associated with sites that have never been observed hosting malware or phishing content—even if you enable checks in the corresponding category. The list of sites for which the firewall skips credential checking is automatically updated through Applications and Threats content updates.
    1. Select
      User Credential Detection
      .
    2. Select one of the methods to check for corporate credential submissions to web pages from the
      User Credential Detection
      drop-down:
      • Use IP User Mapping
        —Checks for valid corporate username submissions and verifies that the username matches the user logged in to the source IP address of the session. The firewall matches the submitted username against its IP address-to-username mapping table. You can use any of the user mapping methods described in Map IP Addresses to Users.
      • Use Domain Credential Filter
        —Checks for valid corporate usernames and password submissions and verifies that the username maps to the IP address of the logged-in user. See Configure User Mapping Using the Windows User-ID Agent for instructions on how to set up User-ID to enable this method.
      • Use Group Mapping
        —Checks for valid username submissions based on the user-to-group mapping table populated when you configure the firewall to map users to groups.
        With group mapping, you can apply credential detection to
        any
        part of the directory or to a specific group, such as groups like IT that have access to your most sensitive applications.
      This method is prone to false positives in environments that do not have uniquely structured usernames, so you should only use this method to protect your high-value user accounts.
    3. Set the
      Valid Username Detected Log Severity
      that the firewall uses to log detection of corporate credential submissions (default is medium).
  4. Configure the URL Filtering profile to detect phishing and malicious JavaScript in real-time using URL Filtering Inline ML.
  5. Allow or block users from submitting corporate credentials to sites based on URL category to prevent credential phishing.
    To ensure the best performance and a low false positive rate, the firewall automatically skips checking the credential submissions for any App-ID associated with sites that have never been observed hosting malware or phishing content—even if you enable checks in the corresponding category. The list of sites for which the firewall skips credential checking is automatically updated through Applications and Threats content updates.
    1. For each URL category to which you allow
      Site Access
      , select how you want to treat
      User Credential Submissions
      :
      • alert
        —Allow users to submit credentials to the website but generate a URL Filtering alert log each time a user submits credentials to sites in this URL category.
      • allow
        (default)—Allow users to submit credentials to the website.
      • block
        —Displays the Anti Phishing Block Page to block users from submitting credentials to the website.
      • continue
        —Present the Anti Phishing Continue Page to require users to click
        Continue
        to access the site.
  6. Define URL category exception lists to specify websites that should always be blocked or allowed, regardless of URL category.
    For example, to reduce URL Filtering logs, you may want to add your corporate websites to the allow list so that no logs are generated for those sites or, if there is a website that is being overly used and is not work-related, you can add that site to the block list.
    The policy actions configured for custom URL categories have priority enforcement over matching URLs in external dynamic lists.
    Traffic to websites in the block list is always blocked regardless of the action for the associated category and traffic to URLs in the allow list is always allowed.
    For more information on the proper format and wildcard usage, review the URL category exception list guidelines.
  7. Log only Container Pages for URL filtering events.
    1. Select
      URL Filtering Settings
      . Enable
      Log container page only
      (default) so that the firewall logs only the main page that matches the category, not subsequent pages or categories that loaded within the container page.
    2. To enable logging for all pages and categories, disable the
      Log container page only
      option.
  8. Enable HTTP Header Logging for one or more of the supported HTTP header fields.
    Select
    URL Filtering Settings
    and select one or more of the following fields to log:
    • User-Agent
    • Referer
    • X-Forwarded-For
  9. Save the URL Filtering profile and commit your changes.
    1. Click
      OK
      .
    2. Click
      Commit
      .
  10. Test your URL Filtering policy configuration.
    1. Access a website in the desired URL category and observe the firewall’s behavior.
      Use Palo Alto Networks URL Filtering Test Pages (urlfiltering.paloaltonetworks.com/test-
      <url-category>
      ) if you want to avoid directly accessing a site. Palo Alto Networks has test URLs for benign and malicious URL categories. For example, to test your block policy for malware, visit https://urlfiltering.paloaltonetworks.com/test-malware.
    2. Review the Traffic and URL Filtering logs (
      Monitor
      Logs
      ) to confirm that the correct policy rule is logged.
  11. Enable
    Hold client request for category lookup
    to block client requests while the firewall performs URL category lookups.
    1. Select
      Device
      Setup
      Content-ID
      .
    2. Select
      Hold client request for category lookup
      .
    3. Commit your changes.
    Enable this feature as a URL Filtering best practice.
  12. Set the amount of time, in seconds, before a URL category lookup times out.
    1. Select
      Device
      Setup
      Content-ID
      gear icon
      .
    2. Enter a number in
      Category lookup timeout (sec)
      .
    3. Click
      OK
      .
    4. Commit your changes.

Recommended For You