URL Category Exceptions
Table of Contents
Expand All
|
Collapse All
Next-Generation Firewall Docs
-
-
- Cloud Management of NGFWs
- PAN-OS 10.0 (EoL)
- PAN-OS 10.1
- PAN-OS 10.2
- PAN-OS 11.0
- PAN-OS 11.1 & Later
- PAN-OS 9.1 (EoL)
-
- PAN-OS 10.1
- PAN-OS 10.2
- PAN-OS 11.0
- PAN-OS 11.1 & Later
-
-
-
- Cloud Management and AIOps for NGFW
- PAN-OS 10.0 (EoL)
- PAN-OS 10.1
- PAN-OS 10.2
- PAN-OS 11.0
- PAN-OS 11.1
- PAN-OS 11.2
- PAN-OS 8.1 (EoL)
- PAN-OS 9.0 (EoL)
- PAN-OS 9.1 (EoL)
End-of-Life (EoL)
URL Category Exceptions
Guidelines for adding entries to a custom URL list or
external dynamic list you want to use in a URL Filtering profile
or policy.
You can exclude specific websites from URL category
enforcement, ensuring that these websites are blocked or allowed
regardless of the policy action associated with its URL categories.
For example, you might block the social-networking URL category
but allow access to LinkedIn. To create exceptions to URL category
policy enforcement:
- Add the IP addresses or URLs of sites you want to block
or allow to a custom URL category of
type URL List (ObjectsCustom ObjectsURL Category). Then,
define site access for the custom URL category in a URL Filtering
profile. Finally, attach the profile to a Security policy rule.You can also use a custom URL category as match criteria in a Security policy rule (PoliciesSecurity, and select Service/URL Category). The exception rule must be placed above any rules that block or allow the categories to which the URL exceptions belong.
- Add the URLs of sites you want to block or allow to an external dynamic list of type URL List (ObjectsExternal Dynamic Lists). Then, use the external dynamic list in a URL Filtering profile or as match criteria in a Security policy rule. The benefit to using an external dynamic list is that you can update the list without performing a configuration change or commit on the firewall.
External dynamic lists of type URL List should
not be confused with external dynamic lists of type
Domain List or IP Address. While external
dynamic lists of URLs permit domains and IP addresses, the reverse
is not true and result in invalid entries.
The following guidelines describe how to populate URL category
exception lists—custom URL categories or external dynamic lists
of URLs:
- Basic Guidelines For URL Category Exception Lists
- Wildcard Guidelines for URL Category Exception Lists
- URL Category Exception List—Examples
Basic Guidelines For URL Category Exception Lists
Consider the potential matches an entry might have before
adding it to a URL category exception list. The following guidelines
specify how to create an entry that blocks or allows the websites
and pages you intend. Create and evaluate existing entries against
these guidelines to ensure you do not block or allow access to more
URLs than anticipated.
The firewall assumes an implicit asterisk at the end of
domain entries that do not end in a trailing slash (/) or
asterisk (*). For example, if you add example.com to
a URL list of allowed websites, the firewall interprets that entry
as example.com.*. As a result, the firewall
allows access to sites such as example.com.domain.xyz.
To
prevent the firewall from assuming the implicit asterisk, you can
append a trailing slash to domain entries that do not end in a / or *.
The addition of the trailing slash changes the URLs that the firewall
considers a match and for which it enforces policy. In non-wildcard
domain entries, the trailing slash limits matches to the given domain
and its subdirectories. For example, example.com (example.com/ after
processing) matches itself and example.com/search.
In
wildcard domain entries (entries with asterisks or carets), the
trailing slash limits matches to URLs that conform to the specified
pattern. For example, to match the entry *.example.com,
a URL must include at least one subdomain and end with the root
domain, example.com. The pattern is: <subdomain>.example.com; news.example.com is
a match, but example.com is not because it
lacks a subdomain.
You can enable the firewall to automatically
append a trailing slash to applicable entries using the following
command line interface (CLI) commands:
admin@PA-850> debug device-server append-end-token on
admin@PA-850> configure
admin@PA-850# commit
We
recommend manually adding trailing slashes to clarify the intended matching
behavior of an entry for anyone who inspects it. The trailing slash
is invisible if added by the firewall. To disable this feature:
admin@PA-850> debug device-server append-end-token off
admin@PA-850> configure
admin@PA-850# commit
Panorama™ management servers running PAN-OS® 10.2
cannot enable this feature for firewalls running PAN-OS 10.1 or
earlier. You have to enable this feature on each firewall running
PAN-OS 10.1 or earlier.
- List entries are case-insensitive.
- Omit http and https from URL entries.
- Each URL entry can be up to 255 characters in length.
- Every domain that does not end in a / or an * has an implicit asterisk to its end. The firewall processes the entry as if you entered: <domain>.*.
- Enter an exact match to the website you want to block or allow or use wildcards to create a pattern match.Different entries result in different exact matches. If you enter the URL for a specific web page (example.com/contact), the firewall limits matches to that page alone. Exact matching for a domain with a trailing slash restricts matches to the domain itself and its subdirectories. If you enter a domain (without a trailing slash), the firewall matches additional URLs due to the implicit asterisk.
- Consider adding the URLs most commonly used to access a website or page to your exception list (for example, blog.paloaltonetworks.com and paloaltonetworks.com/blog) if the original entry is accessible from more than URL.
- The entry example.com is distinct from www.example.com. The domain name is the same, but the second entry contains the www subdomain.
Palo Alto Networks does not support regular expression
use in custom URL category or external dynamic list entries. You
must know the specific URLs or be able to construct the URL patterns
you want to match using wildcards and the following characters: . / ? & = ; +.
Wildcard Guidelines for URL Category Exception Lists
You can use asterisks (*) and carets (^) in URL category
exception lists to configure a single entry to match multiple subdomains,
domains, top-level domains (TLD), or pages without specifying exact
URLs.
Add a trailing slash (/) to domain entries to ensure the
firewall ignores matches to the right of the domain. For the trailing
slash to have an impact, an entry must not end in a *.
How to Use Asterisk (*) and Caret (^) Wildcards
The following
characters are token separators: . / ? & = ; +.
Every string separated by one or two of these characters is a token. Use
wildcard characters as token placeholders to indicate that a specific
token can contain any value. In the entry docs.paloaltonetworks.com, the
tokens are “docs”, “paloaltonetworks”, and “com”.
The following
table describes how asterisks and carets work and provides examples.
* | ^ |
---|---|
Indicates one or more variable subdomains,
domains, TLDs, or subdirectories. Can use asterisk after trailing
slash, for example, example.com/*. Ex: *.domain.com matches docs.domain.com and abc.xyz.domain.com. | Indicates one variable subdomain, root domain,
or TLD. Cannot use caret after . The following entry is invalid: example.com/^. Ex: ^.domain.com matches docs.domain.com and blog.domain.com. |
Key Point: Asterisks
match a greater range of URLs than carets. An asterisk corresponds
to any number of consecutive tokens, while a caret corresponds to
exactly one token. An entry like xyz.*.com matches
more sites than xyz.^.^.com; xyz.*.com matches
sites with any number of tokens between the strings, and xyz.^.^.com matches sites
with exactly two tokens. The firewall interprets entries
that do not end in a / or * with an implicit asterisk to their
end, which further increases the potential matches. |
- A wildcard must be the only character within a token. For example, example*.com is an invalid entry because example and * are in the same token. An entry can contain wildcards in more than one token, however.
- You can use asterisks and carets in the same entry (for example, *.example.^).
Do not create an entry with consecutive
asterisks (*) or more than nine consecutive carets (^)—entries like
these can affect firewall performance.
For example, do
not add an entry like mail.*.*.com. Instead,
depending on the range of websites you want to control access to,
enter mail.*.com or mail.^.^.com.
URL Category Exception List—Examples
The following table displays example URL list entries,
matching sites, and explanations for the matching behavior. URL Category Exceptions—Examples (starting
at PAN-OS 10.2) shows matching behavior when the firewall appends
a trailing slash to domain entries without a trailing slash or terminating
asterisk by default.
URL Exception List Entry | Matching Sites | Explanation and Notes |
---|---|---|
Example Set 1 | ||
paloaltonetworks.com | paloaltonetworks.com paloaltonetworks.com.au paloaltonetworks.com.random.org paloaltonetworks.com/your-page | An implicit asterisk is assumed at the end
of the domain because a trailing slash is not present. Therefore,
matches also include all URLs that begin with the domain. Add a trailing slash to your entry to limit
matches to the exact domain and its subdirectories. |
paloaltonetworks.com/example | paloaltonetworks.com/example | The domain is followed by the subdirectory example.
When you enter the URL for a specific web page, the firewall applies
the exception action to the specified web page. |
Example Set 2—Asterisks | ||
*.example.com | www.example.com docs.example.com support.tools.example.com.uk blog.example.com/your-page | The asterisk expands matches to all example.com subdomains. An
implicit asterisk is assumed at the end of the domain because a
is not present. Therefore, matches also include URLs that begin with
the pattern <subdomain>.example.com. To ensure your entry matches only subdomains
of the domain, include a . |
mail.example.* | mail.example.com mail.example.co.uk mail.example.com/#inbox | The asterisk expands matches to any URL
following the mail.example.<TLD> pattern. An
asterisk is not implied at the end because the entry already ends
in one. |
example.*.com | example.yoursite.com example.es.domain.com example.a.b.com.info.us example.company.com/1234 | The asterisk expands matches to URLs where example is
the left-most subdomain and the last token represented by the asterisk
is followed by .com (example.<domain>.com). An
implicit asterisk is assumed at the end of the domain because a
is not present. |
example.com/* | example.com/photos example.com/blog/latest any
example.com subdirectory | The domain is followed by a / and
an asterisk, which indicates that a subdirectory must be present. The
asterisk serves as a token placeholder for any example.com subdirectory. An
asterisk is not implied at the end because the entry ends in one. |
Example Set 3—Carets | ||
google.^ Patterns such as example.co.^ are
typically used to match country-specific domains such as example.co.jp.
However, generic top-level domains (gTLDs) result in patterns such as
example.co.^ matching example.co.info or example.co.amzn, which
may not belong to the same organization. | google.com google.co.uk google.com/search?q=paloaltonetworks | An implicit asterisk is assumed to the right
of the caret because a trailing slash is not present. Therefore, matches
also include URLs with more than one token after google. Add a trailing slash to limit matches to sites
with one token after the domain. |
^.google.com | www.google.com news.google.com.test.info docs.google.com/document | The caret expands matches to single-level subdomains
of google.com. An implicit asterisk
is assumed at the end of the domain because a trailing slash is
not present. Therefore, matches also include URLs that begin with
the <subdomain>.google.com pattern. |
^.^.google.com | www.maps.google.com support.tools.google.com.abc.xyz www.tools.google.com/example-page | The two carets expand matches to URLs with
two consecutive subdomains before google.com. An
implicit asterisk is assumed at the end of the domain because a
trailing slash is not present. Therefore, matches also include URLs
that begin with the <subdomain>.<subdomain>.google.com pattern. |
google.^.com | google.example.com google.company.com.it google.info.com/example | The caret expands matches to URLs where google is
the left-most subdomain, followed by one token and .com. An
implicit asterisk is assumed at the end of the domain because a
trailing slash is not present. Therefore, matches also include URLs
that begin with the google.<domain>.com pattern. |