: Global URL Analysis
Focus
Focus

Global URL Analysis

Table of Contents

Global URL Analysis

The initial release of URL analysis in July 2020 was only available to users connecting their firewall to the WildFire global cloud (U.S.). This update allows all regional cloud users to access this feature.
Palo Alto Networks now provides improved URL analysis capabilities for all WildFire global and regional clouds, by delivering standardized web page verdicts and reports through the API, as well as enhanced malicious email link detection on the firewall. Not only does this generate a more accurate verdict by aggregating threat analysis details from all Palo Alto Networks services, but it also provides consistent URL analysis data, regardless of which Palo Alto Networks products you rely on to protect your network.
The URL analyzers operating in the WildFire global cloud processes URL feeds, correlated URL sources (such as email links), NRD (newly registered domain) lists, PAN-DB content, and manually uploaded URLs, to provide all WildFire clouds with the improved capabilities, without affecting GDPR compliance. After a URL has been processed, you can retrieve the WildFire URL analysis report, which includes the verdict, detection reasons with evidence, screenshots, and analysis data generated for the web request. You can also retrieve web page artifacts (downloaded files and screenshots) seen during URL analysis to further investigate anomalous activity. The new enhancements found in the URL analysis service enables WildFire to play a larger role in defending your network by supporting your SOC and incident response teams with more accurate verdicts and better visibility into URL analysis.
No additional configuration is necessary to take advantage of this feature, however, if you want to automatically submit email links for analysis (which are now analyzed through this service), you must configure your firewall to forward email links.
Verdicts that you suspect are either false positives or false negatives can be submitted to the Palo Alto Networks threat team for additional analysis.
Important information about WildFire URL Analysis.
  • The WildFire portal currently does not allow retrieval of reports or submissions of web page URLs.
  • WildFire reports are not currently available on the firewall.
You can use the WildFire API to retrieve URL analysis reports, verdicts, and related web artifacts. The following table describes the new and updated API endpoints that are now available.
API Resource
Description
XML Response or Additional Info
Updated API Endpoints
/get/verdict
* Updates do not apply to the /get/verdicts endpoint
Get a verdict for a specified web page url.
<wildfire> <get-verdict-info> <url>http://www.google.com</url> <verdict>0</verdict> <analysis_time>2020-06-29T16:33:17Z</analysis_time> <valid>Yes/valid> </get-verdict-info </wildfire>
Using a hash value to retrieve a web page verdict, instead of the new url parameter, can yield inaccurate results. This is because API requests using the url parameter retrieve verdicts that have been processed using URL analysis, while hash requests retrieve verdicts through the legacy analyzer. Palo Alto Networks recommends using the url parameter when retrieving web page verdicts for the most accurate and up to date information.
The verdict ID number is as follows:
  • 0: Benign
  • 1: Malware
  • 2: Grayware
  • 4: Phishing
  • 5: C2 (New)
WildFire Submissions that have been classified with the newly introduced verdict of C2 are currently only displayed in WildFire API reports and verdict queries. The firewall does not currently support the C2 verdict; consequently, URLs classified with the C2 verdict are shown as malware.
The valid entry in the response indicates whether or not the verdict is up-to-date. URLs that have not been analyzed recently are considered obsolete and are designated as being no longer valid.
/get/report
Get a JSON report of analysis results for a specified url.
When using the new url parameter, the API attempts to find an exact match of the specified url. If none is found, WildFire delivers a best guess match. The match is indicated by the url_type entry in the XML response. original indicates an exact match, while best_match is shown for the closest match found by URL analysis.
{ "success": true, "result": { "analysis_time": "2020-04-22:42:30Z", "url_type": "original", "report": "<MAEC report>"/"", }
Using a hash value to retrieve a web page report, instead of a URL, can yield differing results. This is because API requests using the url parameter retrieve reports that have been processed using URL analysis, while hash requests retrieve verdicts through the legacy analyzer service. Palo Alto Networks recommends using the url parameter when retrieving web page reports for the most accurate information.
The following API endpoints do not support URL analysis functionality at this time: /get/pcap and /get/verdicts.
New API Endpoints
/get/webartifacts
Get web artifacts associated with a specified URL.
The XML response downloads a .tgz file package which includes all of the requested web artifacts. A field in the response header displays the time and date of the last URL analysis execution:
Last-Modified: Fri Apr 3 19:18:09 2020