Step 1: Review the alert details and record the affected ION, interface, duplicate IPv4 address, and conflicting device’s MAC address.

Step 2: Determine whether the alert occurred during or immediately after an ION RMA replacement.

Step 3: If this is an RMA replacement, confirm that the original ION is offline. Verify that any preconfigured static IPv4 address on the replacement ION is unique and excluded from the active DHCP pool.

Step 4: For a non-RMA event, go to Configuration > Prisma SD-WAN > ION Devices > Claimed, select the affected interface, and determine whether its IPv4 address is statically or dynamically assigned.

You can also verify the interface configuration from the CLI:

dump interface config <interface>

Step 5: If the ION uses a static IPv4 address, confirm that the address:

Is not statically configured on another device.

Is not included in an active DHCP address pool.

Is correctly reserved or excluded on the DHCP server, if applicable.

Step 6: If the address was assigned through DHCP, review the DHCP server’s leases, reservations, and address pool. Remove any conflicting static assignment or incorrect reservation and ensure that the ION receives a unique address.

Step 7: Use the conflicting MAC address from the alert to locate the other device through the connected switch’s MAC address table or network inventory.

Step 8: Correct the duplicate assignment by changing the IPv4 address on either the ION or the conflicting device. If the affected interface provides controller, WAN, or production connectivity, perform the change during an appropriate maintenance window.

Step 9: If additional verification is needed, run:

inspect system arp interface=<interface>

Confirm that the duplicate IPv4 address is no longer associated with the conflicting MAC address.

Step 10: Confirm that the ION remains reachable and that the alert clears. If the alert remains active after the duplicate assignment is corrected, save the alert details and CLI outputs and contact Palo Alto Networks Support.

Step 1: Review the incident details and record the affected site, ION, incident time, software version, the affected cellular interface, the carrier-negotiated MTU, and the configured MTU.

Step 2: Run dump cellular status all to review the current negotiated MTU and active data session parameters.

Step 3: In SCM, go to Configuration > Prisma SD-WAN > ION Devices > Claimed > [Device] > Interfaces > Cellular Interfaces and review the configured MTU for the affected interface.

Step 4: If the carrier-negotiated MTU is consistently lower than the configured value, update the interface MTU in SCM to match the carrier-negotiated value and push the configuration to the device.

Step 5: Monitor the cellular interface to confirm the incident clears after the configuration update.

Step 6: If the mismatch persists or traffic issues continue, run dump-support all file=<descriptive_filename> (Release 6.4.1 and later) or dump-support outputs file=<descriptive_filename> (earlier releases), where <descriptive_filename> is a user-selected name such as event_name_ion-name_YYYYMMDDTHHMM.

Step 1: Review the incident details and record the affected site, ION, incident time, software version, and the specific internet circuit that is down.

Step 2: Check the physical WAN interface status for the affected circuit using dump interface status interface <port>.

Step 3: Review related DEVICEHW_INTERFACE_DOWN or NETWORK_ANYNETLINK_DOWN incidents at the same site during the same period.

Step 4: Confirm whether the circuit outage is due to a local cabling issue, hardware fault, or ISP or carrier outage.

Step 5: If the outage is carrier or ISP related, contact the carrier with the circuit details and outage start time.

Step 6: In SCM, verify the circuit configuration and confirm that no recent configuration change caused the outage.

Step 7: Monitor the circuit to confirm it returns to operational status and the incident clears.

Step 8: If the circuit does not recover, run dump-support all file=<descriptive_filename> (Release 6.4.1 and later) or dump-support outputs file=<descriptive_filename> (earlier releases), where <descriptive_filename> is a user-selected name such as event_name_ion-name_YYYYMMDDTHHMM.

Step 1: Review the incident details and record the affected site, ION, incident time, software version, and the specific private WAN circuit that is down.

Step 2: Check the physical WAN interface status for the affected circuit:

dump interface status interface <port>

Use the interface associated with the affected private WAN circuit as <port>.

Step 3: Review related DEVICEHW_INTERFACE_DOWN or NETWORK_ANYNETLINK_DOWN incidents at the same site during the same period.

Step 4: Confirm whether the circuit outage is caused by a local cabling issue, hardware fault, or private WAN carrier or service-provider outage.

Step 5: If the outage is carrier-related, contact the private WAN carrier or service provider with the circuit details and outage start time.

Step 6: In SCM, verify the private WAN circuit configuration and confirm that no recent configuration change caused the outage.

Step 7: Monitor the private WAN circuit and confirm that it returns to operational status and the incident clears.

Step 8: If the circuit does not recover, collect a support bundle and open a Palo Alto Networks Support case:

dump-support all file=<descriptive_filename>

For releases earlier than 6.4.1 that do not support the all option:

dump-support outputs file=<descriptive_filename>

Use a filename containing the event, ION name, and collection time, such as directprivate_down_branch-ion1_YYYYMMDDTHHMM.

Step 1: Review Detection Details and open the correlated incidents to identify the affected sites, circuits, WAN interfaces, and incident times.

Step 2: For each affected site, navigate to Site > Site Summary and review the Link Quality Metrics for the affected circuit and WAN interface during the incident period.

Step 3: Check latency, jitter, packet loss, and MOS in both directions to identify which metrics exceeded the configured Performance SLA thresholds.

Step 4: Review circuit and bandwidth utilization to determine whether congestion contributed to the degradation. Also verify the local WAN interface, cabling, and provider equipment.

Step 5: Review the applicable Performance Policy Rule and SLA thresholds. Confirm that the configured thresholds are appropriate for the affected carrier and circuit type.

Step 6: If the same carrier shows consistent SLA violations across multiple sites, contact the service provider. Provide the affected sites, circuits, WAN interfaces, incident times, and degraded link-quality metrics.

Step 7: After remediation, confirm that the link-quality metrics have returned to acceptable levels and that the correlated circuit incidents and carrier incident are cleared.

Step 8: If the issue persists, collect the incident details, correlated circuit incidents, and link-quality metrics, and contact Palo Alto Networks Support.

Step 1: Review the incident details and note the affected site, circuit, WAN interface, Performance Policy Rule, Performance Policy Set, and incident time.

Step 2: Navigate to Site > Site Summary and review the Link Quality Metrics for the affected circuit and WAN interface during the incident period.

Step 3: Check latency, jitter, and packet loss in both directions to identify which metric exceeded the SLA threshold defined in the performance policy.

Step 4: Review bandwidth utilization to determine whether congestion contributed to the degradation. Also verify the local WAN interface, cabling, and service-provider equipment.

Step 5: Confirm that the Performance Policy Rule and its SLA thresholds are appropriate for the affected circuit. Do not modify the thresholds only to suppress the incident.

Step 6: If the circuit consistently violates the SLA, contact the service provider and provide the affected site, circuit, WAN interface, incident time, and degraded link-quality metrics.

Step 7: After remediation, confirm that the link-quality metrics have returned to acceptable levels and that the incident is cleared.

Step 8: If the issue persists, collect the incident details and link-quality metrics, and contact Palo Alto Networks Support.

Step 1: Review the incident details and record the affected site, ION, incident time, software version, home carrier, and the roaming carrier detected.

Step 2: Run dump cellular status all to confirm the current carrier name, MCC, MNC, and roaming status.

Step 3: Verify whether roaming is expected at this site based on carrier coverage and the device location.

Step 4: If roaming is not expected, confirm that the SIM is provisioned for the correct carrier and coverage area with your carrier administrator.

Step 5: If roaming continues unexpectedly or causes service degradation, run dump-support all file=<descriptive_filename> (Release 6.4.1 and later) or dump-support outputs file=<descriptive_filename> (earlier releases), where <descriptive_filename> is a user-selected name such as event_name_ion-name_YYYYMMDDTHHMM.

Step 1: Review the incident details and record the affected site, ION, incident time, software version, the affected cellular interface, and the measured signal strength reported with the incident.

Step 2: Run dump cellular stats all to review the current RSRP, RSRQ, RSSI, and SNR values and compare them with the incident values.

Step 3: Inspect the ION installation and cellular antenna. Verify that the antenna is connected correctly, positioned appropriately, and unobstructed.

Step 4: Review signal history in SCM under Insights > Branch Sites > Prisma SD-WAN > Activity > Cellular to determine whether the condition is intermittent or persistent.

Step 5: If signal strength remains low, consult the carrier about local coverage or consider repositioning the antenna.

Step 6: If the incident persists or materially affects service, collect a support bundle and open a Palo Alto Networks Support case. Run dump-support all file=<descriptive_filename> for Release 6.4.1 and later or dump-support outputs file=<descriptive_filename> for earlier releases.

Step 1: Review the incident details and record the affected site, ION, cellular interface, incident time, software version, previous SIM, new active SIM, and whether the switchover was expected.

Step 2: Run dump cellular status all and dump cellular stats all to confirm the active SIM, carrier registration, signal state, and data-session status.

Step 3: If the switchover was expected and connectivity is operational, no corrective action is required; continue monitoring.

Step 4: If the switchover was unexpected, review related SIM, signal, roaming, modem, and carrier incidents to determine why the previous SIM lost service.

Step 5: Verify the intended SIM and failover configuration in SCM under Configuration > Prisma SD-WAN > ION Devices > Claimed. Select the device and review the cellular module Main Configuration.

Step 6: Correct any SIM or carrier issue and confirm that cellular connectivity remains stable on the intended active SIM.

Step 7: If unexpected switchovers recur or service remains impaired, collect a support bundle and open a Palo Alto Networks Support case.

Step 1: Review the incident details and record the affected site, ION, incident time, software version, the previous technology mode, and the new technology detected.

Step 2: Run dump cellular status all to confirm the current radio access technology, carrier name, MCC, and MNC.

Step 3: Run dump cellular stats all to review current signal strength metrics and data session information.

Step 4: Verify whether the technology change was expected due to carrier infrastructure updates or the device's site location.

Step 5: Review signal strength history in SCM under Insights > Branch Sites > Prisma SD-WAN > Activity > Cellular to determine whether the change is persistent or intermittent.

Step 6: If the technology downgrade is unexpected, persistent, or is causing service degradation, consult the carrier and run dump-support all file=<descriptive_filename> (Release 6.4.1 and later) or dump-support outputs file=<descriptive_filename> (earlier releases), where <descriptive_filename> is a user-selected name such as event_name_ion-name_YYYYMMDDTHHMM.

Step 1: Review the incident details and record the affected site, ION, circuit identifier, incident time, software version, and current utilization relative to the configured threshold.

Step 2: In SCM, go to Configuration > Prisma SD-WAN > Policies > Performance > Performance Sets, open the applicable System Health rule, and review the circuit utilization threshold configured for the affected site or circuit.

Step 3: Review circuit utilization trends in SCM under Insights > Prisma SD-WAN to determine whether the exceedance is a sustained trend or a temporary burst.

Step 4: Identify the applications or traffic flows contributing most to circuit utilization.

Step 5: If high utilization is expected and persistent, consider increasing circuit capacity, adjusting the Performance Policy threshold, or applying QoS policies to prioritize critical traffic.

Step 6: Monitor circuit utilization to confirm it returns below the threshold.

Step 7: If the utilization remains unexpectedly high or causes service impact, run dump-support all file=<descriptive_filename> (Release 6.4.1 and later) or dump-support outputs file=<descriptive_filename> (earlier releases), where <descriptive_filename> is a user-selected name such as event_name_ion-name_YYYYMMDDTHHMM.

Step 1: Review the incident details and record the affected site, ION, incident time, software version, and current flow count relative to the configured threshold.

Step 2: In SCM, go to Configuration > Prisma SD-WAN > Policies > Performance > Performance Sets, open the applicable System Health rule, and review the concurrent flow threshold configured for the affected ION.

Step 3: Review flow count trends to determine whether the exceedance is a persistent issue or a temporary burst.

Step 4: Identify applications or sources generating the most concurrent flows.

Step 5: If the flow count is consistently high and legitimate, review the threshold setting and consider adjusting it within the device's supported limits.

Step 6: If the flow count is unexpectedly high, investigate whether a misconfigured application or scanning activity is generating excess flows.

Step 7: Monitor the flow count to confirm it returns below the threshold.

Step 8: If the incident persists or service is affected, run dump-support all file=<descriptive_filename> (Release 6.4.1 and later) or dump-support outputs file=<descriptive_filename> (earlier releases), where <descriptive_filename> is a user-selected name such as event_name_ion-name_YYYYMMDDTHHMM.

Step 1: Review the incident details and record the affected site, ION, incident time, software version, and the NAT stack, NAT set, and rule containing the ALG action.

Step 2: In SCM, go to Configuration > Prisma SD-WAN > Policies > NAT > NAT Stacks > Advanced > NAT Sets. Open the applicable NAT set and confirm the ALG action configured in the affected rule.

Do not change the NAT policy solely to clear this incident. The override is caused by a legacy configuration present on the ION, not by NAT policy rule priority.

Step 3: Collect a support bundle:

dump-support all file=<descriptive_filename>

For releases earlier than 6.4.1 that do not support the all option:

dump-support outputs file=<descriptive_filename>

Use a filename containing the event, ION name, and collection time, such as legacy_alg_override_branch-ion1_20260820T1430.

Step 4: Open a Palo Alto Networks Support case and request removal of the legacy ALG configuration from the ION. Include the incident details, applicable NAT policy information, and support bundle.

Step 5: After Support removes the legacy configuration, confirm that the incident clears and that the intended NAT policy ALG action is applied.

Step 1: Review the incident details and record the affected site, ION, incident time, software version, private WAN circuit, affected peer sites, and unreachable prefixes.

Step 2: Run the following read-only commands:

dump routing peer status all

dump routing peer received-routes all

dump routing peer advertised-routes all

Confirm that the applicable BGP peer is established. Compare the received and advertised routes and identify the expected prefixes that are missing.

Step 3: In SCM, review the BGP peer configuration associated with the affected private WAN connection, including the peer address, remote AS number, address family, VRF, prefix filters, and route maps.

Step 4: Verify that the routing device connected to the private WAN circuit is advertising the expected prefixes. Confirm that the private WAN provider is not filtering or summarizing the routes in a way that removes the required prefixes.

Step 5: Review related PEERING_BGP_DOWN, PEERING_CORE_DOWN, or PEERING_EDGE_DOWN incidents at the affected site during the same period, as applicable.

Step 6: Correct any ION-side BGP, prefix-filter, or route-map configuration through SCM. If the missing routes originate from an upstream routing device or the private WAN provider, work with the device owner or provider to correct the route advertisement.

Step 7: Run the routing commands again and confirm that the expected prefixes are received, the affected destinations are reachable, and the incident clears.

Step 8: If the degradation persists, collect a support bundle and open a Palo Alto Networks Support case:

dump-support all file=<descriptive_filename>

For releases earlier than 6.4.1 that do not support the all option:

dump-support outputs file=<descriptive_filename>

Use a filename containing the event, ION name, and collection time, such as privatewan_degraded_branch-ion1_YYYYMMDDTHHMM.

Step 1: Review the incident details and record the affected site, ION, incident time, software version, private WAN circuit, and unreachable peer sites or prefixes.

Step 2: Review related DEVICEHW_INTERFACE_DOWN, NETWORK_DIRECTPRIVATE_DOWN, PEERING_BGP_DOWN, PEERING_CORE_DOWN, or PEERING_EDGE_DOWN incidents at the affected site during the same period, as applicable.

Step 3: Check the physical interface associated with the affected private WAN circuit:

dump interface status interface <port>

Use the interface associated with the affected private WAN circuit as <port>. Confirm that the interface is operational and that no physical errors are reported.

Step 4: Run the following read-only commands:

dump routing peer status all

dump routing peer received-routes all

dump routing peer advertised-routes all

Confirm that the applicable BGP peer is established and that the routes required to reach the affected peer sites are being received and advertised.

Step 5: In SCM, review the private WAN circuit and BGP peer configuration, including the peer address, remote AS number, address family, VRF, prefix filters, and route maps.

Step 6: Verify that the upstream private WAN routing device is operational and advertising the expected prefixes. If the routes are missing from the provider network, work with the private WAN carrier or service provider.

Step 7: Correct any ION-side circuit, BGP, prefix-filter, or route-map configuration through SCM. Coordinate with the upstream routing-device owner or private WAN provider for changes outside SCM.

Step 8: Run the interface and routing commands again and confirm that the affected peer sites are reachable and the incident clears.

Step 9: If the unreachability persists, collect a support bundle and open a Palo Alto Networks Support case:

dump-support all file=<descriptive_filename>

For releases earlier than 6.4.1 that do not support the all option:

dump-support outputs file=<descriptive_filename>

Use a filename containing the event, ION name, and collection time, such as privatewan_unreachable_branch-ion1_YYYYMMDDTHHMM.

Step 1: Review the incident details and record the affected site, ION, incident time, software version, and BGP peer that is down.

Step 2: In SCM, go to Configuration > Prisma SD-WAN > ION Devices > Claimed, select the device, and select Configure the Device > Routing > BGP/Peers. Review the peer address, local and remote AS numbers, address family, authentication, timers, source interface, and routing policy.

Step 3: Run dump routing peer status all and confirm the affected peer state and reported reason. Run dump routing peer received-routes all and dump routing peer advertised-routes all to review route exchange after the session is established.

Step 4: Verify connectivity from the ION to the peer address and review related interface, circuit, or routing incidents.

Step 5: Confirm that peer address, AS numbers, authentication, timers, and address-family settings match the peer configuration.

Step 6: Correct any ION-side configuration through SCM, coordinate remote-side changes with the peer administrator, and confirm that the session becomes Established.

Step 7: If PEERING_BGP_DOWN remains active, collect a support bundle and open a Palo Alto Networks Support case.

Step 1: Review the incident details and record the affected site, ION, incident time, software version, and core peer that is down.

Step 2: In SCM, go to Configuration > Prisma SD-WAN > ION Devices > Claimed, select the device, and select Configure the Device > Routing > BGP/Peers. Review the peer address, local and remote AS numbers, address family, authentication, timers, source interface, and routing policy.

Step 3: Run dump routing peer status all and confirm the affected peer state and reported reason. Run dump routing peer received-routes all and dump routing peer advertised-routes all to review route exchange after the session is established.

Step 4: Verify connectivity from the ION to the core-device address and review related interface, circuit, or routing incidents.

Step 5: Confirm that peer address, AS numbers, authentication, timers, and address-family settings match the core-device configuration.

Step 6: Correct any ION-side configuration through SCM, coordinate remote-side changes with the peer administrator, and confirm that the session becomes Established.

Step 7: If PEERING_CORE_DOWN remains active, collect a support bundle and open a Palo Alto Networks Support case.

Step 1: Review the incident details and record the affected site, ION, incident time, software version, and WAN edge peer that is down.

Step 2: In SCM, go to Configuration > Prisma SD-WAN > ION Devices > Claimed, select the device, and select Configure the Device > Routing > BGP/Peers. Review the peer address, local and remote AS numbers, address family, authentication, timers, source interface, and routing policy.

Step 3: Run dump routing peer status all and confirm the affected peer state and reported reason. Run dump routing peer received-routes all and dump routing peer advertised-routes all to review route exchange after the session is established.

Step 4: Verify connectivity from the ION to the WAN-edge address and review related interface, circuit, or routing incidents.

Step 5: Confirm that peer address, AS numbers, authentication, timers, and address-family settings match the WAN-edge configuration.

Step 6: Correct any ION-side configuration through SCM, coordinate remote-side changes with the peer administrator, and confirm that the session becomes Established.

Step 7: If PEERING_EDGE_DOWN remains active, collect a support bundle and open a Palo Alto Networks Support case.

Step 1: Review the incident details and record the affected site, ION, incident time, software version, Standard VPN endpoint, and service link interfaces that are down.

Step 2: In SCM, review the Standard VPN service link configuration for the affected endpoint. Verify the endpoint address, Standard VPN protocol, parent interface, local address, peer address, and IPsec profile or GRE configuration, as applicable.

Step 3: Run the following read-only command:

dump servicelink summary all

Identify the service links associated with the affected endpoint. Review the SlDev, SlName, Status, ExtState, ParentDev, LocalIP, Peer, Type, and IPsecProfile fields.

Step 4: For each down service link, use the SlDev value from dump servicelink summary all and run:

dump servicelink status sldev=<SlDev>

For example:

dump servicelink status sldev=sl1

Review the detailed Standard VPN configuration, tunnel status, IKE and ESP information, authentication state, and reported failure reason.

Step 5: Use the ExtState reported by dump servicelink summary all to identify the likely cause:

For parent_no_ip or lowerlayerdown, check the parent interface using:

dump interface status interface <ParentDev>

Also review related DEVICEHW_INTERFACE_DOWN, NETWORK_DIRECTINTERNET_DOWN, or NETWORK_DIRECTPRIVATE_DOWN incidents.

For peer_address_unresolved or invalid_service_endpoint, verify the configured endpoint and peer address in SCM.

For retransmit_send, retransmit_send_timeout, or another no-response condition, verify routing and confirm that any upstream or northbound firewall permits communication with the Standard VPN endpoint. Confirm that the remote endpoint is operational and accepting connections.

For proposal_mismatch_ike, proposal_mismatch_child, or transform_selector_mismatch, compare the IKE, IPsec, and traffic-selector settings on both sides.

For an authentication or certificate error, verify the authentication parameters and certificate validity on both sides.

Step 6: Correct any ION-side service link, endpoint, parent-interface, routing, or IPsec configuration through SCM. Coordinate with the Standard VPN endpoint administrator for changes required on the remote endpoint.

Step 7: Run the following command again:

dump servicelink summary all

Confirm that at least one service link to the endpoint is operational and that the incident clears.

Step 8: If the endpoint remains down, collect a support bundle and open a Palo Alto Networks Support case:

dump-support all file=<descriptive_filename>

For releases earlier than 6.4.1 that do not support the all option:

dump-support outputs file=<descriptive_filename>

Use a filename containing the event, ION name, and collection time, such as standard_vpn_endpoint_down_branch-ion1_YYYYMMDDTHHMM.

Step 1: Review the incident details and record the affected sites, ION identifiers, incident time, software version, and the VPN link affected.

Step 2: If running Release 5.4.1 or later, also review the associated NETWORK_ANYNETLINK_DOWN or NETWORK_ANYNETLINK_DEGRADED incident for the same Secure Fabric Link and follow the remediation steps for that incident.

Step 3: Run dump vpn summary all to review the current state of all VPN paths.

Step 4: Check the physical WAN interface and carrier connectivity for the affected circuit.

Step 5: Review related DEVICEHW_INTERFACE_DOWN or NETWORK_DIRECTINTERNET_DOWN incidents at the same site.

Step 6: If the BFD failure is due to high latency or packet loss on the underlying circuit, address the circuit issue with the carrier or ISP.

Step 7: If the incident persists, run dump-support all file=<descriptive_filename> (Release 6.4.1 and later) or dump-support outputs file=<descriptive_filename> (earlier releases), where <descriptive_filename> is a user-selected name such as event_name_ion-name_YYYYMMDDTHHMM.

Step 1: Review the incident details and record the affected site, ION, incident time, software version, and any related controller-connectivity or VPN incidents.

Step 2: Run dump controller status and confirm whether the ION is connected, partially connected, or not connected to the controller. Determine whether controller communication was unavailable for more than three days.

Step 3: If the ION is not fully connected, run debug controller reachability <interface>, where <interface> is the ION interface used for controller access. Review the output for DNS, routing, certificate, gateway, or connection errors.

Step 4: Verify that the controller-facing interface is operational and that DNS, routing, NAT, and firewall policies permit the ION to communicate with the Prisma SD-WAN controller. Make any required configuration corrections through SCM.

Step 5: Run dump vpn summary all to review the current VPN-link states. If a VPN is down, obtain its VPN ID from the command output and run dump vpn status VpnID=<vpn ID> to review the reported reason.

Step 6: After controller connectivity is restored, run dump controller status again and confirm that the ION is connected. Monitor the affected VPN links and confirm that the KEK incident clears.

Step 7: If the KEK remains unavailable after controller connectivity is restored, collect a support bundle and open a Palo Alto Networks Support case:

dump-support all file=<descriptive_filename>

For releases earlier than 6.4.1 that do not support the all option:

dump-support outputs file=<descriptive_filename>

Step 1: Review the incident details and record the two affected sites, ION identifiers, incident time, software versions, and the affected Secure Fabric link.

Step 2: In SCM, open each affected site, select the Configuration tab, and under Branch Site Details select Configure Ciphers. Compare the Secure Fabric ciphers enabled at both sites.

Step 3: Confirm that both sites have at least one common cipher supported by the software versions running on their IONs. AES-GCM requires ION software release 6.5.1 or later.

Step 4: If no common cipher is enabled, update the site cipher configuration through SCM so that both sites share at least one supported cipher, and then save the changes. The controller selects the best common cipher and applies it to both sites.

Step 5: Run dump vpn summary all and confirm that the affected Secure Fabric tunnel is established. If the VPN remains down, obtain its VPN ID from the output and run dump vpn status VpnID=<vpn ID> to review the reported reason.

Step 6: If a common cipher is configured but the tunnel is not established, collect a support bundle and open a Palo Alto Networks Support case:

dump-support all file=<descriptive_filename>

For releases earlier than 6.4.1 that do not support the all option:

dump-support outputs file=<descriptive_filename>

Step 1: Review the incident details and record the affected sites, ION identifiers, incident time, software version, and the VPN link that is down.

Step 2: If running Release 5.4.1 or later, also review the associated NETWORK_ANYNETLINK_DOWN or NETWORK_ANYNETLINK_DEGRADED incident for the same Secure Fabric Link and follow the remediation steps for that incident.

Step 3: Run dump vpn summary all to review the current state of all VPN paths.

Step 4: Check the WAN interface and carrier connectivity for the affected circuit.

Step 5: Review related DEVICEHW_INTERFACE_DOWN or NETWORK_DIRECTINTERNET_DOWN incidents at either site.

Step 6: If the VPN link remains down after addressing identified issues, run dump-support all file=<descriptive_filename> (Release 6.4.1 and later) or dump-support outputs file=<descriptive_filename> (earlier releases), where <descriptive_filename> is a user-selected name such as event_name_ion-name_YYYYMMDDTHHMM.

Step 1: Review the incident details and record the local and remote sites, ION identifiers, incident time, software versions, and the affected VPN link.

Step 2: For Release 5.4.1 and later, review the associated NETWORK_ANYNETLINK_DOWN or NETWORK_ANYNETLINK_DEGRADED incident and follow its remediation. If necessary, enable Include Suppressed in the incident filters to view the underlying VPN peer incident.

Step 3: Run dump vpn summary all and identify the affected VPN using the remote site, circuit, and link status. Record the VPN ID shown in the output.

Step 4: Run dump vpn status VpnID=<vpn ID>, where <vpn ID> is obtained from dump vpn summary all. Review the link status, usability, remote peer address and port, and reported reason.

Step 5: Review related incidents at the remote site, particularly NETWORK_VPNLINK_DOWN, PEERING_CORE_DOWN, DEVICESW_GENERAL_PROCESSSTOP, DEVICESW_DISCONNECTED_FROM_CONTROLLER, DEVICEHW_INTERFACE_DOWN, and SITE_CONNECTIVITY_DOWN.

Step 6: On the remote ION, run dump overview and dump controller status. Confirm that the ION is operational and connected to the controller. Check the WAN interface associated with the affected VPN using dump interface status interface <port>, where <port> is the WAN interface assigned to the affected circuit in SCM.

Step 7: If the remote ION is operational but the VPN peer remains unavailable, confirm that the public IP address, or external NAT address, configured for each VPN endpoint is reachable from the opposite site. Verify that the peer IP addresses and NAT configuration are correct.

Confirm that the northbound firewalls permit the required UDP port 4500 IPsec traffic and ESP traffic using IP protocol 50 between the VPN peer addresses. If an ION is behind NAT, confirm that the External NAT Address identifies the correct public IP address and that the configured External NAT Port is forwarded to UDP port 4500 on the ION WAN interface. If the public IP address is not reachable or the required traffic is blocked upstream, work with the ISP or private WAN carrier.

Step 8: Correct any identified interface, routing, NAT, firewall, underlay, controller-connectivity, or configuration issue. Make configuration corrections through SCM.

Step 9: Run dump vpn status VpnID=<vpn ID> again and confirm that the VPN link is Up and Usable and that the incident clears.

Step 10: If the remote peer remains unavailable, collect a support bundle and open a Palo Alto Networks Support case:

dump-support all file=<descriptive_filename>

For releases earlier than 6.4.1 that do not support the all option:

dump-support outputs file=<descriptive_filename>

Step 1: Review the incident details and record the local and remote sites, ION identifiers, incident time, software versions, and the affected VPN link.

Step 2: For Release 5.4.1 and later, review the associated NETWORK_ANYNETLINK_DOWN or NETWORK_ANYNETLINK_DEGRADED incident and follow its remediation. If necessary, enable Include Suppressed in the incident filters to view the underlying VPN peer incident.

Step 3: Run dump vpn summary all and identify the affected VPN using the remote site, circuit, and link status. Record the VPN ID shown in the output.

Step 4: Run dump vpn status VpnID=<vpn ID>, where <vpn ID> is obtained from dump vpn summary all. Review the reported reason, remote peer address and port, link status, and usability.

Step 5: Check the WAN interface associated with the affected VPN at both sites using dump interface status interface <port>, where <port> is the WAN interface assigned to the affected circuit in SCM. Review related DEVICEHW_INTERFACE_DOWN, NETWORK_DIRECTINTERNET_DOWN, or NETWORK_DIRECTPRIVATE_DOWN incidents.

Step 6: If the reported reason is Peer Unreachable, confirm that the public IP address, or external NAT address, configured for each VPN endpoint is reachable from the opposite site. Verify that the peer IP addresses, routing, and NAT configuration provide connectivity between the two IONs.

Confirm that the northbound firewalls permit the required UDP port 4500 IPsec traffic and ESP traffic using IP protocol 50 between the VPN peer addresses. If an ION is behind NAT, confirm that the External NAT Address identifies the correct public IP address and that the configured External NAT Port is forwarded to UDP port 4500 on the ION WAN interface.

If the public IP address is not reachable or the required traffic is blocked upstream, work with the ISP or private WAN carrier.

Step 7: Correct any identified interface, routing, NAT, firewall, underlay, or VPN endpoint configuration issue. Make configuration corrections through SCM.

Step 8: Run dump vpn status VpnID=<vpn ID> again and confirm that the VPN link is Up and Usable and that the incident clears.

Step 9: If the VPN peer remains unreachable, collect a support bundle and open a Palo Alto Networks Support case:

dump-support all file=<descriptive_filename>

For releases earlier than 6.4.1 that do not support the all option:

dump-support outputs file=<descriptive_filename>

Step 1: Review the incident details and record the local and remote sites, ION identifiers, incident time, software versions, and affected VPN link.

Step 2: For Release 5.4.1 and later, review the associated NETWORK_ANYNETLINK_DOWN or NETWORK_ANYNETLINK_DEGRADED incident. Enable Include Suppressed if necessary to view the underlying shared-secret incident.

Step 3: Run dump vpn summary all and record the VepID for the affected VPN, then run dump vpn status VpnID=<vpn-id> on both IONs.

Step 4: On both IONs, run dump vpn sharedsecret VpnID=<vpn-id>. Compare Current time and the shared-secret start and end times. Confirm that both peers have an overlapping currently valid window and future windows. Do not expose or attempt to configure any shared-secret value.

Step 5: On both IONs, run dump time config and dump time status. If synchronization is abnormal, review dump time log and related DEVICESW_NTP_NO_SYNC incidents, then correct NTP, DNS, routing, or firewall issues through supported configuration workflows.

Step 6: Run dump controller status on both IONs. If either ION is not fully connected, run debug controller reachability <interface> and correct the reported DNS, routing, gateway, certificate, or connection problem.

Step 7: After time and controller connectivity are healthy, allow the controller to redistribute the valid shared-secret windows. Confirm that both peers show an overlapping current window and that the VPN becomes Up and Usable.

Step 8: If the mismatch persists, collect support bundles from both IONs and open a Palo Alto Networks Support case.

Step 1: Review the incident details and record the affected site, ION, incident time, software version, VPN link, and controller disconnection duration. Review related DEVICESW_DISCONNECTED_FROM_CONTROLLER and NETWORK_VPNKEK_UNAVAILABLE incidents.

Step 2: For Release 5.4.1 and later, review the associated NETWORK_ANYNETLINK incident and enable Include Suppressed if necessary to view the underlying shared-secret incident.

Step 3: Run dump vpn summary all, record the affected VepID, and run dump vpn status VpnID=<vpn-id>.

Step 4: Run dump vpn sharedsecret VpnID=<vpn-id> and compare Current time with the displayed validity windows. Confirm whether no currently valid shared secret or no future validity window is available. Do not expose or attempt to configure any shared-secret value.

Step 5: Run dump time status and dump controller status. If time is incorrect, review dump time config and dump time log. If controller connectivity is impaired, run debug controller reachability <interface> and correct the reported DNS, routing, gateway, certificate, NAT, or firewall issue.

Step 6: After controller connectivity and accurate time are restored, allow the controller to distribute the current and future shared-secret windows automatically.

Step 7: Run dump vpn sharedsecret VpnID=<vpn-id> and dump vpn status VpnID=<vpn-id> again. Confirm that a current window is available and the VPN is Up and Usable.

Step 8: If the shared secret remains unavailable, collect a support bundle and open a Palo Alto Networks Support case.

Step 1: Review the incident details and record the affected tenant, region, service endpoint, NPN or SPN element identifier, incident time, and reported key-distribution error.

Step 2: Review the affected SASE or SPN endpoint status and any related controller-service, endpoint-connectivity, or key-distribution incidents in SCM.

Step 3: If customer traffic is affected, validate branch-to-service reachability and follow any related SITE_CONNECTIVITY, NETWORK_ANYNETLINK, or service-link incident remediation.

Step 4: Do not attempt to view, replace, or configure GENEVE keys on customer ION devices. Key generation and distribution are controller/service functions.

Step 5: If the incident remains active, recurs, or affects service, open a Palo Alto Networks Support case with the controller incident details. An ION support bundle is not required unless Support requests it or a related ION incident is present.

Step 1: Review the incident details and record the affected site, ION, incident time, software version, and the WAN paths, Secure Fabric Links, or service links reported down.

Step 2: Review related NETWORK_DIRECTINTERNET_DOWN, NETWORK_DIRECTPRIVATE_DOWN, NETWORK_ANYNETLINK_DOWN, NETWORK_STANDARD_VPN_ENDPOINT_DOWN, or interface incidents to identify the contributing failure.

Step 3: Run dump vpn summary all to review Secure Fabric VPN paths. If service links are implicated, run dump servicelink summary all.

Step 4: For an affected WAN interface, run dump interface status interface <port> and verify physical state, address assignment, gateway reachability, and carrier status.

Step 5: Follow the remediation for the specific down circuit, VPN, service link, or interface and correct configuration through SCM.

Step 6: Confirm that all WAN paths return to operational status and the degraded-site incident clears.

Step 7: If connectivity cannot be restored, collect a support bundle and open a Palo Alto Networks Support case.

Step 1: Review the incident details and record the affected site, ION, incident time, software version, outage duration, and affected WAN circuits. Review related DEVICESW_DISCONNECTED_FROM_CONTROLLER, NETWORK_ANYNETLINK_DOWN, NETWORK_DIRECTINTERNET_DOWN, NETWORK_DIRECTPRIVATE_DOWN, and DEVICEHW_INTERFACE_DOWN incidents.

Step 2: Because the site is offline, the SCM Remote CLI Toolkit cannot reach the ION. Access the ION through direct SSH from a reachable local or out-of-band management network, or through the physical console. Do not make configuration changes locally through the CLI.

Step 3: Verify that the ION is powered on and check its cabling, link LEDs, and connected ISP modem, router, or other upstream equipment. For each WAN interface, run dump interface status interface <wan-interface> and confirm that the interface is administratively and operationally up and has the expected IP address.

Step 4: Test connectivity through each internet WAN interface. First, ping the WAN gateway, and then ping a known reachable public IP address:

ping <wan-interface> <wan-gateway-ip>

ping <wan-interface> <public-ip-address>

Replace <wan-interface> with the WAN interface name or ID. Use the gateway configured for that circuit and a customer-approved public destination that normally responds to ICMP.

Step 5: If the WAN gateway is unreachable, investigate the local cabling, upstream modem or router, and WAN circuit. If the gateway is reachable but multiple known public destinations are unreachable, check the upstream routing, NAT, or firewall configuration and work with the ISP or carrier to verify the circuit. If a public IP address is reachable but a hostname is not, investigate DNS resolution.

Step 6: If internet reachability is operational, run:

dump controller status

debug controller reachability <wan-interface>

Confirm that the ION can resolve and reach the controller and that the certificate and connection tests pass. If internet access works but the controller remains unreachable, investigate DNS, routing, and the upstream firewall or proxy between the ION and the controller.

Step 7: After verifying controller reachability, run dump vpn summary all and identify the VepID of each VPN path that remains down. For each affected VPN, run:

dump vpn status VpnID=<vpn-id>

Replace <vpn-id> with the VepID shown in dump vpn summary all. Review the local and remote public IP addresses, VPN status, active and usable states, and the reported down reason.

Step 8: If the VPN down reason is peer unreachable, verify that the remote peer’s public IP address is reachable through the selected WAN circuit. Check the northbound firewall, ISP modem, router, NAT configuration, and ISP network at both sites. Ensure that IPsec traffic is permitted, including IKE on UDP port 500, NAT Traversal on UDP port 4500, and ESP IP protocol 50 when ESP is not encapsulated by NAT Traversal.

Step 9: After resolving the WAN, controller-connectivity, or VPN issue, confirm that the ION reconnects to the controller, at least one VPN path becomes active and usable, and the incident clears.

Step 10: If the site remains offline, collect a support bundle through the SSH or console session and open a Palo Alto Networks Support case. Run dump-support all file=<descriptive_filename> for Release 6.4.1 and later or dump-support outputs file=<descriptive_filename> for earlier releases. Use a filename such as site_connectivity_down_ion-name_YYYYMMDDTHHMM.