Migrate Logs to a New M-Series Appliance Model in Panorama Mode in High Availability

If you need to replace an M-700, M-600, M-500, M-300, M-200 or M-100 appliance in Panorama mode (Panorama management server) with a different M-Series appliance than the M-Series appliance being replaced, you can migrate the logs it collected from firewalls by moving its RAID disks to the new M-Series appliance. Moving the disks enables you to migrate logs as part of a hardware upgrade (from an M-100 appliance to an M-500 appliance). You can migrate an M-100 appliance to and from an M-500 appliance. M-100 and M-500 appliances cannot be migrated to or from M-200 or M-600 appliances.

Migrating logs by removing the logging disks from any M-Series appliance and loading them into an M-600 Panorama management server is not supported. To migrate to an M-600 appliance, set up the M-600 appliance, configure log forwarding to the new M-600 appliance and configure the M-Series appliance as a managed Log Collector until you no longer needed access to the logs stored on the M-Series appliance.

This migration procedure covers the following scenarios:

• One Panorama HA peer has a managed collector (Log Collector) in a Collector Group.

• Both Panorama HA peers have managed collectors that belong to a single Collector Group. For details, see Multiple Local Log Collectors Per Collector Group.
• Both Panorama HA peers have a managed collector and each is assigned to a separate Collector Group. For details, see Single Local Log Collector Per Collector Group.

1. Forward any logs on the SSD of the old M-Series appliance to an external destination if you want to preserve them.

   The SSD stores the System and Config logs that Panorama and Log Collectors generate. You cannot move the SSD between M-Series appliances.

   Configure Log Forwarding from Panorama to External Destinations.
2. Export the Panorama configuration from the Primary decommissioned M-Series appliance in Panorama mode.

   1. Log in to the Panorama web interface of the M-Series appliance you are replacing and select PanoramaSetupOperations.
   2. Click Save named Panorama configuration snapshot, enter a Name to identify the configuration, and click OK.
   3. Click Export named Panorama configuration snapshot, select the Name of the configuration you just saved, and click OK. Panorama exports the configuration to your client system as an XML file.
3. Remove the RAID disks from the old M-Series appliance.

   1. Power off the old M-Series appliance by pressing the Power button until the system shuts down.
   2. Remove the disk pairs. For details, refer to the disk replacement procedure in the M-Series Appliance Hardware Reference Guides.
4. Perform initial setup of the new M-Series appliance.

   Repeat this step for each of the new M-Series appliances in the HA configuration.

   1. Rack mount the M-Series appliance. Refer to the M-Series Appliance Hardware Reference Guides for instructions.
   2. Perform Initial Configuration of the M-Series Appliance.
   3. Register Panorama.
   4. Purchase and activate a Panorama support license or transfer licenses as follows only if the new M-Series appliance is the same hardware model as the old M-Series appliance. If the new M-Series appliance is a different model than the old M-Series appliance, you must purchase new licenses.

      1. Log in to the Palo Alto Networks Customer Support web site.
      2. Select the Assets tab and click the Spares link.
      3. Click the Serial Number of the new M-Series appliance.
      4. Click Transfer Licenses.
      5. Select the old M-Series appliance and click Submit.
   5. Activate a firewall management license. If you are migrating from an M-100 appliance to an M-500 appliance, enter the auth-code associated with the migration license.
   6. Install Content and Software Updates for Panorama. For important details about software versions, see Panorama, Log Collector, Firewall, and WildFire Version Compatibility.
   7. Set Up HA on Panorama. The new M-Series appliance must have the same priority as the HA peer you are replacing.
5. Load the Panorama configuration snapshot that you exported from the Primary decommissioned M-Series appliance into the new Primary M-Series appliance in Panorama mode.

   1. Log in to the Panorama web interface of the new M-Series appliance and select PanoramaSetupOperations.
   2. Click Import named Panorama configuration snapshot, Browse to the configuration file you exported from the decommissioned M-Series appliance, and click OK.
   3. Click Load named Panorama configuration snapshot, select the Name of the configuration you just imported, select a Decryption Key (the master key for Panorama), and click OK. Panorama overwrites its current candidate configuration with the loaded configuration. Panorama displays any errors that occur when loading the configuration file. If errors occurred, save them to a local file. Resolve each error to ensure the migrated configuration is valid.

      To replace an RMA Panorama, make sure you Retain Rule UUIDs when you load the named Panorama configuration snapshot. If you do not select this option, Panorama removes all previous rule UUIDs from the configuration snapshot and assigns new UUIDs to the rules on Panorama, which means it does not retain information associated with the previous UUIDs, such as the policy rule hit count.
   4. Perform any additional configuration changes as needed.

      If the old M-Series appliance used interfaces other than the MGT interface for Panorama services (such as log collection), you must define those interfaces on the new M-Series appliance (PanoramaSetupInterfaces).
   5. Select CommitCommit to Panorama and Validate Commit. Resolve any errors before proceeding.
   6. Commit your changes to the Panorama configuration. Once committed, the Panorama configuration is synced across the HA peers.
6. Insert the disks into the new M-Series appliance. For details, refer to the disk replacement procedure in the M-Series Appliance Hardware Reference Guides.

   Repeat this step for each of the new M-Series appliances in the HA configuration.

   The disk carriers of the M-100 appliance are incompatible with those of the M-500 appliance. Therefore, when migrating between these hardware models, you must unscrew each disk from its old carrier and insert the disk in the new carrier before inserting the disk in the new appliance.

   You must maintain the disk pair association. Although you can place a disk pair from slot A1/A2 on the old appliance into slot B1/B2 on the new appliance, you must keep the disks together in the same slot; otherwise, Panorama might not restore the data successfully.
7. Contact Palo Alto Networks Customer Support to copy log collector group metadata from the decommissioned M-Series appliance to the new M-Series appliance and restart the mgmtsrvr process.

   Refer to the Palo Alto Networks Knowledge Base when working with your Palo Alto Networks TAC engineer.
8. If the M-Series appliance was part of a Collector Group, verify that the decommissioned M-Series appliance serial number is still part of the correct Collector Group:

   debug log-collector-group show name <Log CollectorGroup name>

   If the decommissioned M-Series appliance serial number is no longer a part of the correct Collector Group, then the Tech Support folders were incorrectly copied in the previous step. Contact Palo Alto Networks Customer Support again to copy the Tech Support folders to the correct location.
9. Prepare the disks for migration.

   Generating the metadata for each disk pair rebuilds the indexes. Therefore, depending on the data size, this process can take a long time to complete. To expedite the process, you can launch multiple CLI sessions and run the metadata regeneration command in each session to complete the process simultaneously for every pair. For details, see Regenerate Metadata for M-Series Appliance RAID Pairs.

   1. Enable the disk pairs by running the following CLI command for each pair:

      admin> request system raid add <slot> force no-format

      For example:

      admin> request system raid add A1 force no-format 
      admin> request system raid add A2 force no-format 

      The force and no-format arguments are required. The force argument associates the disk pair with the new appliance. The no-format argument prevents reformatting of the drives and retains the logs stored on the disks.
   2. Generate the metadata for each disk pair.

      This step may take up to 60 hours to complete depending on the volume of log data on the disks. The new M-Series appliance must be in an active state. Panorama may experience issues getting the assigned Log Collector ID if Panorama is in a suspended state.

      If the new M-Series appliance is the passive HA peer, log in to the Panorama web interface of the currently active HA peer and select Panorama > High Availability and Suspend local Panorama for high availability.

      After you successfully migrate the logs, Make local Panorama functional for high availability.

      admin> request metadata-regenerate slot <slot_number>

      For example:

      admin> request metadata-regenerate slot 1

10. Configure the local Log Collector on the new M-Series appliance.

    For all steps with commands that require a serial number, you must type the entire serial number; pressing the Tab key won’t complete a partial serial number.

    Don’t enable the disks on the new M-Series appliance at this point. When you successfully migrate the logs, Panorama automatically enables the disks.

    1. Configure the local Log Collector as a managed collector using the Panorama web interface or using the following CLI commands:

       admin> configure 
       admin# set log-collector <log-collector_SN> deviceconfig system hostname <log-collector-hostname>  
       admin# exit 

    2. Commit your changes to Panorama. Don’t commit the changes to the Collector Group just yet.

       admin> configure 
       admin# commit 

    3. Verify that the local Log Collector is connected to Panorama and that the status of its disk pairs is present/available.

       admin> show log-collector serial-number <log-collector_SN>

       The disk pairs will display as disabled at this stage of the restoration process.
11. Add a Log Collector with no disks to a Collector Group.

    From this point, only commits that are required to complete the migration process on Panorama and the Log Collectors. Hold off making any other changes.

    1. Access the Panorama CLI of the new M-Series appliance.
    2. Overwrite Panorama restriction to allow Log Collector with no disk to be added to a Collector Group: request log-migration-set-start
    3. Commit the changes to Panorama.

       admin> configure 
       admin# commit force 

12. Migrate the logs.

    The new M-Series appliance must be the active HA peer before you can begin migrating logs. If the new M-Series appliance, log in to the Panorama web interface of the active HA peer and select Panorama > High Availability and Suspend local Panorama for high availability.

    After you successfully migrate the logs, Make local Panorama functional for high availability.

    1. Access the Panorama CLI of the new M-Series appliance.
    2. Add the new local Log Collector as a member of the Collector Group and commit your changes to Panorama.

       admin# set log-collector-group <collector_group_name> logfwd-setting collectors <SN_managed_collector> 
       admin# commit 
       admin# exit 

       The old local Log Collector still appears in the list of members, because you haven’t deleted it from the configuration.
    3. For each disk pair, migrate the logs to the new appliance.

       admin> request log-migration from <old_LC_serial_number> old-disk-pair <log_disk_pair> to <new_LC_serial_number> new-disk-pair <log_disk_pair> 

       For example:

       admin> request log-migration from 003001000010 old-disk-pair A to 00300100038 new-disk-pair A 

    4. Commit the changes to Panorama.

       admin> configure 
       admin# commit 

13. Reconfigure the Collector Group.

    1. Log in to the Panorama web interface of the new M-Series appliance to assign the new Log Collector to the firewalls that forward logs (PanoramaCollector GroupsDevice Log Forwarding). Give the new Log Collector the same priority in the firewall preference lists as the old Log Collector.

       You cannot use the CLI change the priority assignments of firewall preference lists.
    2. Access the Panorama CLI of the new M-Series appliance.
    3. Delete the old Log Collector from the Collector Group.

       admin# delete log-collector-group <group_name> logfwd-setting collectors <old_LC_serial_number>

       For example:

       admin# delete log-collector-group DC-Collector-Group logfwd-setting collectors 003001000010

    4. Delete the old Log Collector from the Panorama configuration and commit your changes to Panorama.

       admin# delete log-collector <old_LC_serial_number> 
       admin# commit 
       admin# exit 

    5. Commit the Collector Group changes so that the managed firewalls can send logs to the new Log Collector.

       admin> commit-all log-collector-config log-collector-group <collector_group_name>

       For example:

       admin> commit-all log-collector-config log-collector-group DC-Collector-Group

14. Generate new keys on the new Log Collector.

    This command is required in order to add the new Log Collector to the Collector Group and should only be run for the Collector Group of the Log Collector being replaced. This step deletes the existing RSA keys and allows Panorama to create new RSA keys.

    1. Access the Panorama CLI of the new M-Series appliance.
    2. Delete all RSA keys on the new Log Collector:

       request logdb update-collector-group-after-replace collector-group <collector-group-name>

       The process can take up to 10 minutes to completed.
15. Confirm that SearchEngine Status is Active for all Log Collectors in the Collector Group.

    Do not continue until SearchEngine Status is Active for all Log Collectors in the Collector Group. This will result in purging of logs from the Log Collector being replaced.

    1. Access the Panorama CLI of the new M-Series appliance.
    2. Show the Log Collector details by running the following commands either:

       • On Panorama for all Log Collectors:
         show log-collector all

       Alternatively, you can run the following command on each Dedicated Log Collector:

       show log-collector detail

    3. Confirm that SearchEngine Status is Active.

       Redistribution status:       none

       Last commit-all: commit succeeded, current ring version 1

       SearchEngine status:     Active

       md5sum 4e5055a359f7662fab8f8c4f57e24525 updated at 2017/06/14 09:58:19

16. On the new Log Collector, replace previous Log Collector serial number with the new Log Collector serial number.

    You must replace the old Log Collector serial number with the new Log Collector serial number so that the new Log Collector will not run in to purging issues, resulting in the Log Collector being unable to purge old data from the migrated logs when necessary.

    (PAN-OS 11.1 and later releases) When the Log Collector serial number changes, you must erase all Elasticsearch data before executing the logger update command. If you skip this step, Elasticsearch will not start correctly after the migration.

    1. Access the Log Collector CLI.
    2. Erase all existing Elasticsearch data on the new Log Collector:

       debug elasticsearch erase data
    3. Replace old Log Collector serial number with new Log Collector serial number:

       request log-migration-update-logger from <old-log-collector-serial-number> to <new-log-collector-serial-number>
    4. Reingest raw logs for each required log type and time window. Repeat the command for each log type and monitor progress before initiating the next. debug logdb migrate-lc start log-type <log-type> time period start-date <YYYY/MM/DD> end-date <YYYY/MM/DD>
17. Set up the new secondary Panorama high availability peer.

    1. Forward any logs on the SSD of the old M-Series appliance to an external destination if you want to preserve them.
    2. Remove the RAID disks from the old M-Series appliance.
    3. Perform initial setup of the new M-Series appliance.
    4. Insert the disks into the new M-Series appliance.
    5. Repeat Steps 7 through 16 to migrate the logs from the old M-Series appliance to the new M-Series appliance.
    6. Set Up HA on Panorama. The new M-Series appliance must have the same priority as the HA peer you are replacing.
    7. Log in to the Panorama web interface of the primary HA peer and click DashboardHigh AvailabilitySync to peer to synchronize the configuration of the M-Series appliance HA peers.