How to restore Red Hat Satellite using file system-level backup

Updated

1. Summary

This article provides the necessary steps to recover a Red Hat Satellite service using a file system-level backup.

Critical Support Note

  • Before proceeding, please note that the only officially supported method for backing up and restoring Red Hat Satellite is via the native utility:

    satellite-maintenance backup

  • Using this command ensures data consistency across the PostgreSQL database, Pulp records, and Candlepin archives. While file system backups (such as rsync or third-party backup agents) can be used as a recovery vector, they must be executed while the Satellite services are completely stopped to prevent data corruption.

  • This guide focuses specifically on the restoration of core Satellite services (the database, repository metadata, and fundamental API functionality). Depending on the complexity of your environment and the specific failure scenario, additional manual intervention may be required to fully recover secondary components.

2. Requirements

  • There are available backups for the next File systems:

    • /var/lib/pulp
    • /var/lib/pgsql
    • /etc/

  • Confirm that the /var/lib/pgsql directory is consistent. This state is achieved by either:

    • Performing a cold backup (shutting down the PostgreSQL service before taking the backup).
    • Using a hot-backup tool to manage files while the database is online.

  • Satellite certificates backup is also needed. (Usually /root/ssl-build folder for default certificates)

    cat /path_to_backup/etc/foreman-installer/scenarios.d/satellite-answers.yaml | grep -E "server_cert:|server_key:|server_ca_cert:"

  • The used Restoration tool restores properly the files permissions and owners.

3. Recommendations

  • Do NOT install any additional software like fapolicyd or security tools. The goal is to restore the Satellite software, they should be installed later.

4. Preparing the new Satellite host

  1. Install a fresh RHEL host using the same minor version as the old server, do not update the Operating system. E.g: RHEL8.4

  2. Rename the host to match the old-satellite hostname.

  3. Configure the same network interfaces and IPs.

  4. If not yet enrolled and IdM, FreeIPA or Active Directory is used, enroll the host using the proper procedure.

  5. Get User groups from the old Satellite /etc backup

    cat /path_to_backup/etc/group | grep -E "^postgres|^foreman|^pulp|^puppet|^redis|^tomcat"

  6. On the new Satellite server add the group lines to /etc/group

  7. Get Users from the old Satellite /etc backup

    cat /path_to_bakup/etc/passwd | grep -E "^postgres|^foreman|^pulp|^puppet|^redis|^tomcat"

  8. On the new Satellite server add the user lines to /etc/passwd

  9. Get Users password lines from the old Satellite /etc backup

    cat /path_to_bakup/etc/shadow | grep -E "^postgres|^foreman|^pulp|^puppet|^redis|^tomcat"

  10. On the new Satellite server add the shadow lines to /etc/shadow. This file is Read Only, to save it using vi use :w!.

  11. Take a Satellite snapshot at this point. It could be useful.

5. Restoring the Satellite software

  1. Enable Satellite repositories and satellite module:

    subscription-manager repos --disable "*"
subscription-manager repos \
--enable=rhel-8-for-x86_64-baseos-rpms \
--enable=rhel-8-for-x86_64-appstream-rpms \
--enable=satellite-6.16-for-rhel-8-x86_64-rpms \
--enable=satellite-maintenance-6.16-for-rhel-8-x86_64-rpms
dnf module enable satellite:el8

  2. Get the exact version that was installed the old Satellite /etc backup. We'll need this <OLD_VERSION> later.

    cat /path_to_backup/etc/rhsm/facts/satellite.facts | grep version | cut -d '"' -f 4

  3. Get the previously installed package version. (Replace <OLD_VERSION> placeholder)

    dnf search satellite --showduplicates --disableplugin foreman-protector | grep satellite-<OLD_VERSION>

  4. Install the same version on the new Satellite using the <PKG_NAME> from he previous command

    dnf install <PKG_NAME> -y

  5. Get the initial Satellite settings. We'll need this values in next step.

    cat /etc/foreman-installer/scenarios.d/satellite-answers.yaml | grep -E "initial_admin_username|initial_admin_password|initial_organization|initial_location"

  6. Configure Satellite using <OLD_ORGANIZATION>, <OLD_LOCATION>, <OLD_ADMIN> and <OLD_PASS> from previous command.

    satellite-installer --foreman-initial-organization "<OLD_ORGANIZATION>" \
                    --foreman-initial-location "<OLD_LOCATION>" \
                    --foreman-initial-admin-username "<OLD_ADMIN>" \
                    --foreman-initial-admin-password "<OLD_PASS>"

  7. Take another Satellite snapshot. It could be useful.

  8. Restore next files: (Note the * on paths)

    /var/lib/pgsql/
/var/lib/pulp/
/etc/candlepin/
/etc/firewalld/
/etc/foreman*
/etc/hammer/
/etc/pulp/
/etc/redis*

  9. Restore the Certificates on the same paths shown by next command:

    cat /etc/foreman-installer/scenarios.d/satellite-answers.yaml | grep -E "server_cert:|server_key:|server_ca_cert:"

  10. Restart firewalld

systemctl restart firewalld
  1. Confirm SELinux labels are right.
restorecon -r /var/lib/pulp \
              /var/lib/pgsql \
              /etc/
  1. Execute Satellite installer again to restore old Satellite configuration. It will read all the configurations from /etc/ restored files.
satellite-installer

6. Optional steps

  • Depending on the scenario some additional steps may be needed:
    • Restore Puppet files.
    • Restore Ansible roles.
    • Any other customization.
    • Upload a new Manifest.
    • Some repositories may need to be re-synchronized.
    • Some Content View may need to Publish and Promote a new version.

For more KB articles/solutions related to Red Hat Satellite 6.x backup issues, please refer to the Consolidated Troubleshooting Article for Red Hat Satellite 6.x backup-related Issues

SBR
Product(s)
Tags
Article Type