Known issues that might block the Framework for Upgrades 16.2 to 17.1
Before you perform an upgrade from Red Hat OpenStack Platform (RHOSP) 16.2 to 17.1, review the following known issues to prevent failures during the upgrade.
Leapp is stuck in Interim System when - -debug is specified
If you upgrade your operating system from RHEL 7.x to RHEL 8.x, or from RHEL 8.x to RHEL 9.x, do not run a Leapp upgrade with the --debug option. The system remains in the early console in setup code state and does not reboot automatically. To avoid this issue, the UpgradeLeappDebug parameter is set to false by default. Do not change this value in your templates.
Controller-0 does not perform Leapp upgrade due to packages missing ovn2.15 openvswitch2.15
If you upgrade from Red Hat OpenStack Platform (RHOSP) 13 to 16.1 or 16.2, or from RHOSP 16.2 to 17.1, do not include the system_upgrade.yaml file in the --answers-file answer-upgrade.yaml file. If the system_upgrade.yaml file is included in that file, the environments/lifecycle/upgrade-prepare.yaml file overwrites the parameters in the system_upgrade.yaml file. To avoid this issue, append the system_upgrade.yaml file to the openstack overcloud upgrade prepare command. For example:
$ openstack overcloud upgrade prepare --answers-file answer-upgrade.yaml /
-r roles-data.yaml /
-n networking-data.yaml /
-e system_upgrade.yaml /
-e upgrade_environment.yaml /
With this workaround, the parameters that are configured in the system_upgrade.yaml file overwrite the default parameters in the environments/lifecycle/upgrade-prepare.yaml file.
Cinder volume NFS mounts on Compute nodes are preventing Leapp upgrade
During an upgrade from RHOSP 16.2 to 17.1, the operating system upgrade from RHEL 8.4 to RHEL 9.2 fails if Cinder volume NFS mounts are present on Compute nodes. Contact your Red Hat support representative for a workaround.
overcloud_upgrade_prepare.sh fails pulling registry.access.redhat.com/ubi8/pause during upgrade from RHOSP 16.2 to 17.1 with no internet on undercloud
To avoid this issue, manually add a pause container to your Satellite server:
-
Import a pause container to your Satellite server, for example,
k8s.gcr.io/pause:3.5orregistry.access.redhat.com/ubi8/pause. -
In the
/usr/share/containers/containers.conffile, specify the pause container in your local Satellite URL. For example:infra_image="<LOCAL_SATELLITE_URL/pause:3.5>"- Replace
<LOCAL_SATELLITE_URL/pause:3.5>with your local Satellite URL and the pause container that you imported.
- Replace
-
Confirm that you can start a pod:
$ podman pod create
Leapp upgrade of Red Hat Ceph Storage nodes is failing encrypted partition detected
When you upgrade from Red Hat OpenStack Platform (RHOSP) 16.2 to RHOSP 17.1, the Leapp upgrade of the Red Hat Ceph Storage nodes fails because of an encrypted ceph-osd. Before you run the Leapp upgrade on your Red Hat Ceph Storage nodes, apply the workaround in the Red Hat Knowledgebase solution (FFU 16.2->17) leapp upgrade of ceph nodes is failing encrypted partition detected.
bridge_name is not translated to br-ex in RHOSP 17.1
The bridge_name variable is no longer valid for nic-config templates in RHOSP 17.1. After an upgrade from RHOSP 16.2 to 17.1, if you run a stack update and the nic-config templates still include the bridge_name variable, an outage occurs. Before you upgrade to RHOSP 17.1, you need to rename the bridge_name variable.
For more information, see the Red Hat Knowledgebase solution bridge_name is still present in templates during and post FFU causing further updates failure.
After cephadm adoption, haproxy fails to start when alertmanager is deployed
If you deployed Alertmanager in a director-deployed Red Hat Ceph Storage environment, the upgrade from Red Hat Ceph Storage version 4 to version 5 fails. The failure occurs because HAProxy does not restart after you run the following command to configure cephadm on the Red Hat Ceph Storage nodes:
$ openstack overcloud external-upgrade run \
--skip-tags ceph_ansible_remote_tmp \
--stack <stack> \
--tags cephadm_adopt 2>&1
After you run the command, the Red Hat Ceph Storage cluster status is HEALTH_WARN.
For a workaround for this issue, see the Red Hat Knowledgebase solution HAProxy does not restart during RHOSP upgrade when RHCS is director-deployed and Alertmanager is enabled.
BLUESTORE_NO_PER_POOL_OMAP OSD reporting legacy BlueStore omap usage stats
You might see a health warning message similar to the following after upgrading from Red Hat Ceph Storage 5 to 6:
[WRN] BLUESTORE_NO_PER_POOL_OMAP
You can clear this health warning message by following the instructions in the Red Hat Knowledgebase solution RHCS 6 - BLUESTORE_NO_PER_POOL_OMAP OSD(s) reporting legacy (not per-pool) BlueStore omap usage stats.
Undercloud upgrade fails on "migrate existing introspection data" with lost connection to mysql
If the undercloud upgrade fails, you must restart the mySQL service before you run the undercloud upgrade again. For more information about restarting the mySQL service, see the Red Hat Knowledgebase solution Update from 16.2 to 17.1 failed on migrate existing introspection data in the undercloud.
Upgrades takes longer time when cloud consist of 350+ nodes
The time you will need to upgrade from Red Hat OpenStack Platform (RHOSP) 16.2 to RHOSP 17.1 increases with the number of nodes in a single role. To reduce the amount of time it takes to complete the upgrade, you can split your nodes into multiple roles. For more information, see the Red Hat Knowledgebase article How to split roles during upgrade from RHOSP 16.2 to RHOSP 17.1.
RHOSP 16.2 to RHOSP 17.1 upgrade breaks GRUB and makes it try to boot RHEL 7
When you upgrade from Red Hat OpenStack Platform (RHOSP) 16.2 to 17.1, during the system upgrade, a known issue causes GRUB to contain Red Hat Enterprise Linux (RHEL) 7 entries instead of RHEL 8 entries. As a result, the hosts cannot reboot. This issue affects environments that previously ran RHOSP 13.0 or earlier.
Workaround: See the Red Hat Knowledgebase solution Openstack 16 to 17 FFU - During LEAPP upgrade UEFI systems do not boot due to invalid /boot/grub2/grub.cfg.
No disk space check causing unbootable node during Leapp upgrade
The Leapp version that upgrades Red Hat Enterprise Linux 8.4 to 9.2 does not verify whether all partitions have enough disk space. Before you perform the Red Hat OpenStack Platform system upgrade, you must manually check that all partitions have at least 3 GB of disk space. Failure to do so can cause the node to reboot and enter into an emergency shell.
Incorrect validation of Podman version
If you perform an upgrade of your RHOSP environment to 17.1.x, the pre-upgrade package_version validation fails because the validation cannot find a matching podman version.
Workaround: To skip the package_version validation when you run the pre-upgrade validation, use the --skiplist option and specify your skiplist file in either .json or .yaml format. For example:
-
JSON:
$ validation run -i inventory.yaml --group pre-upgrade --skiplist skiplist.json -
YAML:
$ validation run -i inventory.yaml --group pre-upgrade --skiplist skiplist.yaml
Horizon dashboard internal server error
Static file compression does not run automatically after an upgrade from Red Hat OpenStack Platform (RHOSP) 16.2 to 17.1. As a result, the missing static files cause the RHOSP dashboard (horizon) to fail. To run the compression manually after the upgrade, see Compressing Red Hat OpenStack Platform dashboard files.
EUS repositories override repositories in Red Hat Ceph Storage role
If you are using director-deployed Red Hat Ceph Storage 5 nodes, during an upgrade from Red Hat OpenStack Platform 16.2 to 17.1, the Extended Update Support (EUS) repositories that are specified in the UpgradeInitCommand parameter override the repositories in the Red Hat Ceph Storage role.
Workaround: To use the repositories listed in your Red Hat Ceph Storage nodes, add the following parameters:
-
In the
upgrades-environment.yamlfile, add theCephStorageUpgradeInitCommand:parameter_defaults: ... UpgradeInitCommand: | sudo subscription-manager repos --disable=* ... CephStorageUpgradeInitCommand: | sudo subscription-manager repos --disable=* if $( grep -q 9.2 /etc/os-release ) then sudo subscription-manager repos --enable=rhel-9-for-x86_64-baseos-rpms --enable=rhel-9-for-x86_64-appstream-rpms --enable=openstack-17.1-deployment-tools-for-rhel-9-x86_64-rpms --enable=openstack-17.1-for-rhel-9-x86_64-rpms --enable=fast-datapath-for-rhel-9-x86_64-rpms sudo podman ps | grep -q ceph && subscription-manager repos --enable=rhceph-5-tools-for-rhel-9-x86_64-rpms sudo subscription-manager release --set=9.2 else sudo subscription-manager repos --enable=rhel-8-for-x86_64-baseos-aus-rpms --enable=rhel-8-for-x86_64-appstream-aus-rpms --enable=rhel-8-for-x86_64-highavailability-aus-rpms --enable=openstack-17.1-for-rhel-8-x86_64-rpms --enable=fast-datapath-for-rhel-8-x86_64-rpms sudo podman ps | grep -q ceph && subscription-manager repos --enable=rhceph-5-tools-for-rhel-8-x86_64-rpms sudo subscription-manager release --set=8.4 fi if $(sudo podman ps | grep -q ceph ) then sudo dnf -y install cephadm fi -
In the
system_upgrade.yamlfile, add theCephStorageUpgradeLeappCommandOptionsandCephStorageLeappInitCommandparameters:LeappRepoInitCommand: | subscription-manager repos --disable=* ... CephStorageUpgradeLeappCommandOptions: "--enablerepo=rhel-9-for-x86_64-baseos-rpms --enablerepo=rhel-9-for-x86_64-appstream-rpms --enablerepo=openstack-17.1-for-rhel-9-x86_64-rpms --enablerepo=fast-datapath-for-rhel-9-x86_64-rpms CephStorageLeappInitCommand: | subscription-manager repos --disable=* subscription-manager release --unset subscription-manager repos --enable=rhel-9-for-x86_64-baseos-rpms --enable=rhel-9-for-x86_64-appstream-rpms --enable=openstack-17.1-for-rhel-9-x86_64-rpms --enable=fast-datapath-for-rhel-9-x86_64-rpms leapp answer --add --section check_vdo.confirm=True leapp answer --add --section check_vdo.no_vdo_devices=True
Upgrade from RHOSP 16.2 to 17.1 fails if extra openvswitch packages are installed
An upgrade from Red Hat OpenStack Platform (RHOSP) 16.2 to 17.1 fails if additional, non-core openvswitch packages are installed.
Workaround: See the Red Hat Knowledgebase solution FFU is failing on Ansible task special treatment for OpenvSwitch.
grafana.update.checker trying to reach Internet in a disconnected environment
After an upgrade from Red Hat Ceph Storage 6 to 7, if you have a disconnected Red Hat OpenStack environment, Grafana attempts to access the internet to download updates. As a result, Grafana times out.
Workaround: See This content is not included.BZ#2346107.
Upgrade from RHOSP 16.2.6 to 17.1.4 - openstack leapp upgrade of compute with NVIDIA GPU card failed
If you attempt to perform a Leapp OS upgrade with NVIDIA drivers, the system upgrade fails with the following error in /var/log/leapp/leapp-report.txt:
Summary: Leapp has detected that the NVIDIA proprietary driver has been loaded, which also means the nouveau driver is blacklisted. If you upgrade now, you will end up without a graphical session, as the newer kernel won't be able to load the NVIDIA driver module and nouveau will still be blacklisted.
Please uninstall the NVIDIA graphics driver before upgrading to make sure you have a graphical session after upgrading.
Workaround:
-
Remove the NVIDIA driver. For example:
$ sudo dnf remove -y NVIDIA-vGPU-rhel-8.4-525.105.14.x86_64 -
Upgrade the Compute node:
$ openstack overcloud upgrade run --tag system_upgrade --limit <compute-0> -
After the server reboot, re-install the NVIDIA drivers for the appropriate operating system (RHEL 9.2).
-
If necessary, re-create the
mdevdevices.
Validation fails because inventory file contains wrong parameter
If you use an inventory file that is located in a RHOSP 16.2 /var/lib/mistral directory, the undercloud-service-status validation fails when you run validation run -i inventory.yaml --group post-upgrade. The failure occurs because the undercloud_service_list parameter either exists with the wrong services, or it does not exist and the wrong default value is used in its place.
Workaround:
-
Use one of the following options:
- Use the inventory file that is created on
/home/stack/overcloud-deploy/${stack}/ - Add the following configuration to the inventory file:
Undercloud: hosts: undercloud: {} vars: ... undercloud_service_list: - tripleo_ironic_conductor ... - Use the inventory file that is created on
-
Run the validation again and confirm that the status is
PASSED:$ validation run -i inventory.yaml --group post-upgrade