ROSA HCP cluster upgrade fail because of OpenID authentication IDP error
Environment
- Red Hat OpenShift Service on AWS (ROSA) HCP
- 4
Issue
- ROSA HCP cluster upgrade fails because of OpenID authentication IDP error, from service log , it can be detect that the Control Plane upgrade maintenance beginning for long time but not progressed.
Control Plane upgrade maintenance beginning Cluster Updates Info service-account-ocm-cs-production-2 xx xx 2024, xx:xx UTC
Cluster's control plane is currently being upgraded to version '4.xx.xx'
Resolution
-
Check the CA setting on openid config CA section https://access.redhat.com/solutions/4608081
-
Towards OpenID authentication IDP error for ROSA HCP, there are multiple internal bugs have been reported, Based on the newest one OCPBUGS-43092 , the issue fix has been verified on 2024-10-14, which should be for example fixed on 4.16.18 or later version.
Root Cause
- The issue has been investigated on below BUG
This content is not included.OCPBUGS-32166
This content is not included.RFE-5638
This content is not included.OCPBUGS-38132
This content is not included.OCPBUGS-41372
This content is not included.OCPBUGS-43092
Diagnostic Steps
- Check managed cluster side logs found below error
{"level":"error","ts":"2024-xx-xxTxx:xx:xxZ","msg":"Reconciler error","controller":"hostedcontrolplane","controllerGroup":"hypershift.openshift.io","controllerKind":"HostedControlPlane","HostedControlPlane":{"name":"xxxxxxx","namespace":"ocm-production-xxxxxxx-xxxxxxx"},"namespace":"ocm-production-xxxxxxx-xxxxxxx","name":"xxxxxxx","reconcileID":"xxxxxxxx","error":"failed to update control plane: failed to reconcile openshift oauth apiserver: failed to reconcile oauth server config: failed to generate oauth config: failed to apply IDP azuread config: tls: failed to verify certificate: x509: certificate signed by unknown authority","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.
This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.