How do I migrate AAP 2.6 from an RPM to Containerized installation?

Solution Verified - Updated 6 Jul 2026

Environment

Ansible Automation Platform 2.6 RPM
Ansible Automation Platform 2.6 Containerized

Issue

Ansible Automation Platform RPM installer will be deprecated with AAP 2.7
I need to keep my environment in a supported state

Resolution

1. Prerequisites and Requirements

These instructions are specifically for migrating from AAP 2.6 RPM to AAP 2.6 Containerized.
These instructions have not been tested on any other version of AAP.
These instructions assume the AAP 2.6 RPM installer is located on one of the servers in the environment. If you install AAP 2.6 from a Bastion host, you may need to adjust these instructions.
These instructions assume the AAP 2.6 Containerized installer is located on the Containerized Gateway node. If you install AAP 2.6 from a Bastion host, you may need to adjust these instructions.
These instructions assume you have the following AAP components in your AAP 2.6 RPM environment: Gateway, Controller, Hub, EDA. If you do not have one or more of these components, you may need to skip some steps.
Your RPM environment MUST be on the latest version of AAP 2.6.
The Containerized environment should be all newly created VMs. You cannot reuse the servers from the RPM environment.
Ensure your Containerized environment meets the System requirements.
The Containerized environment should NOT have AAP installed before proceeding with these instructions.
- You must uninstall your AAP 2.6 Containerized environment before proceeding.

2. Prepare and assess the RPM environment

The migration from the RPM environment to the Containerized environment may require additional storage to store the artifact.tar file which will be created in section 3.
The following instructions will help determine how much storage will be required for the migration.

IMPORTANT: If using multiple database servers in the RPM environment, you must perform this step on each database server.

2.1 Connect to the PostgreSQL database and switch to the superuser

Run the following commands on the RPM database as the root user:

su - postgres
psql

NOTE: If you do not have command line access to the database server (web-service database), connect to the database as a superuser.

2.2 Verify PostgreSQL version

While connected to the postgres database, run the following command:

SELECT version();

Example:

# su - postgres
$ psql
postgres=# SELECT version();
                                                    version                                                     
----------------------------------------------------------------------------------------------------------------
 PostgreSQL 15.15 on x86_64-redhat-linux-gnu, compiled by gcc (GCC) 11.5.0 20240719 (Red Hat 11.5.0-11), 64-bit
(1 row)

IMPORTANT: If you are not on version 15 or higher, you must upgrade to the minimum required version of PostgreSQL 15.

2.3 Verify PostgreSQL size

While connected to the postgres database, run the following command:

SELECT datname as database_name, pg_size_pretty(pg_database_size(datname)) size FROM pg_database WHERE datistemplate <> 'true' AND datname <> 'postgres';

Example:

postgres=# SELECT datname as database_name, pg_size_pretty(pg_database_size(datname)) size FROM pg_database WHERE datistemplate <> 'true' AND datname <> 'postgres';
      database_name      |  size   
-------------------------+---------
 awx                     | 4447 MB
 automationhub           | 91 MB
 automationedacontroller | 14 MB
 automationgateway       | 14 MB
(4 rows)

2.4 Adjust the available storage accordingly

In a future step, you will be performing a manual backup of the databases for the Controller, Gateway, EDA, and Hub.
The backups will be stored in the /tmp directory on the RPM Gateway node, and later on the Containerized Gateway node.
It is difficult to predetermine the required storage for the backups, but this will give an approximation.
Add up the sizes for the databases and ensure you have at least that amount available on the RPM Gateway node and Containerized Gateway node.
Again, it is difficult to know how much additional storage will be required, so adding more storage is better.

3. Create and verify the migration artifact

The migration artifact is a tarball file that will contain the data from your RPM environment, which includes:

secrets.yml: information about the databases
sha256sum.txt: used to validate the backup contents
gateway/gateway.pgc: Gateway database backup
controller/controller.pgc: Controller database backup
hub/hub.pgc: Hub database backup [optional]
eda/eda.pgc: EDA database backup [optional]
controller/custom_configs/: Controller custom configurations [optional]

3.1 Create the artifact directory

Run the following commands on the RPM Gateway node as the root user:

mkdir -p /tmp/backups/artifact/{controller,gateway,hub,eda}
mkdir -p /tmp/backups/artifact/controller/custom_configs
touch /tmp/backups/artifact/secrets.yml

3.2 Populate the secrets.yml

In the following steps you will run commands on each component node in the RPM environment.
You will then copy the output and paste it into the secrets.yml artifact.

3.2.1 Get the Gateway database connection and SECRET_KEY values

a. Run the following commands on the RPM Gateway node as the root user:

line=$(aap-gateway-manage print_settings | grep '^DATABASES'); echo "$line" | sed -n "s/.*'HOST': *'\([^']*\)'.*/gateway_pg_host: \1/p"; echo "$line" | sed -n "s/.*'USER': *'\([^']*\)'.*/gateway_pg_username: \1/p"; echo "$line" | sed -n "s/.*'NAME': *'\([^']*\)'.*/gateway_pg_database: \1/p"; echo "$line" | sed -n "s/.*'PORT': *'\([^']*\)'.*/gateway_pg_port: \1/p"; echo "$line" | sed -n "s/.*'PASSWORD': *'\([^']*\)'.*/gateway_pg_password: \1/p";

awk '{print "gateway_secret_key: \047" $0 "\047"}' /etc/ansible-automation-platform/gateway/SECRET_KEY

b. Copy the outputs and paste them into the secrets.yml

3.2.2 Get the Controller database connection and SECRET_KEY values

a. Run the following commands on the RPM Controller node as the root user:

line=$(awx-manage print_settings | grep '^DATABASES'); echo "$line" | sed -n "s/.*'HOST': *'\([^']*\)'.*/controller_pg_host: \1/p"; echo "$line" | sed -n "s/.*'USER': *'\([^']*\)'.*/controller_pg_username: \1/p"; echo "$line" | sed -n "s/.*'NAME': *'\([^']*\)'.*/controller_pg_database: \1/p"; echo "$line" | sed -n "s/.*'PORT': *'\([^']*\)'.*/controller_pg_port: \1/p"; echo "$line" | sed -n "s/.*'PASSWORD': *'\([^']*\)'.*/controller_pg_password: \1/p";

awk '{print "controller_secret_key: \047" $0 "\047"}' /etc/tower/SECRET_KEY

b. Copy the outputs and paste them into the secrets.yml

3.2.3 Get the Hub database connection and SECRET_KEY values

a. Run the following commands on the RPM Hub node as the root user:

line=$(grep '^DATABASES' /etc/pulp/settings.py); echo "$line" | sed -n "s/.*'HOST': *'\([^']*\)'.*/hub_pg_host: \1/p"; echo "$line" | sed -n "s/.*'USER': *'\([^']*\)'.*/hub_pg_username: \1/p"; echo "$line" | sed -n "s/.*'NAME': *'\([^']*\)'.*/hub_pg_database: \1/p"; echo "$line" | sed -n "s/.*'PORT': *'\([^']*\)'.*/hub_pg_port: \1/p"; echo "$line" | sed -n "s/.*'PASSWORD': *'\([^']*\)'.*/hub_pg_password: \1/p";

grep '^SECRET_KEY' /etc/pulp/settings.py | awk -F'=' '{ print "hub_secret_key:" $2 }'

awk '{print "hub_db_fields_encryption_key: \047" $0 "\047"}' /etc/pulp/certs/database_fields.symmetric.key

b. Copy the outputs and paste them into the secrets.yml

3.2.4 Get the EDA database connection and SECRET_KEY values

a. Run the following commands on the RPM EDA node as the root user:

egrep ' HOST| NAME| USER| PASSWORD| PORT' /etc/ansible-automation-platform/eda/settings.yaml | sed -e 's/^.*HOST/eda_pg_host/g' -e 's/^.*NAME/eda_pg_database/g' -e 's/^.*USER/eda_pg_username/g' -e 's/^.*PASSWORD/eda_pg_password/g' -e 's/^.*PORT/eda_pg_port/g'

awk '{print "eda_secret_key: \047" $0 "\047"}' /etc/ansible-automation-platform/eda/SECRET_KEY

b. Copy the outputs and paste them into the secrets.yml

3.2.5 Verify the secrets.yml

Make sure the secrets.yml contains the following variables:

gateway_pg_host
gateway_pg_username
gateway_pg_database
gateway_pg_port
gateway_pg_password
gateway_secret_key
controller_pg_host
controller_pg_username
controller_pg_database
controller_pg_port
controller_pg_password
controller_secret_key
hub_pg_host
hub_pg_username
hub_pg_database
hub_pg_port
hub_pg_password
hub_secret_key
hub_db_fields_encryption_key
eda_pg_host
eda_pg_username
eda_pg_database
eda_pg_port
eda_pg_password
eda_secret_key

Example:

gateway_pg_host: my.rpm.database
gateway_pg_username: automationgateway
gateway_pg_database: automationgateway
gateway_pg_port: 5432
gateway_pg_password: password
gateway_secret_key: 'AI7gE6BPIaUKjwHuQ12tIONsBkZyAPbjoQ(.....)piByCtW3hfikIxFHMZbTFRlPRA85dwD'

controller_pg_host: my.rpm.database
controller_pg_username: awx
controller_pg_database: awx
controller_pg_port: 5432
controller_pg_password: password
controller_secret_key: 'ZhetN24ovutz6DowTGACMUOyZ04GjO5puenPynBkSwqf'

hub_pg_host: my.rpm.database
hub_pg_username: automationhub
hub_pg_database: automationhub
hub_pg_port: 5432
hub_pg_password: password
hub_secret_key: 'NhP0t1WEoUWX__OE1ZK-kG2yNjH4EnaaJO:1.e(.....))sLDJ:KTrsGu,zKtcbfOJwB6N6Mg01'
hub_db_fields_encryption_key: 'sWNmc6YgoXBtBsu3zyu1xg9Yu_s6p23AKl0X1vHwoXk='

eda_pg_host: my.rpm.database
eda_pg_database: automationedacontroller
eda_pg_password: password
eda_pg_port: '5432'
eda_pg_username: automationedacontroller
eda_secret_key: 'wYvyyvPi0GT8lcS8uxhIIuVqleBBUBTluWhF2h(.....)OdUjxqLkxuJuH86NkNDi0ioWKYzuD0'

3.3 Copy custom configs

If any custom settings exist in /etc/tower/conf.d, copy them to /tmp/backups/artifact/controller/custom_configs.

NOTE: The following configuration files on the automation controller are NOT considered custom:

/etc/tower/conf.d/postgres.py
/etc/tower/conf.d/channels.py
/etc/tower/conf.d/caching.py
/etc/tower/conf.d/cluster_host_id.py

3.4 Create the pgc files (perform the database backups)

IMPORTANT: These steps run full database dumps (pg_dump) and may increase load on the database servers. Consider running them during a maintenance window.

NOTES:

The following steps will create the database backups.
The commands may take a while to complete based on the size of the database, resources available, and if AAP is currently running.
All commands should be executed on the RPM Gateway node as the root user.

3.4.1 Back up the Gateway database

Run the following command on the RPM Gateway node as the root user:

ansible localhost -m shell -a "export PGPASSWORD={{ gateway_pg_password }}; pg_dump -h {{ gateway_pg_host }} -U {{ gateway_pg_username }} -d {{ gateway_pg_database }} -p {{ gateway_pg_port }} --clean --create -Fc -f /tmp/backups/artifact/gateway/gateway.pgc" -e "@/tmp/backups/artifact/secrets.yml"

Example:

# ansible localhost -m shell -a "export PGPASSWORD={{ gateway_pg_password }}; pg_dump -h {{ gateway_pg_host }} -U {{ gateway_pg_username }} -d {{ gateway_pg_database }} -p {{ gateway_pg_port }} --clean --create -Fc -f /tmp/backups/artifact/gateway/gateway.pgc" -e "@/tmp/backups/artifact/secrets.yml"
localhost | CHANGED | rc=0 >>

3.4.2 Back up the Controller database

Run the following command on the RPM Gateway node as the root user:

ansible localhost -m shell -a "export PGPASSWORD={{ controller_pg_password }}; pg_dump -h {{ controller_pg_host }} -U {{ controller_pg_username }} -d {{ controller_pg_database }} -p {{ controller_pg_port }} --clean --create -Fc -f /tmp/backups/artifact/controller/controller.pgc" -e "@/tmp/backups/artifact/secrets.yml"

3.4.3 Back up the Hub database

Run the following command on the RPM Gateway node as the root user:

ansible localhost -m shell -a "export PGPASSWORD={{ hub_pg_password }}; pg_dump -h {{ hub_pg_host }} -U {{ hub_pg_username }} -d {{ hub_pg_database }} -p {{ hub_pg_port }} --clean --create -Fc -f /tmp/backups/artifact/hub/hub.pgc" -e "@/tmp/backups/artifact/secrets.yml"

3.4.4 Back up the EDA database

Run the following command on the RPM Gateway node as the root user:

ansible localhost -m shell -a "export PGPASSWORD={{ eda_pg_password }}; pg_dump -h {{ eda_pg_host }} -U {{ eda_pg_username }} -d {{ eda_pg_database }} -p {{ eda_pg_port }} --clean --create -Fc -f /tmp/backups/artifact/eda/eda.pgc" -e "@/tmp/backups/artifact/secrets.yml"

3.4.5 Verify the backups exist

Run the following command on the RPM Gateway node as the root user:

find /tmp/backups/artifact/ -name "*.pgc"

Example:

# find /tmp/backups/artifact/ -name "*.pgc"
/tmp/backups/artifact/controller/controller.pgc
/tmp/backups/artifact/gateway/gateway.pgc
/tmp/backups/artifact/hub/hub.pgc
/tmp/backups/artifact/eda/eda.pgc

3.5 Create the sha256sum.txt and verify it was created

3.5.1 Create the sha256sum.txt file

Run the following command on the Gateway node as the root user:

[ -f /tmp/backups/artifact/sha256sum.txt ] && rm -f /tmp/backups/artifact/sha256sum.txt; find /tmp/backups/artifact/ -type f -name "*.pgc" -exec sha256sum {} \; >> /tmp/backups/artifact/sha256sum.txt

3.5.2 Verify the file was created

Run the following command on the Gateway node as the root user:

ls -l /tmp/backups/artifact/sha256sum.txt

Example:

# ls -l /tmp/backups/artifact/sha256sum.txt
-rw-r--r--. 1 root root 322 Mar 15 20:23 /tmp/backups/artifact/sha256sum.txt

3.6 Create the artifact.tar and verify it was created

3.6.1 Create the artifact.tar file

Run the following command on the Gateway node as the root user:

cd /tmp/backups/ && tar cf artifact.tar artifact && mv artifact.tar /tmp

3.6.2 Verify the file was created

Run the following command on the Gateway node as the root user:

ls -l /tmp/artifact.tar

Example:

# ls -l /tmp/artifact.tar
-rw-r--r--. 1 root root 30515200 Mar 15 20:28 /tmp/artifact.tar

3.7 Create and validate the artifact.tar.sha256

3.7.1 Create a sha256 file for artifact.tar

Run the following command on the Gateway node as the root user:

sha256sum /tmp/artifact.tar > /tmp/artifact.tar.sha256

3.7.2 Validate the sha256 file

Run the following command on the Gateway node as the root user:

sha256sum --check /tmp/artifact.tar.sha256

Example:

sha256sum --check /tmp/artifact.tar.sha256
/tmp/artifact.tar: OK

3.7.3 Validate the contents of artifact.tar

Run the following command on the Gateway node as the root user:

tar tvf /tmp/artifact.tar

Example:

# tar tvf /tmp/artifact.tar
drwxr-xr-x root/root         0 2026-03-30 22:58 artifact/
drwxr-xr-x root/root         0 2026-03-30 22:47 artifact/controller/
drwxr-xr-x root/root         0 2026-03-30 22:42 artifact/controller/custom_configs/
-rw-r--r-- root/root  35808771 2026-03-30 22:51 artifact/controller/controller.pgc
drwxr-xr-x root/root         0 2026-03-30 22:53 artifact/gateway/
-rw-r--r-- root/root    306333 2026-03-30 22:53 artifact/gateway/gateway.pgc
drwxr-xr-x root/root         0 2026-03-30 22:54 artifact/hub/
-rw-r--r-- root/root  11759565 2026-03-30 22:56 artifact/hub/hub.pgc
drwxr-xr-x root/root         0 2026-03-30 22:57 artifact/eda/
-rw-r--r-- root/root    482932 2026-03-30 22:57 artifact/eda/eda.pgc
-rw------- root/root      5085 2026-03-30 22:57 artifact/secrets.yml
-rw-r--r-- root/root       422 2026-03-30 22:58 artifact/sha256sum.txt

4. Import the artifact to the Containerized environment

4.1 Move artifact to the Containerized environment

4.1.1 SSH into the main Containerized Gateway node as the ansible_user

This content is not included.Instructions to create the ansible_user

4.1.2 Copy the artifact.tar and artifact.tar.sha256 from the RPM environment to the Containerized environment as the ansible_user

Run the following command on the Containerized Gateway node as the ansible_user:

sudo scp $SSH_USER@$RPM_GATEWAY_NODE:/tmp/artifact.tar* /tmp

Example:

$ sudo scp my-ssh-user@my.rpm.gateway:/tmp/artifact.tar* /tmp
artifact.tar                                        100%  248MB 555.3MB/s   00:00    
artifact.tar.sha256                                 100%   84   160.9KB/s   00:00

4.1.3 Change permissions of artifact.tar and artifact.tar.sha256

Run the following command on the Containerized Gateway node as the ansible_user:

sudo chown $ANSIBLE_USER:$ANSIBLE_USER /tmp/artifact.tar /tmp/artifact.tar.sha256

Example:

$ sudo chown ansible_user:ansible_user /tmp/artifact.tar /tmp/artifact.tar.sha256

4.1.4 Verify permissions were set correctly

Run the following command on the Containerized Gateway node as the ansible_user:

ls -l /tmp/artifact*

Example:

$ ls -l /tmp/artifact*
-rw-r--r--. 1 ansible_user ansible_user 259727360 Jun 28 13:21 /tmp/artifact.tar
-rw-r--r--. 1 ansible_user ansible_user        84 Jun 28 13:21 /tmp/artifact.tar.sha256

4.2 Ensure transfers were completed successfully by checking the sha256sum

4.2.1 Validate the artifact.tar.sha256 as the ansible_user

Run the following command on the Containerized Gateway node as the ansible_user:

sha256sum --check /tmp/artifact.tar.sha256

Example:

$ sha256sum --check /tmp/artifact.tar.sha256
/tmp/artifact.tar: OK

4.2.2 Extract the artifact.tar

Run the following command on the Containerized Gateway node as the ansible_user:

cd /tmp && tar xf /tmp/artifact.tar

4.2.3 Validate the sha256sum.txt

Run the following command on the Containerized Gateway node as the ansible_user:

sed 's/backups\///g' /tmp/artifact/sha256sum.txt | sha256sum --check

4.3 Download and extract the latest Ansible Automation Platform Containerized installer

Perform only the "Downloading Ansible Automation Platform" steps (section 4.5 in the linked document):
This content is not included.Downloading Ansible Automation Platform

NOTE: Perform the unpacking steps as the ansible_user.

4.4 Update the secrets.yml

4.4.1 Ensure ansible-core is installed

Run the following command on the Containerized Gateway node as the ansible_user:

sudo dnf install ansible-core

Example:

$ sudo dnf install ansible-core
(.....)
Installed:
  ansible-core-1:2.14.18-3.el9.x86_64      git-core-2.52.0-1.el9.x86_64              python3-cffi-1.14.5-5.el9.x86_64     python3-cryptography-36.0.1-5.el9_6.x86_64     python3-packaging-20.9-5.el9.noarch     python3-ply-3.11-14.el9.noarch     python3-pycparser-2.20-6.el9.noarch    
  python3-pyparsing-2.4.7-9.el9.noarch     python3-resolvelib-0.5.4-5.el9.noarch     sshpass-1.09-4.el9.x86_64           

Complete!

4.4.2 View the current database

Run the following command on the Containerized Gateway node as the ansible_user:

grep pg_host /tmp/artifact/secrets.yml

Example:

$ grep pg_host /tmp/artifact/secrets.yml 
gateway_pg_host: my.rpm.database
controller_pg_host: my.rpm.database
hub_pg_host: my.rpm.database
eda_pg_host: my.rpm.database

4.4.3 Update the pg_host settings in the secrets.yml

Run the following command on the Containerized Gateway node as the ansible_user:

sed -i 's/$RPM_DATABASE_SERVER/$CONTAINERIZED_DATABASE_SERVER/g' /tmp/artifact/secrets.yml

Example:

sed -i 's/my.rpm.database/my.containerized.database/g' /tmp/artifact/secrets.yml

4.4.4 Verify the change was made

Run the following command on the Containerized Gateway node as the ansible_user:

grep pg_host /tmp/artifact/secrets.yml

Example:

$ grep pg_host /tmp/artifact/secrets.yml 
gateway_pg_host: my.containerized.database
controller_pg_host: my.containerized.database
hub_pg_host: my.containerized.database
eda_pg_host: my.containerized.database

4.5 Update the inventory file

Follow the steps for This content is not included.Configuring the inventory file

NOTES:

Only add the component types that are in your RPM environment. You can add other components after the migration has completed successfully.
We will use the secrets.yml file when running the installer to pass in the database variables.
You do not have to update the following variables and can leave them as is:
- *_pg_host=externaldb.example.org
- *_pg_database=<set your own>
- *_pg_username=<set your own>
- *_pg_password=<set your own>
- *_pg_port

Example of updated bundled installer inventory-growth file:

[automationgateway]
my.containerized.gateway

[automationcontroller]
my.containerized.controller

[automationhub]
my.containerized.hub

[automationeda]
my.containerized.eda

[database]
my.containerized.database

[all:vars]
postgresql_admin_username=postgres
postgresql_admin_password=MySecurePostgresPassword

bundle_install=true
bundle_dir='{{ lookup("ansible.builtin.env", "PWD") }}/bundle'

gateway_admin_password=MySecureGatewayAdminPassword
gateway_pg_host=aap.example.org
gateway_pg_password=<set your own>

controller_admin_password=MySecureControllerAdminPassword
controller_pg_host=aap.example.org
controller_pg_password=<set your own>
controller_percent_memory_capacity=0.5

hub_admin_password=MySecureHubAdminPassword
hub_pg_host=aap.example.org
hub_pg_password=<set your own>
hub_seed_collections=false

eda_admin_password=MySecureEdaAdminPassword
eda_pg_host=aap.example.org
eda_pg_password=<set your own>

4.6 Run the containerized installer with secrets.yml

Run the following command on the Containerized Gateway node as the ansible_user from within the installer directory:

ansible-playbook -i inventory-growth ansible.containerized_installer.install -e @/tmp/artifact/secrets.yml -e "__hub_database_fields='{{ hub_db_fields_encryption_key }}'"

NOTES:

The above command may need the name of the inventory file changed.
-e @secrets.yml passes in the variables to the installer playbook and overrides the values in the inventory file.
-e "__hub_database_fields='{{ hub_db_fields_encryption_key }}'" tells the installer to include the __hub_database_fields variable using the value for hub_db_fields_encryption_key.

4.7 Verify the installation was successful

IMPORTANT: It is very important to perform ALL of these validation steps. Failure to do so can put your environment into an unrecoverable state.

4.7.1 Check for errors in the aap_install.log

The aap_install.log is appended to, so previous installation attempts may be included.
Review the aap_install.log for tasks that failed but weren't ignored.

4.7.2 Verify the SECRET_KEYs and hub_db_fields_encryption_key exist and are correct for each component

Run the following commands on the Containerized Gateway node as the ansible_user:

ansible automationgateway -i inventory-growth -m shell -a 'KEYS="$(mktemp)"; chmod 600 "$KEYS"; cat /tmp/artifact/secrets.yml | egrep '\''gateway_secret_key'\'' | sed -E "s/^[^:]+:[[:space:]]*'\''//; s/'\''[[:space:]]*$//" > "$KEYS"; MATCHES="$(grep -rIlF -f "$KEYS" "$HOME/.local/share/containers/storage/overlay-containers/" 2>/dev/null)"; rm -f "$KEYS"; status(){ echo "$MATCHES" | grep -qF "$1" && echo OK || echo FAIL; }; printf "gateway_secret_key: %s\n" "$(status "userdata/secrets/gateway_secret_key")"'

ansible automationcontroller -i inventory-growth -m shell -a 'KEYS="$(mktemp)"; chmod 600 "$KEYS"; cat /tmp/artifact/secrets.yml | egrep '\''controller_secret_key'\'' | sed -E "s/^[^:]+:[[:space:]]*'\''//; s/'\''[[:space:]]*$//" > "$KEYS"; MATCHES="$(grep -rIlF -f "$KEYS" "$HOME/.local/share/containers/storage/overlay-containers/" 2>/dev/null)"; rm -f "$KEYS"; status(){ echo "$MATCHES" | grep -qF "$1" && echo OK || echo FAIL; }; printf "controller_secret_key: %s\n" "$(status "userdata/secrets/controller_secret_key")"'

ansible automationhub -i inventory-growth -m shell -a 'KEYS="$(mktemp)"; chmod 600 "$KEYS"; cat /tmp/artifact/secrets.yml | egrep '\''hub_secret_key|hub_db_fields_encryption_key'\'' | sed -E "s/^[^:]+:[[:space:]]*'\''//; s/'\''[[:space:]]*$//" > "$KEYS"; MATCHES="$(grep -rIlF -f "$KEYS" "$HOME/.local/share/containers/storage/overlay-containers/" 2>/dev/null)"; rm -f "$KEYS"; status(){ echo "$MATCHES" | grep -qF "$1" && echo OK || echo FAIL; }; printf "hub_secret_key: %s\n" "$(status "userdata/config.json")"; printf "hub_db_fields_encryption_key: %s\n" "$(status "userdata/secrets/hub_database_fields")"'

ansible automationeda -i inventory-growth -m shell -a 'KEYS="$(mktemp)"; chmod 600 "$KEYS"; cat /tmp/artifact/secrets.yml | egrep '\''eda_secret_key'\'' | sed -E "s/^[^:]+:[[:space:]]*'\''//; s/'\''[[:space:]]*$//" > "$KEYS"; MATCHES="$(grep -rIlF -f "$KEYS" "$HOME/.local/share/containers/storage/overlay-containers/" 2>/dev/null)"; rm -f "$KEYS"; status(){ echo "$MATCHES" | grep -qF "$1" && echo OK || echo FAIL; }; printf "eda_secret_key: %s\n" "$(status "userdata/secrets/eda_secret_key")"'

Example:

$ ansible automationgateway -i inventory-growth -m shell -a 'KEYS="$(mktemp)"; chmod 600 "$KEYS"; cat /tmp/artifact/secrets.yml | egrep '\''gateway_secret_key'\'' | sed -E "s/^[^:]+:[[:space:]]*'\''//; s/'\''[[:space:]]*$//" > "$KEYS"; MATCHES="$(grep -rIlF -f "$KEYS" "$HOME/.local/share/containers/storage/overlay-containers/" 2>/dev/null)"; rm -f "$KEYS"; status(){ echo "$MATCHES" | grep -qF "$1" && echo OK || echo FAIL; }; printf "gateway_secret_key: %s\n" "$(status "userdata/secrets/gateway_secret_key")"'
my.containerized.node | CHANGED | rc=0 >>
gateway_secret_key: OK

$ ansible automationcontroller -i inventory-growth -u cloud-user -m shell -a 'KEYS="$(mktemp)"; chmod 600 "$KEYS"; cat /tmp/artifact/secrets.yml | egrep '\''controller_secret_key'\'' | sed -E "s/^[^:]+:[[:space:]]*'\''//; s/'\''[[:space:]]*$//" > "$KEYS"; MATCHES="$(grep -rIlF -f "$KEYS" "$HOME/.local/share/containers/storage/overlay-containers/" 2>/dev/null)"; rm -f "$KEYS"; status(){ echo "$MATCHES" | grep -qF "$1" && echo OK || echo FAIL; }; printf "controller_secret_key: %s\n" "$(status "userdata/secrets/controller_secret_key")"'
my.containerized.node | CHANGED | rc=0 >>
controller_secret_key: OK

$ ansible automationhub -i inventory-growth -u cloud-user -m shell -a 'KEYS="$(mktemp)"; chmod 600 "$KEYS"; cat /tmp/artifact/secrets.yml | egrep '\''hub_secret_key|hub_db_fields_encryption_key'\'' | sed -E "s/^[^:]+:[[:space:]]*'\''//; s/'\''[[:space:]]*$//" > "$KEYS"; MATCHES="$(grep -rIlF -f "$KEYS" "$HOME/.local/share/containers/storage/overlay-containers/" 2>/dev/null)"; rm -f "$KEYS"; status(){ echo "$MATCHES" | grep -qF "$1" && echo OK || echo FAIL; }; printf "hub_secret_key: %s\n" "$(status "userdata/config.json")"; printf "hub_db_fields_encryption_key: %s\n" "$(status "userdata/secrets/hub_database_fields")"'
my.containerized.node | CHANGED | rc=0 >>
hub_secret_key: OK
hub_db_fields_encryption_key: OK

$ ansible automationeda -i inventory-growth -u cloud-user -m shell -a 'KEYS="$(mktemp)"; chmod 600 "$KEYS"; cat /tmp/artifact/secrets.yml | egrep '\''eda_secret_key'\'' | sed -E "s/^[^:]+:[[:space:]]*'\''//; s/'\''[[:space:]]*$//" > "$KEYS"; MATCHES="$(grep -rIlF -f "$KEYS" "$HOME/.local/share/containers/storage/overlay-containers/" 2>/dev/null)"; rm -f "$KEYS"; status(){ echo "$MATCHES" | grep -qF "$1" && echo OK || echo FAIL; }; printf "eda_secret_key: %s\n" "$(status "userdata/secrets/eda_secret_key")"'
my.containerized.node | CHANGED | rc=0 >>
eda_secret_key: OK

NOTE: The above commands may need the name of the inventory file changed.

4.7.3 Log into the AAP UI as the admin user

NOTE: You do not need to add the subscription at this time. Seeing the login screen and being able to log into AAP should be enough validation for now.

4.8 Restore the databases

4.8.1 Stop container services on the Containerized nodes

Run the following command on the Containerized Gateway node as the ansible_user from within the installer directory:

ansible automationgateway -i inventory-growth -m ansible.builtin.shell -a 'for svc in automation-gateway.service automation-gateway-proxy.service redis-tcp.service redis-unix.service pcp.service; do load=$(systemctl --user show -p LoadState --value "$svc" 2>/dev/null || echo not-found); if [ "$load" = loaded ]; then systemctl --user stop "$svc" && echo "stopped: $svc" || echo "stop failed: $svc"; else echo "not installed: $svc"; fi; done'

ansible automationcontroller -i inventory-growth -m ansible.builtin.shell -a 'for svc in automation-controller-rsyslog.service automation-controller-task.service automation-controller-web.service metrics-utility-gather.service metrics-utility-build-report.service metrics-utility-gather.timer metrics-utility-build-report.timer receptor.service pcp.service; do load=$(systemctl --user show -p LoadState --value "$svc" 2>/dev/null || echo not-found); if [ "$load" = loaded ]; then systemctl --user stop "$svc" && echo "stopped: $svc" || echo "stop failed: $svc"; else echo "not installed: $svc"; fi; done'

ansible automationhub -i inventory-growth -m ansible.builtin.shell -a 'for svc in automation-hub-api.service automation-hub-content.service automation-hub-web.service automation-hub-worker-1.service automation-hub-worker-2.service pcp.service; do load=$(systemctl --user show -p LoadState --value "$svc" 2>/dev/null || echo not-found); if [ "$load" = loaded ]; then systemctl --user stop "$svc" && echo "stopped: $svc" || echo "stop failed: $svc"; else echo "not installed: $svc"; fi; done'

ansible automationeda -i inventory-growth -m ansible.builtin.shell -a 'for svc in automation-eda-api.service automation-eda-web.service automation-eda-daphne.service automation-eda-worker-1.service automation-eda-worker-2.service automation-eda-activation-worker-1.service automation-eda-activation-worker-2.service automation-eda-scheduler.service pcp.service; do load=$(systemctl --user show -p LoadState --value "$svc" 2>/dev/null || echo not-found); if [ "$load" = loaded ]; then systemctl --user stop "$svc" && echo "stopped: $svc" || echo "stop failed: $svc"; else echo "not installed: $svc"; fi; done'

ansible redis -i inventory-growth -m ansible.builtin.shell -a 'for svc in redis-tcp.service redis-unix.service; do load=$(systemctl --user show -p LoadState --value "$svc" 2>/dev/null || echo not-found); if [ "$load" = loaded ]; then systemctl --user stop "$svc" && echo "stopped: $svc" || echo "stop failed: $svc"; else echo "not installed: $svc"; fi; done'

Example:

$ ansible automationgateway -i inventory-growth -m ansible.builtin.shell -a 'for svc in automation-gateway.service automation-gateway-proxy.service redis-tcp.service redis-unix.service pcp.service; do load=$(systemctl --user show -p LoadState --value "$svc" 2>/dev/null || echo not-found); if [ "$load" = loaded ]; then systemctl --user stop "$svc" && echo "stopped: $svc" || echo "stop failed: $svc"; else echo "not installed: $svc"; fi; done'
my.containerized.gateway | CHANGED | rc=0 >>
stopped: automation-gateway.service
stopped: automation-gateway-proxy.service
stopped: redis-tcp.service
stopped: redis-unix.service
not installed: pcp.service

$ ansible automationcontroller -i inventory-growth -m ansible.builtin.shell -a 'for svc in automation-controller-rsyslog.service automation-controller-task.service automation-controller-web.service metrics-utility-gather.service metrics-utility-build-report.service metrics-utility-gather.timer metrics-utility-build-report.timer receptor.service pcp.service; do load=$(systemctl --user show -p LoadState --value "$svc" 2>/dev/null || echo not-found); if [ "$load" = loaded ]; then systemctl --user stop "$svc" && echo "stopped: $svc" || echo "stop failed: $svc"; else echo "not installed: $svc"; fi; done'
my.containerized.controller | CHANGED | rc=0 >>
stopped: automation-controller-rsyslog.service
stopped: automation-controller-task.service
stopped: automation-controller-web.service
not installed: metrics-utility-gather.service
not installed: metrics-utility-build-report.service
not installed: metrics-utility-gather.timer
not installed: metrics-utility-build-report.timer
stopped: receptor.service
not installed: pcp.service

$ ansible automationhub -i inventory-growth -m ansible.builtin.shell -a 'for svc in automation-hub-api.service automation-hub-content.service automation-hub-web.service automation-hub-worker-1.service automation-hub-worker-2.service pcp.service; do load=$(systemctl --user show -p LoadState --value "$svc" 2>/dev/null || echo not-found); if [ "$load" = loaded ]; then systemctl --user stop "$svc" && echo "stopped: $svc" || echo "stop failed: $svc"; else echo "not installed: $svc"; fi; done'
my.containerized.hub | CHANGED | rc=0 >>
stopped: automation-hub-api.service
stopped: automation-hub-content.service
stopped: automation-hub-web.service
stopped: automation-hub-worker-1.service
stopped: automation-hub-worker-2.service
not installed: pcp.service

$ ansible automationeda -i inventory-growth -m ansible.builtin.shell -a 'for svc in automation-eda-api.service automation-eda-web.service automation-eda-daphne.service automation-eda-worker-1.service automation-eda-worker-2.service automation-eda-activation-worker-1.service automation-eda-activation-worker-2.service automation-eda-scheduler.service pcp.service; do load=$(systemctl --user show -p LoadState --value "$svc" 2>/dev/null || echo not-found); if [ "$load" = loaded ]; then systemctl --user stop "$svc" && echo "stopped: $svc" || echo "stop failed: $svc"; else echo "not installed: $svc"; fi; done'
my.containerized.eda | CHANGED | rc=0 >>
stopped: automation-eda-api.service
stopped: automation-eda-web.service
stopped: automation-eda-daphne.service
stopped: automation-eda-worker-1.service
stopped: automation-eda-worker-2.service
stopped: automation-eda-activation-worker-1.service
stopped: automation-eda-activation-worker-2.service
stopped: automation-eda-scheduler.service
not installed: pcp.service

$ ansible redis -i inventory-growth -m ansible.builtin.shell -a 'for svc in redis-tcp.service redis-unix.service; do load=$(systemctl --user show -p LoadState --value "$svc" 2>/dev/null || echo not-found); if [ "$load" = loaded ]; then systemctl --user stop "$svc" && echo "stopped: $svc" || echo "stop failed: $svc"; else echo "not installed: $svc"; fi; done'
[WARNING]: Could not match supplied host pattern, ignoring: redis
[WARNING]: No hosts matched, nothing to do

4.8.2 Change permission of the artifact directory so it is accessible within the container

Run the following command on the Containerized Gateway node as the ansible_user:

chmod -R a+rX /tmp/artifact/

4.8.3a UNMANAGED DATABASE - Restore the database backups for customer-provided databases

IMPORTANT: Only perform these steps if you are using an unmanaged database (if NO database server is listed under the [database] section of your inventory file).
If you are using a managed database, go to step 4.8.3b MANAGED DATABASE - Restore the database backups for managed databases.

a. Provide the Create DB role for the component database users

b. Perform a pg_restore using the pgc files

c. Revoke the Create DB role for the component database users

4.8.3b MANAGED DATABASE - Restore the database backups for managed databases

IMPORTANT: Only perform these steps if you are using a managed database (if the database server is listed under the [database] section of your inventory file).
If you are using a customer-managed database, go to step 4.8.3a UNMANAGED DATABASE - Restore the database backups for customer-provided databases.

a. Update secrets.yml with the postgres user's password

Run the following command on the Containerized Gateway node as the ansible_user from within the installer directory:

grep postgresql_admin_password inventory-growth >> /tmp/artifact/secrets.yml

NOTE: You may need to modify the inventory file name.

b. Verify the secrets.yml was updated

Run the following command on the Containerized Gateway node as the ansible_user:

grep postgresql_admin_password /tmp/artifact/secrets.yml

Example:

$ grep postgresql_admin_password /tmp/artifact/secrets.yml 
postgresql_admin_password=test

NOTE: It is important to keep the format using the "=" instead of the ":"

c. Create a temporary postgres container to import the database backups

Run the following command on the Containerized Gateway node as the ansible_user:

podman run -it --rm --name postgresql_restore_temp --network host --volume ~/aap/tls/extracted:/etc/pki/ca-trust/extracted:z --volume ~/aap/postgresql/server.crt:/var/lib/pgsql/server.crt:ro,z --volume ~/aap/postgresql/server.key:/var/lib/pgsql/server.key:ro,z --volume /tmp/artifact:/var/lib/pgsql/backups:ro,z registry.redhat.io/rhel9/postgresql-15:latest bash

d. Provide the Create DB role to the component database users

bash-5.1$ GATEWAY_PG_USERNAME=$(awk -F': ' '/^gateway_pg_username:/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); GATEWAY_PG_HOST=$(awk -F': ' '/^gateway_pg_host:/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); POSTGRESQL_ADMIN_PASSWORD=$(awk -F'=' '/^postgresql_admin_password=/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); PGPASSWORD="$POSTGRESQL_ADMIN_PASSWORD" psql -h "$GATEWAY_PG_HOST" -U postgres -c "alter role $GATEWAY_PG_USERNAME with createdb;"

bash-5.1$ CONTROLLER_PG_USERNAME=$(awk -F': ' '/^controller_pg_username:/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); CONTROLLER_PG_HOST=$(awk -F': ' '/^controller_pg_host:/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); POSTGRESQL_ADMIN_PASSWORD=$(awk -F'=' '/^postgresql_admin_password=/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); PGPASSWORD=$POSTGRESQL_ADMIN_PASSWORD psql -h $CONTROLLER_PG_HOST -U postgres -c "alter role $CONTROLLER_PG_USERNAME with createdb;"

bash-5.1$ HUB_PG_USERNAME=$(awk -F': ' '/^hub_pg_username:/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); HUB_PG_HOST=$(awk -F': ' '/^hub_pg_host:/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); POSTGRESQL_ADMIN_PASSWORD=$(awk -F'=' '/^postgresql_admin_password=/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); PGPASSWORD=$POSTGRESQL_ADMIN_PASSWORD psql -h $HUB_PG_HOST -U postgres -c "alter role $HUB_PG_USERNAME with createdb;"

bash-5.1$ EDA_PG_USERNAME=$(awk -F': ' '/^eda_pg_username:/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); EDA_PG_HOST=$(awk -F': ' '/^eda_pg_host:/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); POSTGRESQL_ADMIN_PASSWORD=$(awk -F'=' '/^postgresql_admin_password=/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); PGPASSWORD=$POSTGRESQL_ADMIN_PASSWORD psql -h $EDA_PG_HOST -U postgres -c "alter role $EDA_PG_USERNAME with createdb;"

Example:

bash-5.1$ GATEWAY_PG_USERNAME=$(awk -F': ' '/^gateway_pg_username:/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); GATEWAY_PG_HOST=$(awk -F': ' '/^gateway_pg_host:/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); POSTGRESQL_ADMIN_PASSWORD=$(awk -F'=' '/^postgresql_admin_password=/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); PGPASSWORD="$POSTGRESQL_ADMIN_PASSWORD" psql -h "$GATEWAY_PG_HOST" -U postgres -c "alter role $GATEWAY_PG_USERNAME with createdb;"
ALTER ROLE

bash-5.1$ CONTROLLER_PG_USERNAME=$(awk -F': ' '/^controller_pg_username:/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); CONTROLLER_PG_HOST=$(awk -F': ' '/^controller_pg_host:/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); POSTGRESQL_ADMIN_PASSWORD=$(awk -F'=' '/^postgresql_admin_password=/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); PGPASSWORD=$POSTGRESQL_ADMIN_PASSWORD psql -h $CONTROLLER_PG_HOST -U postgres -c "alter role $CONTROLLER_PG_USERNAME with createdb;"
ALTER ROLE

bash-5.1$ HUB_PG_USERNAME=$(awk -F': ' '/^hub_pg_username:/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); HUB_PG_HOST=$(awk -F': ' '/^hub_pg_host:/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); POSTGRESQL_ADMIN_PASSWORD=$(awk -F'=' '/^postgresql_admin_password=/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); PGPASSWORD=$POSTGRESQL_ADMIN_PASSWORD psql -h $HUB_PG_HOST -U postgres -c "alter role $HUB_PG_USERNAME with createdb;"
ALTER ROLE

bash-5.1$ EDA_PG_USERNAME=$(awk -F': ' '/^eda_pg_username:/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); EDA_PG_HOST=$(awk -F': ' '/^eda_pg_host:/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); POSTGRESQL_ADMIN_PASSWORD=$(awk -F'=' '/^postgresql_admin_password=/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); PGPASSWORD=$POSTGRESQL_ADMIN_PASSWORD psql -h $EDA_PG_HOST -U postgres -c "alter role $EDA_PG_USERNAME with createdb;"
ALTER ROLE

e. Verify database users have the Create DB role

bash-5.1$ GATEWAY_PG_USERNAME=$(awk -F': ' '/^gateway_pg_username:/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); GATEWAY_PG_HOST=$(awk -F': ' '/^gateway_pg_host:/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); POSTGRESQL_ADMIN_PASSWORD=$(awk -F'=' '/^postgresql_admin_password=/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); PGPASSWORD="$POSTGRESQL_ADMIN_PASSWORD" psql -h "$GATEWAY_PG_HOST" -U postgres -c "\du"

Example:

bash-5.1$ GATEWAY_PG_USERNAME=$(awk -F': ' '/^gateway_pg_username:/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); GATEWAY_PG_HOST=$(awk -F': ' '/^gateway_pg_host:/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); POSTGRESQL_ADMIN_PASSWORD=$(awk -F'=' '/^postgresql_admin_password=/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); PGPASSWORD="$POSTGRESQL_ADMIN_PASSWORD" psql -h "$GATEWAY_PG_HOST" -U postgres -c "\du"
                                                                    List of roles
        Role name        |                         Attributes                         |                           Member of                           
-------------------------+------------------------------------------------------------+---------------------------------------------------------------
 automationedacontroller | Create DB                                                  | {}
 automationgateway       | Create DB                                                  | {}
 automationhub           | Create DB                                                  | {}
 awx                     | Create DB                                                  | {}
 postgres                | Superuser, Create role, Create DB, Replication, Bypass RLS | {awx,automationhub,automationedacontroller,automationgateway}

f. Restore the Gateway database backup

bash-5.1$ GATEWAY_PG_HOST=$(awk -F': ' '/^gateway_pg_host:/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); GATEWAY_PG_USERNAME=$(awk -F': ' '/^gateway_pg_username:/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); GATEWAY_PG_PASSWORD=$(awk -F': ' '/^gateway_pg_password:/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); PGPASSWORD="$GATEWAY_PG_PASSWORD" pg_restore --clean --create --no-owner -h "$GATEWAY_PG_HOST" -U "$GATEWAY_PG_USERNAME" -d template1 /var/lib/pgsql/backups/gateway/gateway.pgc

g. Restore the Controller database backup

bash-5.1$ CONTROLLER_PG_HOST=$(awk -F': ' '/^controller_pg_host:/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); CONTROLLER_PG_USERNAME=$(awk -F': ' '/^controller_pg_username:/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); CONTROLLER_PG_PASSWORD=$(awk -F': ' '/^controller_pg_password:/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); PGPASSWORD="$CONTROLLER_PG_PASSWORD" pg_restore --clean --create --no-owner -h "$CONTROLLER_PG_HOST" -U "$CONTROLLER_PG_USERNAME" -d template1 /var/lib/pgsql/backups/controller/controller.pgc

h. Restore the Hub database backup

bash-5.1$ HUB_PG_HOST=$(awk -F': ' '/^hub_pg_host:/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); HUB_PG_USERNAME=$(awk -F': ' '/^hub_pg_username:/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); HUB_PG_PASSWORD=$(awk -F': ' '/^hub_pg_password:/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); PGPASSWORD="$HUB_PG_PASSWORD" pg_restore --clean --create --no-owner -h "$HUB_PG_HOST" -U "$HUB_PG_USERNAME" -d template1 /var/lib/pgsql/backups/hub/hub.pgc

i. Restore the EDA database backup

bash-5.1$ EDA_PG_HOST=$(awk -F': ' '/^eda_pg_host:/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); EDA_PG_USERNAME=$(awk -F': ' '/^eda_pg_username:/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); EDA_PG_PASSWORD=$(awk -F': ' '/^eda_pg_password:/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); PGPASSWORD="$EDA_PG_PASSWORD" pg_restore --clean --create --no-owner -h "$EDA_PG_HOST" -U "$EDA_PG_USERNAME" -d template1 /var/lib/pgsql/backups/eda/eda.pgc

j. Revoke the CREATEDB role for the component database users

bash-5.1$ GATEWAY_PG_USERNAME=$(awk -F': ' '/^gateway_pg_username:/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); GATEWAY_PG_HOST=$(awk -F': ' '/^gateway_pg_host:/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); POSTGRESQL_ADMIN_PASSWORD=$(awk -F'=' '/^postgresql_admin_password=/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); PGPASSWORD="$POSTGRESQL_ADMIN_PASSWORD" psql -h "$GATEWAY_PG_HOST" -U postgres -c "alter role $GATEWAY_PG_USERNAME with nocreatedb;"

bash-5.1$ CONTROLLER_PG_USERNAME=$(awk -F': ' '/^controller_pg_username:/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); CONTROLLER_PG_HOST=$(awk -F': ' '/^controller_pg_host:/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); POSTGRESQL_ADMIN_PASSWORD=$(awk -F'=' '/^postgresql_admin_password=/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); PGPASSWORD=$POSTGRESQL_ADMIN_PASSWORD psql -h $CONTROLLER_PG_HOST -U postgres -c "alter role $CONTROLLER_PG_USERNAME with nocreatedb;"

bash-5.1$ HUB_PG_USERNAME=$(awk -F': ' '/^hub_pg_username:/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); HUB_PG_HOST=$(awk -F': ' '/^hub_pg_host:/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); POSTGRESQL_ADMIN_PASSWORD=$(awk -F'=' '/^postgresql_admin_password=/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); PGPASSWORD=$POSTGRESQL_ADMIN_PASSWORD psql -h $HUB_PG_HOST -U postgres -c "alter role $HUB_PG_USERNAME with nocreatedb;"

bash-5.1$ EDA_PG_USERNAME=$(awk -F': ' '/^eda_pg_username:/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); EDA_PG_HOST=$(awk -F': ' '/^eda_pg_host:/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); POSTGRESQL_ADMIN_PASSWORD=$(awk -F'=' '/^postgresql_admin_password=/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); PGPASSWORD=$POSTGRESQL_ADMIN_PASSWORD psql -h $EDA_PG_HOST -U postgres -c "alter role $EDA_PG_USERNAME with nocreatedb;"

k. Verify database users DO NOT have the create db role

bash-5.1$ GATEWAY_PG_USERNAME=$(awk -F': ' '/^gateway_pg_username:/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); GATEWAY_PG_HOST=$(awk -F': ' '/^gateway_pg_host:/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); POSTGRESQL_ADMIN_PASSWORD=$(awk -F'=' '/^postgresql_admin_password=/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); PGPASSWORD="$POSTGRESQL_ADMIN_PASSWORD" psql -h "$GATEWAY_PG_HOST" -U postgres -c "\du"

Example:

bash-5.1$ GATEWAY_PG_USERNAME=$(awk -F': ' '/^gateway_pg_username:/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); GATEWAY_PG_HOST=$(awk -F': ' '/^gateway_pg_host:/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); POSTGRESQL_ADMIN_PASSWORD=$(awk -F'=' '/^postgresql_admin_password=/ {print $2; exit}' /var/lib/pgsql/backups/secrets.yml); PGPASSWORD="$POSTGRESQL_ADMIN_PASSWORD" psql -h "$GATEWAY_PG_HOST" -U postgres -c "\du"

                                                                    List of roles
        Role name        |                         Attributes                         |                           Member of                           
-------------------------+------------------------------------------------------------+---------------------------------------------------------------
 automationedacontroller |                                                            | {}
 automationgateway       |                                                            | {}
 automationhub           |                                                            | {}
 awx                     |                                                            | {}
 postgres                | Superuser, Create role, Create DB, Replication, Bypass RLS | {awx,automationhub,automationedacontroller,automationgateway}

l. Exit the container

bash-5.1$ exit

4.8.4 Start container services on the Containerized nodes

Run the following commands in the installer directory as the ansible_user to start services

ansible automationgateway -i inventory-growth -m ansible.builtin.shell -a 'for svc in automation-gateway.service automation-gateway-proxy.service redis-tcp.service redis-unix.service pcp.service; do load=$(systemctl --user show -p LoadState --value "$svc" 2>/dev/null || echo not-found); if [ "$load" = loaded ]; then systemctl --user start "$svc" && echo "started: $svc" || echo "start failed: $svc"; else echo "not installed: $svc"; fi; done'

ansible automationcontroller -i inventory-growth -m ansible.builtin.shell -a 'for svc in automation-controller-rsyslog.service automation-controller-task.service automation-controller-web.service metrics-utility-gather.service metrics-utility-build-report.service metrics-utility-gather.timer metrics-utility-build-report.timer receptor.service pcp.service; do load=$(systemctl --user show -p LoadState --value "$svc" 2>/dev/null || echo not-found); if [ "$load" = loaded ]; then systemctl --user start "$svc" && echo "started: $svc" || echo "start failed: $svc"; else echo "not installed: $svc"; fi; done'

ansible automationhub -i inventory-growth -m ansible.builtin.shell -a 'for svc in automation-hub-api.service automation-hub-content.service automation-hub-web.service automation-hub-worker-1.service automation-hub-worker-2.service pcp.service; do load=$(systemctl --user show -p LoadState --value "$svc" 2>/dev/null || echo not-found); if [ "$load" = loaded ]; then systemctl --user start "$svc" && echo "started: $svc" || echo "start failed: $svc"; else echo "not installed: $svc"; fi; done'

ansible automationeda -i inventory-growth -m ansible.builtin.shell -a 'for svc in automation-eda-api.service automation-eda-web.service automation-eda-daphne.service automation-eda-worker-1.service automation-eda-worker-2.service automation-eda-activation-worker-1.service automation-eda-activation-worker-2.service automation-eda-scheduler.service pcp.service; do load=$(systemctl --user show -p LoadState --value "$svc" 2>/dev/null || echo not-found); if [ "$load" = loaded ]; then systemctl --user start "$svc" && echo "started: $svc" || echo "start failed: $svc"; else echo "not installed: $svc"; fi; done'

ansible redis -i inventory-growth -m ansible.builtin.shell -a 'for svc in redis-tcp.service redis-unix.service; do load=$(systemctl --user show -p LoadState --value "$svc" 2>/dev/null || echo not-found); if [ "$load" = loaded ]; then systemctl --user start "$svc" && echo "started: $svc" || echo "start failed: $svc"; else echo "not installed: $svc"; fi; done'

Example:

$ ansible automationgateway -i inventory-growth -m ansible.builtin.shell -a 'for svc in automation-gateway.service automation-gateway-proxy.service redis-tcp.service redis-unix.service pcp.service; do load=$(systemctl --user show -p LoadState --value "$svc" 2>/dev/null || echo not-found); if [ "$load" = loaded ]; then systemctl --user start "$svc" && echo "started: $svc" || echo "start failed: $svc"; else echo "not installed: $svc"; fi; done'
my.containerized.gateway | CHANGED | rc=0 >>
started: automation-gateway.service
started: automation-gateway-proxy.service
started: redis-tcp.service
started: redis-unix.service
not installed: pcp.service

$ ansible automationcontroller -i inventory-growth -m ansible.builtin.shell -a 'for svc in automation-controller-rsyslog.service automation-controller-task.service automation-controller-web.service metrics-utility-gather.service metrics-utility-build-report.service metrics-utility-gather.timer metrics-utility-build-report.timer receptor.service pcp.service; do load=$(systemctl --user show -p LoadState --value "$svc" 2>/dev/null || echo not-found); if [ "$load" = loaded ]; then systemctl --user start "$svc" && echo "started: $svc" || echo "start failed: $svc"; else echo "not installed: $svc"; fi; done'
my.containerized.controller | CHANGED | rc=0 >>
started: automation-controller-rsyslog.service
started: automation-controller-task.service
started: automation-controller-web.service
not installed: metrics-utility-gather.service
not installed: metrics-utility-build-report.service
not installed: metrics-utility-gather.timer
not installed: metrics-utility-build-report.timer
started: receptor.service
not installed: pcp.service

$ ansible automationhub -i inventory-growth -m ansible.builtin.shell -a 'for svc in automation-hub-api.service automation-hub-content.service automation-hub-web.service automation-hub-worker-1.service automation-hub-worker-2.service pcp.service; do load=$(systemctl --user show -p LoadState --value "$svc" 2>/dev/null || echo not-found); if [ "$load" = loaded ]; then systemctl --user start "$svc" && echo "started: $svc" || echo "start failed: $svc"; else echo "not installed: $svc"; fi; done'
my.containerized.hub | CHANGED | rc=0 >>
started: automation-hub-api.service
started: automation-hub-content.service
started: automation-hub-web.service
started: automation-hub-worker-1.service
started: automation-hub-worker-2.service
not installed: pcp.service

$ ansible automationeda -i inventory-growth -m ansible.builtin.shell -a 'for svc in automation-eda-api.service automation-eda-web.service automation-eda-daphne.service automation-eda-worker-1.service automation-eda-worker-2.service automation-eda-activation-worker-1.service automation-eda-activation-worker-2.service automation-eda-scheduler.service pcp.service; do load=$(systemctl --user show -p LoadState --value "$svc" 2>/dev/null || echo not-found); if [ "$load" = loaded ]; then systemctl --user start "$svc" && echo "started: $svc" || echo "start failed: $svc"; else echo "not installed: $svc"; fi; done'
my.containerized.eda | CHANGED | rc=0 >>
started: automation-eda-api.service
started: automation-eda-web.service
started: automation-eda-daphne.service
started: automation-eda-worker-1.service
started: automation-eda-worker-2.service
started: automation-eda-activation-worker-1.service
started: automation-eda-activation-worker-2.service
started: automation-eda-scheduler.service
not installed: pcp.service

$ ansible redis -i inventory-growth -m ansible.builtin.shell -a 'for svc in redis-tcp.service redis-unix.service; do load=$(systemctl --user show -p LoadState --value "$svc" 2>/dev/null || echo not-found); if [ "$load" = loaded ]; then systemctl --user start "$svc" && echo "started: $svc" || echo "start failed: $svc"; else echo "not installed: $svc"; fi; done'
[WARNING]: Could not match supplied host pattern, ignoring: redis
[WARNING]: No hosts matched, nothing to do

5. Reconcile the Containerized environment post-import

5.1 Deprovision the platform gateway configuration

Run the following commands on the Containerized Gateway node as the ansible_user:

$ podman exec -it automation-gateway bash
bash-5.1$ aap-gateway-manage migrate
bash-5.1$ aap-gateway-manage shell_plus
>> HTTPPort.objects.all().delete(); ServiceNode.objects.all().delete(); ServiceCluster.objects.all().delete()
>> exit()
bash-5.1$ exit

5.2 Transfer the hub content from the RPM environment to the containerized environment

IMPORTANT: Be sure to ssh into the Containerized Hub node first. If you have multiple hub nodes, this only has to be done on one of them if using NFS for storage.

Run the following command on the Containerized Hub node as the ansible_user:

rsync -avR $SSH_USER@$RPM_HUB_NODE:/var/lib/pulp/ ~/.local/share/containers/storage/volumes/hub_data/_data/ --rsync-path="sudo rsync"

NOTES:

The above command will copy the contents of RPM /var/lib/pulp to the local storage on the containerized hub node
The $SSH_USER must be able to elevate to root via sudo

Example:

$ rsync -avR my-ssh-user@my.rpm.hub:/var/lib/pulp/ ~/.local/share/containers/storage/volumes/hub_data/_data/ --rsync-path="sudo rsync"
receiving incremental file list
var/lib/pulp/.bash_logout
var/lib/pulp/.bash_profile
var/lib/pulp/.bashrc
(.....)
var/lib/pulp/scripts/
var/lib/pulp/tmp/
var/lib/pulp/tmp/118479@my.containerized.node/
var/lib/pulp/tmp/237988@my.containerized.node//

sent 8,341 bytes  received 1,277,149,507 bytes  364,902,242.29 bytes/sec
total size is 1,276,792,546  speedup is 1.00

5.3 Update your containerized installer inventory file to include the values from secrets.yml

NOTES:

the /tmp directory where the artifact is stored is often cleaned up, so it's necessary to add the values from the secrets.yml to the inventory file
this is a manual process
you do not need to include the "secret_keys" but you can if you want to

5.4 Update your containerized installer inventory file to include the values from custom_configs

Add the following inventory variable in the [all:vars] section of the inventory file

controller_extra_settings=

Populate it with the settings in your custom_configs

Example:

controller_extra_settings=[{"setting": "USE_X_FORWARDED_HOST", "value": True}, {"setting": "LOG_AGGREGATOR_USERNAME", "value": "Spartacus"}]

5.5 Run the installer to update the configurations

Run the following command on the Containerized Gateway node as the ansible_user from within the installer directory:

ansible-playbook -i inventory-growth ansible.containerized_installer.install

NOTE: The above command may need the name of the inventory file changed.

5.6 Adjust configurations as needed

If using an SSO method, you will likely need to update it
- For example, if you have SAML authentication enabled, you will need to edit it in AAP and delete the ACS URL value, then Save the authentication method to have the system automatically regenerate it.
- You will also need to provide the updated ACS URL to your IdP so they can update it from their end
The Base URL of the service will likely be incorrect. You can update this by navigating to Settings > Automation Execution > System Base URL of the service.

6. Validate the Containerized environment

6.1 Verify all migrated components function correctly

Platform gateway: Access the Ansible Automation Platform URL at https://<gateway_hostname>/ and verify that the dashboard loads correctly. Check that the platform gateway service is running and connected to automation controller.
Automation controller: Under Automation Execution, check that projects, inventories, and job templates are present and configured.
Automation hub: Under Automation Content, verify that collections, namespaces, and their contents are visible.
Event-Driven Ansible (if applicable): Under Automation Execution Decisions, verify that rule audits, rulebook activations, and projects are accessible.
For each component, check the logs to ensure there are no startup errors or warnings:

Run the following command on each component node in the Containerized environment as the ansible_user:

podman logs <container_name>

6.2 Test workflows and automation processes

Run job templates: Run several key job templates, including those with dependencies on various credential types.
Test workflow templates: Run workflow templates to ensure that workflow nodes run in the correct order and that the workflow completes successfully.
Verify execution environments: Ensure that jobs run in the appropriate execution environments and can access required dependencies.
Check job artifacts: Verify that job artifacts are properly stored and accessible.
Validate job scheduling: Test scheduled jobs to ensure they run at the expected times.

6.3 Validate user access and permissions

User authentication: Test login functionality with various user accounts to ensure authentication works correctly.
Role-based access controls: Verify that users have appropriate permissions for organizations, projects, inventories, and job templates.
Team memberships: Confirm that team memberships and team-based permissions are intact.
API access: Test API tokens and ensure that API access is functioning properly.
SSO integration (if applicable): Verify that Single Sign-On authentication is working correctly.

6.4 Confirm content synchronization and availability

Collection synchronization: Check that you can synchronize collections from a remote.
Collection Upload: Check that you can upload collections.
Collection repositories: Verify that automation hub makes collections available and that execution environments can use them.
Project synchronization: Check that projects can sync content from source control repositories.
External content sources: Test synchronization from automation hub and Ansible Galaxy (if configured).
Execution environment availability: Confirm that all required execution environments exist and that execution nodes can access them.
Content dependencies: Verify that the system correctly resolves content dependencies when running jobs.

SBR

Ansible

Product(s)

Red Hat Ansible Automation Platform

Components

platform

Category

Install

Tags

migration

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

How do I migrate AAP 2.6 from an RPM to Containerized installation?

Environment

Issue

Resolution

Table of contents

1. Prerequisites and Requirements

2. Prepare and assess the RPM environment

2.1 Connect to the PostgreSQL database and switch to the superuser

2.2 Verify PostgreSQL version

2.3 Verify PostgreSQL size

2.4 Adjust the available storage accordingly

3. Create and verify the migration artifact

3.1 Create the artifact directory

3.2 Populate the secrets.yml

3.2.1 Get the Gateway database connection and SECRET_KEY values

a. Run the following commands on the RPM Gateway node as the root user:

b. Copy the outputs and paste them into the secrets.yml

3.2.2 Get the Controller database connection and SECRET_KEY values

a. Run the following commands on the RPM Controller node as the root user:

b. Copy the outputs and paste them into the secrets.yml

3.2.3 Get the Hub database connection and SECRET_KEY values

a. Run the following commands on the RPM Hub node as the root user:

b. Copy the outputs and paste them into the secrets.yml

3.2.4 Get the EDA database connection and SECRET_KEY values

a. Run the following commands on the RPM EDA node as the root user:

b. Copy the outputs and paste them into the secrets.yml

3.2.5 Verify the secrets.yml

3.3 Copy custom configs

3.4 Create the pgc files (perform the database backups)

3.4.1 Back up the Gateway database

3.4.2 Back up the Controller database

3.4.3 Back up the Hub database

3.4.4 Back up the EDA database

3.4.5 Verify the backups exist

3.5 Create the sha256sum.txt and verify it was created

3.5.1 Create the sha256sum.txt file

3.5.2 Verify the file was created

3.6 Create the artifact.tar and verify it was created

3.6.1 Create the artifact.tar file

3.6.2 Verify the file was created

3.7 Create and validate the artifact.tar.sha256

3.7.1 Create a sha256 file for artifact.tar

3.7.2 Validate the sha256 file

3.7.3 Validate the contents of artifact.tar

4. Import the artifact to the Containerized environment

4.1 Move artifact to the Containerized environment

4.1.1 SSH into the main Containerized Gateway node as the ansible_user

4.1.2 Copy the artifact.tar and artifact.tar.sha256 from the RPM environment to the Containerized environment as the ansible_user

4.1.3 Change permissions of artifact.tar and artifact.tar.sha256

4.1.4 Verify permissions were set correctly

4.2 Ensure transfers were completed successfully by checking the sha256sum

4.2.1 Validate the artifact.tar.sha256 as the ansible_user

4.2.2 Extract the artifact.tar

4.2.3 Validate the sha256sum.txt

4.3 Download and extract the latest Ansible Automation Platform Containerized installer

4.4 Update the secrets.yml

4.4.1 Ensure ansible-core is installed

4.4.2 View the current database

4.4.3 Update the pg_host settings in the secrets.yml

4.4.4 Verify the change was made

4.5 Update the inventory file

4.6 Run the containerized installer with secrets.yml

4.7 Verify the installation was successful

4.7.1 Check for errors in the aap_install.log

4.7.2 Verify the SECRET_KEYs and hub_db_fields_encryption_key exist and are correct for each component

4.7.3 Log into the AAP UI as the admin user

4.8 Restore the databases

4.8.1 Stop container services on the Containerized nodes

4.8.2 Change permission of the artifact directory so it is accessible within the container

4.8.3a UNMANAGED DATABASE - Restore the database backups for customer-provided databases

a. Provide the Create DB role for the component database users

b. Perform a pg_restore using the pgc files

c. Revoke the Create DB role for the component database users

4.8.3b MANAGED DATABASE - Restore the database backups for managed databases

a. Update secrets.yml with the postgres user's password

b. Verify the secrets.yml was updated

c. Create a temporary postgres container to import the database backups

d. Provide the Create DB role to the component database users

e. Verify database users have the Create DB role

f. Restore the Gateway database backup