How to use GKLM KMS with ODF - (Dev Preview - ODF 4.22)
Environment
OpenShift Data Foundation 4.22 and above
Issue
A user wants to use configure GKLM KMS with ODF for PVC and ClusterWide encryption.
Resolution
Important: A developer preview feature is subject to Developer preview support limitations. Developer preview features are not intended to be run in production environments. The clusters deployed with the developer preview features are considered to be development clusters and are not supported through the Red Hat Customer Portal case management system. Development Preview features are meant for customers who are willing to evaluate new products or releases of products in an early stage of product development. If you need assistance with developer preview features, reach out to the ocs-devpreview@redhat.com mailing list and a member of the Red Hat Development Team will assist you as quickly as possible based on availability and work schedules. To know more about the support scope refer to the This content is not included.KCS
Note: This guide does not cover the cases where you have encryption already configured using some other KMS. It will lead to data loss.
Prerequisites
Certificates
We would need both a key serving certificate and a set of client certificates to communicate over KMIP.
Client Certificates
For clients to communicate to a KMIP server, we would need a set of certificate files which we can create by running the commands below.
❯ openssl req -new -newkey rsa:2048 -days 365 -nodes -x509 \
-keyout odf.key -out odf.crt \
-subj "/CN=odf" \
-addext "subjectAltName=DNS:odf"
We will need both the files (odf.key and odf.crt) in the steps that follow.
Server Certificate
There must be a "key-serving" certificate set in the GKLM admin console. Click on Configuration > System Certificates and see if you can find a certificate with role as "Key serving".
If there is no key serving certificate, you can create one by following these steps:
- Go to Configuration > System Certificates
- Click on "Create or import certificates"
- Choose "Create self signed certificate"
- Fill in desired label and description
- Check the box for "Key serving" under "Use this server certificate for"
- Expand the optional parameters by clicking on the chevron
- Fill in the following under DNS names:
- IP address of the GKLM service, e.g 10.xx.xx.xx
- Kubernetes FQDN for the service, e.g. service_name.namespace.svc.cluster.local
Once you have created the certificate, you need to download it next. Go to Configuration > System Certificates click on the 3 dot menu and click Download. Select "base64" as the format and click on download. You will get a .cer file, say gklm-server.cer. We'll need this file later.
Create a new KMIP client in GKLM
Create KMIP client
With both the server and client certificates with us. We now need to create a KMIP client which will be communicating with the GKLM server. Follow these steps:
- Go to "Endpoint Management > Catalog"
- Search for "Custom KMIP Client" and click on "Add"
- Enter "odf" or any name of your choosing, leave the usage as Generic and click "Add".
Configure client certificates for the created client
Post creation you will be presented with the management interface for the client/endpoint. On the left hand panel look for "Communication certificates" and click on the 3 dot menu to select "Modify".
Follow these steps:
- Select "Import client certificate" radio button
- Enter "odf" or any name of your choosing as the name
- In upload certificate select "File" and upload "odf.crt" that we created for the client.
- Click on Modify to apply the changes.
Create a key pair to use for volume encryption
Ceph-CSI requires us to specify a UUID of a key that will be used to encrypt the DEKs for our PVs. On the same endpoint management screen, follow these steps:
- Click on "Create symmetric key"
- Enter 1 as the value for "Number of objects to create"
- Leave algorithm as "AES"
- Set the key size to "256"
- Enter any prefix of your choosing, e.g. "pvc"
- Change cryptographic usage mask to "Encrypt Decrypt"
- Click on Create
In the panel on the right, you will the key we just created. It can be identified by the prefix e.g. "pvcxxxxxxx" of type "SYMMETRIC_KEY". We need to copy its UUID. Click on 3 dot menu to the right of the certificate name and select "View". Copy the UUID, it will be something like: KEY-xxxx-...-xxxx
Configure Ceph-CSI to use GKLM for PV Encryption
We need to create a Secret that contains our server and client certificates along with updating/creating the CSI KMS ConfigMap. Post that we will be creating a StorageClass that will provision PVCs for us using this configuration.
Create the secret
Note: This secret needs to be present in the Namespace where the PVCs will be provisioned. This guide uses openshift-storage as an example.
To execute this step
❯ oc -n openshift-storage create secret generic gklm-credentials \
--from-file=CA_CERT=./gklm-server.cer \
--from-file=CLIENT_CERT=./odf.crt \
--from-file=CLIENT_KEY=./odf.key
We'll need to set the UUID from the above steps to the secret as well:
❯ oc -n openshift-storage set data secret/gklm-credentials UNIQUE_IDENTIFIER="<PASTE_UUID_FROM_ABOVE_HERE>"
Update/Create the CSI KMS ConfigMap
We will need to configure the following values in the ConfigMap:
| Value | Description |
|---|---|
| KMS_PROVIDER | Must be set to "kmip" to use the KMIP protocol. |
| KMIP_ENDPOINT | The GKLM server endpoint address e.g. "sklmapp.sklm.svc.cluster.local:5696". |
| KMS_SERVICE_NAME | A unique name for the key management service within the project e.g. "gklm" |
| KMIP_SECRET_NAME | Name of the Kubernetes Secret containing the credentials for communicating with the GKLM server. |
| USE_CRYPTO_RPC | Indicates whether cryptographic operations should be handled by the KMS server. The KMS provider must support the Encrypt and Decrypt RPC methods defined in the KMIP specification. For GKLM, this setting MUST be false. |
| TLS_SERVER_NAME | The endpoint server name. Useful when the GKLM endpoint does not have a DNS entry. SAN on the key serving certificate must match this. |
If you have existing config in the CSI KMS ConfigMap
Create a new entry/key under data array with the following values:
apiVersion: v1
kind: ConfigMap
metadata:
name: csi-kms-connection-details-test
namespace: openshift-storage
data:
kms-1: '{....}'
kms-2: '{....}'
gklm-kmip: '{
"KMS_PROVIDER": "kmip",
"KMS_SERVICE_NAME": "gklm",
"KMIP_ENDPOINT": "sklmapp.sklm.svc.cluster.local:5696",
"KMIP_SECRET_NAME": "gklm-credentials",
"USE_CRYPTO_RPC": "false",
"TLS_SERVER_NAME": "sklmapp.sklm.svc.cluster.local"
}'
If you do not have existing config in the CSI KMS ConfigMap
You can go ahead and apply the following manifest to create the required ConfigMap:
apiVersion: v1
kind: ConfigMap
metadata:
name: csi-kms-connection-details
namespace: openshift-storage
data:
gklm-kmip: '{
"KMS_PROVIDER": "kmip",
"KMS_SERVICE_NAME": "gklm",
"KMIP_ENDPOINT": "sklmapp.sklm.svc.cluster.local:5696",
"KMIP_SECRET_NAME": "gklm-credentials",
"USE_CRYPTO_RPC": "false",
"TLS_SERVER_NAME": "sklmapp.sklm.svc.cluster.local"
}'
Once the Secret and ConfigMap is created we can proceed to create the StorageClass. We need to specify encryptionKMSID which will be gklm-kmip in our case (the name of they key for GKLM config in csi-kms-connection-details the we created above).
Create the StorageClass
Apply the following manifest to create a StorageClass that is configured to use GKLM KMS for PVC encryption:
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: gklm-sc
parameters:
clusterID: openshift-storage
csi.storage.k8s.io/controller-expand-secret-name: rook-csi-rbd-provisioner
csi.storage.k8s.io/controller-expand-secret-namespace: openshift-storage
csi.storage.k8s.io/fstype: ext4
csi.storage.k8s.io/node-stage-secret-name: rook-csi-rbd-node
csi.storage.k8s.io/node-stage-secret-namespace: openshift-storage
csi.storage.k8s.io/provisioner-secret-name: rook-csi-rbd-provisioner
csi.storage.k8s.io/provisioner-secret-namespace: openshift-storage
encrypted: "true"
encryptionKMSID: gklm-kmip
imageFeatures: layering,deep-flatten,exclusive-lock,object-map,fast-diff
imageFormat: "2"
pool: ocs-storagecluster-cephblockpool
provisioner: openshift-storage.rbd.csi.ceph.com
reclaimPolicy: Delete
volumeBindingMode: Immediate
All the values are pretty standard except the encryptionKMSID which is gklm-kmip as configured in the CSI KMS ConfigMap. You can now go ahead and create a PVC from this StorageClass.
Configure the StorageCluster for GKLM KMS
With the PVC encryption done, now we need to update StorageCluster KMS configuration so that OSDs and Noobaa can also consume it.
Create the Secret
We need to create a secret that contains the GKLM server certificate along with our client key and certificate by running:
❯ oc -n openshift-storage create secret generic gklm-ocs-credentials \
--from-file=CA_CERT=./gklm-server.cer \
--from-file=CLIENT_CERT=./odf.crt \
--from-file=CLIENT_KEY=./odf.key
Apply the KMS ConfigMap
Now we need to apply the ConfigMap which instructs OCS operator and subsequently Rook and Noobaa to use the GKLM server:
apiVersion: v1
kind: ConfigMap
metadata:
name: ocs-kms-connection-details
namespace: openshift-storage
data:
KMIP_ENDPOINT: sklmapp.sklm.svc.cluster.local:5696 # GKLM server endpoint
KMIP_SECRET_NAME: gklm-ocs-credentials # The secret containing the certificates, name must match the secret we created above
KMS_PROVIDER: kmip # Must be KMIP to use GKLM
KMS_SERVICE_NAME: gklm-kmip # Any name of your choosing
TLS_SERVER_NAME: sklmapp.sklm.svc.cluster.local # GKLM server endpoint sans port
Edit the StorageCluster to enable encryption and use KMS
❯ oc edit storagecluster/ocs-storagecluster -n openshift-storage
In your editor make sure spec.encryption is as follows (leave existing values as is, just update the ones listed below):
apiVersion: ocs.openshift.io/v1
kind: StorageCluster
metadata:
name: ocs-storagecluster
namespace: openshift-storage
spec:
...
encryption:
clusterWide: true
kms:
enable: true
Conclusion
With the above steps done, you should now be able to use GKLM KMS with OpenShift Data Foundation.
This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.