SSE-S3 with Vault Agent for RGW (Dev Preview - ODF 4.22)

Updated 9 Jul 2026

Important: A developer preview feature is subject to Developer preview support limitations. Developer preview features are not intended to be run in production environments. The clusters deployed with the developer preview features are considered to be development clusters and are not supported through the Red Hat Customer Portal case management system. Development Preview features are meant for customers who are willing to evaluate new products or releases of products in an early stage of product development. If you need assistance with developer preview features, reach out to the ocs-devpreview@redhat.com mailing list and a member of the Red Hat Development Team will assist you as quickly as possible based on availability and work schedules. To know more about the support scope refer to the This content is not included.KCS

What is SSE-S3?

Server-Side Encryption S3 (SSE-S3) is a form of encryption where all objects stored in an S3-compatible object store (RGW) are encrypted by default using keys managed by a KMS.
Unlike SSE-KMS, where the client must specify a KMS key ID per request, SSE-S3 encrypts every object transparently without any client-side configuration.

What is a Vault Agent?

Vault Agent is a daemon that runs alongside applications and handles authentication to Vault automatically.
Instead of requiring a long-lived Vault token (as the SSE-KMS token-based method does), Vault Agent authenticates using the Kubernetes service account and maintains a cached, automatically renewed token.
ODF deploys a standalone Vault Agent Deployment (2 replicas) that exposes a local HTTP caching proxy on port 8100. RGW connects to this proxy instead of directly to the Vault server.

Prerequisites

ODF cluster with encryption enabled (spec.encryption.enable: true or spec.encryption.clusterWide: true on the StorageCluster CR, or at least one encrypted device set)
KMS enabled on the StorageCluster (spec.encryption.kms.enable: true)
A running HashiCorp Vault instance with:
- The Transit secrets engine enabled
- Kubernetes auth method configured and the OpenShift cluster registered as a trusted auth source.
- A Vault policy and role granting the vault-agent-rgw service account access to the transit engine.

Setup Steps

Configure Vault Server:

Enable the Transit secrets engine (not KV because SSE-S3 uses Transit for envelope encryption):
```
$ vault auth enable kubernetes
```

Enable and configure Kubernetes auth (if not already done for clusterwide/PV encryption):

 $ vault auth enable kubernetes

 $ SA_CA_CRT=$(oc -n openshift-storage get secret odf-vault-auth-token -o jsonpath="{.data['ca\.crt']}" | base64 --decode; echo)
  
 $ OCP_HOST=$(oc config view --minify --flatten -o jsonpath="{.clusters[0].cluster.server}")

 $ vault write auth/kubernetes/config \
     kubernetes_host="$OCP_HOST" \
     kubernetes_ca_cert="$SA_CA_CRT"

Create a policy for RGW transit operations:

$ echo '
  path "transit/keys/*" {
    capabilities = ["create", "read", "update", "delete"]
  }
  path "transit/encrypt/*" {
    capabilities = ["create", "read", "update"]
  }
  path "transit/decrypt/*" {
    capabilities = ["create", "read", "update"]
  }' | vault policy write rgw-sse-s3 -

Create a Kubernetes auth role for the Vault Agent service account:

$ vault write auth/kubernetes/role/rook-ceph-rgw \
      bound_service_account_names=vault-agent-rgw \
      bound_service_account_namespaces=openshift-storage \
      policies=rgw-sse-s3 \
      ttl=1h

NOTE: The default role name is rook-ceph-rgw. If you use a different role name, set VAULT_RGW_ROLE in the KMS ConfigMap (Step 2).

Create or Update the KMS ConfigMap:

Create (or update) the ocs-kms-connection-details ConfigMap in the openshift-storage namespace:

apiVersion: v1
  kind: ConfigMap
  metadata:
    name: ocs-kms-connection-details
    namespace: openshift-storage
  data:
    KMS_PROVIDER: vault
    VAULT_ADDR: https://vault.example.com:8200
    VAULT_RGW_AUTH_METHOD: agent
    #Optional overrides (defaults shown):
    #VAULT_RGW_ROLE: rook-ceph-rgw
    #VAULT_RGW_AUTH_MOUNT_PATH: auth/kubernetes

NOTE: The existing keys for clusterwide and PV encryption (VAULT_BACKEND_PATH, VAULT_BACKEND, etc.) remain in this ConfigMap and are unaffected. SSE-S3 configuration uses separate keys (VAULT_RGW_*).

Configure TLS (if Vault uses TLS):

If your Vault server uses TLS, create Kubernetes Secrets containing the certificates and reference them in the KMS ConfigMap:

   #Create secret for CA certificate
   $ oc create secret generic vault-ca-cert \
       --from-file=cert=/path/to/ca.pem \
       -n openshift-storage

   #Create secret for client certificate (if using mTLS)
   $ oc create secret generic vault-client-cert \
       --from-file=cert=/path/to/client-cert.pem \
       -n openshift-storage

   #Create secret for client key (if using mTLS)
   $ oc create secret generic vault-client-key \
       --from-file=key=/path/to/client-key.pem \
       -n openshift-storage

Then set the corresponding keys in the KMS ConfigMap:

```
  data:
    VAULT_CACERT: vault-ca-cert
    VAULT_CLIENT_CERT: vault-client-cert
    VAULT_CLIENT_KEY: vault-client-key
```

Set the Vault Agent Image (Dev Preview only):
- Since this is a Dev Preview feature, the Vault Agent container image must be set manually on the OCS operator deployment:
```
$ oc set env deployment/ocs-operator \
      VAULT_AGENT_IMAGE=hashicorp/vault:latest \
      -n openshift-storage
```
  NOTE: For GA, this image will be added to the ClusterServiceVersion and set automatically.
Enable Encryption on the StorageCluster:
- Ensure your StorageCluster CR has encryption and KMS enabled:
```
  apiVersion: ocs.openshift.io/v1
  kind: StorageCluster
  metadata:
    name: ocs-storagecluster
    namespace: openshift-storage
  spec:
    encryption:
      enable: true
      kms:
        enable: true
```
- When all conditions are met, ODF automatically creates the following resources in the openshift-storage namespace:
  | Resource Type | Name | Description |
  | --- | --- | --- |
  | ServiceAccount | vault-agent-rgw | Identity for Vault Kubernetes auth |
  | ConfigMap | vault-agent-rgw-config | Generated Vault Agent HCL configuration |
  | Deployment | vault-agent-rgw | 2-replica Vault Agent with cache listener on port 8100 |
  | Service | vault-agent-rgw | ClusterIP service exposing port 8100 |
- The CephObjectStore is configured with SSE-S3 pointing to the Vault Agent service:
```
  security:
    serverSideEncryptionS3:
      connectionDetails:
        KMS_PROVIDER: vault
        VAULT_ADDR: http://vault-agent-rgw.openshift-storage.svc:8100
        VAULT_AUTH_METHOD: agent
        VAULT_SECRET_ENGINE: transit
```

Verification

steps:

Check the Vault Agent pods:

$ oc get pods -l app=vault-agent-rgw -n openshift-storage

Check the Vault Agent service:

$ oc get svc vault-agent-rgw -n openshift-storage

Verify RGW has SSE-S3 configured:

$ oc get cephobjectstore -n openshift-storage -o yaml | grep -A5 serverSideEncryptionS3

Troubleshooting

Vault Agent pods not starting

Check that the VAULT_AGENT_IMAGE environment variable is set on the ocs-operator deployment:

$ oc get deployment ocs-operator -n openshift-storage \
      -o jsonpath='{.spec.template.spec.containers[0].env}' | jq '.[] | select(.name=="VAULT_AGENT_IMAGE")'

Vault Agent fails to authenticate:
- Verify the Vault Kubernetes auth role is configured correctly:
```
$ vault read auth/kubernetes/role/rook-ceph-rgw
```
  Ensure:
  -- bound_service_account_names includes vault-agent-rgw
  -- bound_service_account_namespaces includes openshift-storage

RGW not using encryption:

Check the KMS ConfigMap has VAULT_RGW_AUTH_METHOD set:

$ oc get configmap ocs-kms-connection-details -n openshift-storage -o yaml

Verify encryption is enabled on the StorageCluster:

$ oc get storagecluster -n openshift-storage -o jsonpath='{.items[0].spec.encryption}'

Checking logs:

The rook-ceph-operator will contain logs for RGW encryption configuration:

$ oc logs deployment/rook-ceph-operator -n openshift-storage | grep -i "sse\|vault\|encrypt"

The ocs-operator will contain logs for Vault Agent deployment:

$ oc logs deployment/ocs-operator -n openshift-storage | grep -i "vault-agent\|vault agent"

KMS resources specific to SSE-S3

ocs-kms-connection-details ConfigMap (shared with clusterwide/PV encryption; SSE-S3 uses the VAULT_RGW_* keys)
vault-agent-rgw-config ConfigMap (auto-generated HCL config)
vault-agent-rgw ServiceAccount, Deployment, and Service

SBR

Product(s)

Red Hat OpenShift Data Foundation

Category

Configure

Tags

configuration

Article Type

General