Scale MachineSet using static IP in OpenShift 4 on vSphere
Environment
- Red Hat OpenShift Container Platform (RHOCP) 4.16+
- VMware vSphere with static IPs
Issue
- How to scale a
machinesetusing static IP in RHOCP 4 on vSphere?
Resolution
Scaling machines via MachineSet using static IP in OpenShift on vSphere is a Technology Preview feature starting with OpenShift 4.14 as per the static IP addresses for vSphere nodes documentation, and fully supported in OpenShift 4.16 and newer releases as documented in machine scaling with static IP addresses.
On new installations the installer creates its own IPPool that can be used later on to manually scale machineSets using static IPs, in case there is no external IPAM controller to handle this as mentioned in the documentation above. There will be also a machineSet scaled to 0 where this information can be checked. This machineSet can be used to scale new machines or used as a template for newer ones, as the example below:
$ oc get machineset ocp416-static-vmw-8vz5k-worker-0 -o yaml -n openshift-machine-api
apiVersion: machine.openshift.io/v1beta1
kind: MachineSet
metadata:
annotations:
machine.openshift.io/memoryMb: "24536"
machine.openshift.io/vCPU: "8"
creationTimestamp: "2025-05-01T09:52:30Z"
[...]
spec:
[...]
network:
devices:
- addressesFromPools: ---> the installer provided IPPool
- group: installer.openshift.io
name: default-0
resource: IPPool
nameservers:
- 192.168.1.1
networkName: VLAN Network
numCPUs: 8
numCoresPerSocket: 8
When scaling this machineSet new machines will be created and with them the respective IPAddressClaims:
$ oc get machines -n openshift-machine-api
ocp416-static-vmw-8vz5k-worker-0-fqwnx Running 35m
ocp416-static-vmw-8vz5k-worker-0-tnqrr Running 35m
$ oc get ipaddressclaims -n openshift-machine-api
ocp416-static-vmw-8vz5k-worker-0-fqwnx-claim-0-0 default-0 IPPool 35m
ocp416-static-vmw-8vz5k-worker-0-tnqrr-claim-0-0 default-0 IPPool 35m
However, at this point the nodes will not be created and the respective IPAddresses need to be created according to what is described in the documentation provided in this article. Also for this case the last step is needed to patch the IPAddressClaims with the respective IPAddressRef, in order to bind both CRs together. Confirm with:
$ oc get ipaddresses -o wide -n openshift-machine-api
ocp416-static-vmw-8vz5k-worker-0-fqwnx-ipaddress-0-0 192.168.1.22 default-0 IPPool 31m
ocp416-static-vmw-8vz5k-worker-0-tnqrr-ipaddress-0-0 192.168.1.23 default-0 IPPool 30m
After this has been created and patched, the nodes will be created with the respective IPs:
$ oc get nodes
ocp416-static-vmw-8vz5k-worker-0-fqwnx Ready worker 10m v1.29.14+7cf4c05
ocp416-static-vmw-8vz5k-worker-0-tnqrr Ready worker 9m35s v1.29.14+7cf4c05
To implement a more automated procedure an IPAM controller needs to be created. At the moment there is no supported service integrated in OpenShift for this, but the example below can be used for testing.
On the project openshift-machine-api create the following:
$ cat rbac_policy.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: machine-ipam-controller
namespace: openshift-machine-api
annotations:
include.release.openshift.io/self-managed-high-availability: "true"
include.release.openshift.io/single-node-developer: "true"
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
annotations:
include.release.openshift.io/self-managed-high-availability: "true"
include.release.openshift.io/single-node-developer: "true"
name: machine-ipam-controller
rules:
- apiGroups:
- ipamcontroller.openshift.io
resources:
- ippools
verbs:
- get
- list
- patch
- update
- watch
- apiGroups:
- ipam.cluster.x-k8s.io
resources:
- ipaddressclaims
- ipaddressclaims/status
verbs:
- get
- list
- patch
- update
- watch
- apiGroups:
- ipam.cluster.x-k8s.io
resources:
- ipaddresses
verbs:
- create
- delete
- list
- patch
- watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: machine-ipam-controller
annotations:
include.release.openshift.io/self-managed-high-availability: "true"
include.release.openshift.io/single-node-developer: "true"
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: machine-ipam-controller
subjects:
- kind: ServiceAccount
name: machine-ipam-controller
namespace: openshift-machine-api
$ oc create -f rbac_policy.yaml
$ oc apply -f https://raw.githubusercontent.com/rvanderp3/machine-ipam-controller/refs/heads/main/install/ipamcontroller.openshift.io_ippools.yaml
$ oc apply -f https://raw.githubusercontent.com/rvanderp3/machine-ipam-controller/refs/heads/main/manifests/ipam-deployment.yaml
The pod will not be created which is normal, since the deployment needs to be scaled first:
$ oc scale --replicas=1 deployment.apps/machine-ipam-controllers
When the pod starts you can now create the IPPool by making sure the address-cidr is as per the cluster's static IP:
$ cat ippool.yaml
apiVersion: ipamcontroller.openshift.io/v1
kind: IPPool
metadata:
name: <ippool_name>
spec:
address-cidr: 192.168.1.16/29 --> this one is important. For example if your nodes use network /24, then here you need to create a subnet inside this one and you cannot use /24. For example use ipcalc tool on linux and do ipcalc xxx.xxx.xxx.xxx/24 -S 26 and it will generate some subnets based /26 that can be used here.
prefix: <main_network_prefix>
gateway: <default_gateway>
nameserver:
- <DNS_Server_IPs>
$ oc create -f ippool.yaml
With this created, new machineSets can be deployed referencing these IPPool, like for example:
spec:
lifecycleHooks: {}
metadata:
labels:
node-role.kubernetes.io/infra: ""
providerSpec:
value:
apiVersion: machine.openshift.io/v1beta1
credentialsSecret:
name: vsphere-cloud-credentials
diskGiB: 40
kind: VSphereMachineProviderSpec
memoryMiB: 16536
metadata:
creationTimestamp: null
network:
devices:
- addressesFromPools:
- group: ipamcontroller.openshift.io
name: <ippool_name>
resource: IPPool
nameservers:
- <DNS-servers>
When this machineSet is scaled up all the necessary objects and binds will be done automatically and the nodes will be created, without the need for any additional manual intervention.
WARNING: Important to understand that this should be used for testing purposes only and it is not supported by Red Hat.
Root Cause
Scaling machines via MachineSet using static IP in OpenShift on vSphere is GA in OpenShift 4.16 and newer releases.
A RFE was created for the possibility to have Red Hat supported IPAM controller. To follow check this This content is not included.Jira ticket.
This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.