How do I run top and ps commands when in a container without standard Linux tools
Environment
- Red Hat OpenShift Container Platform (OCP) 4
- Middleware containers based on ubi8
- "Distroless" containers
- Red Hat Hardened Images
Issue
-
Container images such as
jboss-webserver-5/jws56-openjdk11-openshift-rhel8andjboss-eap-7/eap74-openjdk11-openshift-rhel8, which are based onubi8, do not haveprocps-ngpackage as follows:$ oc rsh <mw-pod-name> sh-4.4$ top -b -n 1 -H -p <java-pid> sh: top: command not found sh-4.4$ ps sh: ps: command not found -
Accessing a distroless container such as Red Hat Hardened Images using
oc rshfails with the following error message:$ oc rsh apache-6bc857d844-qmxcr 2026-07-08T19:27:02.257947Z: executable file `/bin/sh` not found: No such file or directory command terminated with exit code 1 -
How do I identify high CPU utilization by Java threads on OCP?
Resolution
There are different procedures depending on the used version of OpenShift Container Platform.
OCP 4.10 or later
Content from kubernetes.io is not included.Ephemeral Containers were introduced in OCP 4.10. You can attach debug containers without restarting the target pod, and debug container runs in the target Pods namespace. The following example attaches the toolbox container including ps and top commands:
- Get debug target pod name and container name inside pod:
$ oc get pod
NAME READY STATUS RESTARTS AGE
helloworld-1-42xpj 1/1 Running 0 15m
$ oc get pod helloworld-1-42xpj -o jsonpath="{.spec.containers[*].name}"
helloworld
- Attach Ephemeral Container by using the
kubectl debugcommand:
kubectl debug -it <target-pod-name> --image=<debug-container-image> --target=<target-container-name>
The oc command does not support launching ephemeral containers, so this example uses kubectl instead of oc:
$ kubectl debug -it helloworld-1-42xpj --image=registry.redhat.io/rhel8/support-tools:latest --target=helloworld
Targeting container "helloworld". If you don't see processes from this container it may be
because the container runtime doesn't support this feature.
Defaulting debug container name to debugger-bq4kx.
If you don't see a command prompt, try pressing enter.
[root@helloworld-1-42xpj /]#
psandtopcommands in toolbox container can be executed in the target pod's namespace as follows:
[root@helloworld-1-42xpj /]# ps aux
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
1000710+ 1 0.1 0.0 15436 6540 ? Ss 04:54 0:00 /bin/bash /opt/eap/bin/openshift-launch.sh
1000710+ 316 0.0 0.0 12184 3232 ? S 04:55 0:00 /bin/sh /opt/eap/bin/standalone.sh -c standalone-openshift.xml -bmanagement 0.0.0.0 -Djboss.server.data.dir=/opt/eap/standalon
1000710+ 644 17.2 0.3 2459952 397220 ? Sl 04:55 0:18 /usr/lib/jvm/java-11/bin/java -D[Standalone] -javaagent:/opt/eap/jboss-modules.jar -server -Xlog:gc*:file=/opt/eap/standalone/
root 901 0.0 0.0 19220 3652 pts/0 Ss 04:55 0:00 /usr/bin/bash
root 985 0.0 0.0 54780 3888 pts/0 R+ 04:56 0:00 ps aux
[root@helloworld-1-42xpj /]# top -b -n 1 -H -p 644 | head -n 15
top - 04:57:27 up 4 days, 23:51, 0 users, load average: 1.04, 1.37, 1.71
Threads: 86 total, 0 running, 86 sleeping, 0 stopped, 0 zombie
%Cpu(s): 15.4 us, 6.0 sy, 0.0 ni, 77.5 id, 0.0 wa, 0.0 hi, 1.1 si, 0.0 st
MiB Mem : 128280.3 total, 68429.5 free, 28357.4 used, 31493.4 buff/cache
MiB Swap: 0.0 total, 0.0 free, 0.0 used. 98877.1 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
644 1000710+ 20 0 2459952 414656 30044 S 0.0 0.3 0:00.00 java
645 1000710+ 20 0 2459952 414656 30044 S 0.0 0.3 0:00.82 java
646 1000710+ 20 0 2459952 414656 30044 S 0.0 0.3 0:00.48 ParGC Thread#0
647 1000710+ 20 0 2459952 414656 30044 S 0.0 0.3 0:00.28 VM Thread
648 1000710+ 20 0 2459952 414656 30044 S 0.0 0.3 0:00.00 Reference Handl
649 1000710+ 20 0 2459952 414656 30044 S 0.0 0.3 0:00.00 Finalizer
650 1000710+ 20 0 2459952 414656 30044 S 0.0 0.3 0:00.00 Signal Dispatch
651 1000710+ 20 0 2459952 414656 30044 S 0.0 0.3 0:00.00 Service Thread
// run java thread dump by kill -3 <pid> and top to target java process every 5 seconds
[root@helloworld-1-42xpj /]# for i in {1..60}; do kill -3 644; top -b -n 1 -H -p 644 >> /tmp/top_644.out; sleep 5; done
[root@helloworld-1-42xpj /]#
- Open another console then copy
/tmp/top_644.outfrom ephemeral container to local system:
$ oc cp helloworld-1-42xpj:tmp/top_644.out top_644.out -c debugger-bq4kx
$ ls
top_644.out
- Run
exitcommand on the console where you rankubectl debug. The ephemeral container will automatically terminate whenexitcommand is called:
[root@helloworld-1-42xpj /]# exit
OCP 4.9 or earlier
Run ps and top commands via host OS shell by oc debug node/<node_name>.
-
Check the worker node where the pod is running and target container-id:
$ oc get pod -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES helloworld-1-bqbnc 1/1 Running 0 118m 10.131.0.62 worker-0.nagetsum410.lab.upshift.rdu2.redhat.com <none> <none> $ oc get pod -o yaml helloworld-1-bqbnc -o jsonpath="{.status.containerStatuses[*].containerID}" cri-o://7ee3a487742764681ae3cdbed1ff5b494ad921734b088f468ac7c1bb896e6fbd -
Start a shell on the worker node where the target pod is running by
oc debug node/<node-name>:
$ oc debug node/worker-0.nagetsum410.lab.upshift.rdu2.redhat.com
Starting pod/worker-0nagetsum410labupshiftrdu2redhatcom-debug ...
To use host binaries, run `chroot /host`
Pod IP: 10.0.94.25
If you don't see a command prompt, try pressing enter.
sh-4.4# chroot /host
- Get a list of PIDs running in the container by
runc ps <container-id>. We can see the PID of the target java process is4040570in the host:
sh-4.4# runc ps 7ee3a487742764681ae3cdbed1ff5b494ad921734b088f468ac7c1bb896e6fbd
UID PID PPID C STIME TTY TIME CMD
1000660+ 4039879 4039867 0 08:11 ? 00:00:00 /bin/bash /opt/eap/bin/openshift-launch.sh
1000660+ 4040220 4039879 0 08:11 ? 00:00:00 /bin/sh /opt/eap/bin/standalone.sh -c standalone-openshift.xml ...
1000660+ 4040570 4040220 0 08:11 ? 00:01:08 /usr/lib/jvm/java-11/bin/java -D[Standalone] -javaagent:/opt/eap/jboss-modules.jar -server ...
psfor target java process that is running in the Pod can be executed from the host OS's shell:
// Check VSZ and RSS of java process launched in the Pod
sh-4.4# ps aux | grep 4040570
1000660+ 4040570 0.8 5.5 2452668 450956 ? Sl 08:11 1:11 /usr/lib/jvm/java-11/bin/java -D[Standalone] -javaagent:/opt/eap/jboss-modules.jar -server
...
topand java thread dump for target java process can be executed from the shell outside Pod:
// run java thread dump by kill -3 <pid> and top command every 5 seconds
$ oc debug node/worker-0.nagetsum410.lab.upshift.rdu2.redhat.com -- bash -c 'for i in {1..60}; do kill -3 4040570; top -b -n 1 -H -p 4040570 -w 512; sleep 5; done' >> top_4040570.out
Root Cause
- ubi8 based MW containers do not have
procps-ngpackage to reduce attach surface. - Red Hat Hardened Images do not ship a shell inside the container image
This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.