A kubectl plugin to display the pods and containers which have recently been OOMKilled.
Via krew
kubectl krew install oomd
kubectl oomd
Or manually from the releases page, for example with linux_amd64
# Download the release for your OS
wget https://github.com/jdockerty/kubectl-oomd/releases/download/v0.0.7/oomd_linux_amd64.tar.gz
# Extract files from the archive
tar -xvf oomd_linux_amd64.tar.gz
# Move the binary into your PATH, renaming the application to 'kubectl-oomd' to
# satisfy the plugin convention
mv oomd "${HOME}"/.local/bin/kubectl-oomd
# Run the plugin
kubectl oomdRunning the command will display the pods that have recently been OOMKilled in your current namespace.
This also shows the specific container which was killed too, helpful in the case of multi-container pods.
kubectl oomd
POD CONTAINER REQUEST LIMIT TERMINATION TIME
my-app-5bcbcdf97-722jp infoapp 1G 8G 2022-11-07 13:03:49 0000 GMT
my-app-5bcbcdf97-7j5rd infoapp 1G 8G 2022-11-07 14:35:34 0000 GMT
my-app-5bcbcdf97-k8g8g infoapp 1G 8G 2022-11-07 14:35:02 0000 GMT
my-app-5bcbcdf97-mf65j infoapp 1G 8G 2022-11-07 14:34:57 0000 GMT
You can specify another namespace, as you would with other kubectl commands or use --all-namespaces/-A to check against them all.
kubectl oomd -n oomkilled
POD CONTAINER REQUEST LIMIT TERMINATION TIME
my-app-5bcbcdf97-722jp infoapp 1G 8G 2022-11-07 13:03:49 0000 GMT
my-app-5bcbcdf97-7j5rd infoapp 1G 8G 2022-11-07 14:35:34 0000 GMT
my-app-5bcbcdf97-k8g8g infoapp 1G 8G 2022-11-07 14:35:02 0000 GMT
my-app-5bcbcdf97-mf65j infoapp 1G 8G 2022-11-07 14:34:57 0000 GMT
kubectl oomd --no-headers
my-app-5bcbcdf97-722jp infoapp 1G 8G 2022-11-07 13:03:49 0000 GMT
my-app-5bcbcdf97-7j5rd infoapp 1G 8G 2022-11-07 14:35:34 0000 GMT
my-app-5bcbcdf97-k8g8g infoapp 1G 8G 2022-11-07 14:35:02 0000 GMT
my-app-5bcbcdf97-mf65j infoapp 1G 8G 2022-11-07 14:34:57 0000 GMT
Experimental sorting is enabled through the --sort-field flag. By default, this is none.
At the moment, only time is supported which sorts by termination time of containers, this is mainly
useful in larger outputs across all namespaces (-A), used in conjunction with a pipe to tail.
A simple example of this in action is shown below.
# The default with no sorting.
kubectl oomd -n tracing
POD CONTAINER REQUEST LIMIT TERMINATION TIME
jaeger-agent-4k845 jaeger-agent 100Mi 100Mi 2022-11-11 21:06:31 0000 GMT
jaeger-agent-j5vb8 jaeger-agent 100Mi 100Mi 2022-11-09 23:20:38 0000 GMT
# Most recently OOMKilled pods are shown first
kubectl oomd -n tracing --sort-field time
POD CONTAINER REQUEST LIMIT TERMINATION TIME
jaeger-agent-j5vb8 jaeger-agent 100Mi 100Mi 2022-11-09 23:20:38 0000 GMT
jaeger-agent-4k845 jaeger-agent 100Mi 100Mi 2022-11-11 21:06:31 0000 GMT
If you wish to force some OOMKilled pods for testing purposes, you can use oomer
or simply run
kubectl apply -f https://raw.githubusercontent.com/jdockerty/oomer/main/oomer.yaml
This will create the oomkilled namespace and a Deployment with pods that continually exit with code 137,
in order to be picked up by oomd.