Ephemeral Containers and kubectl debug

Ephemeral containers let you attach a temporary debugging container to a running pod without restarting it. This is especially powerful for debugging minimal or distroless images that lack shells and diagnostic tools.

Why Ephemeral Containers

Many production images are stripped down for security and size. They contain no shell, no curl, no ps. When something goes wrong, you cannot kubectl exec into them because there is nothing to exec. Ephemeral containers solve this by injecting a fully equipped debug image into the pod's namespace.

Debugging a Running Pod

Add a debug container to an existing pod using kubectl debug:

# Attach an ephemeral container with a busybox image
kubectl debug -it my-pod --image=busybox --target=my-container

# Use a full debug toolkit
kubectl debug -it my-pod --image=nicolaka/netshoot --target=app

The --target flag shares the process namespace with the specified container, letting you see its running processes.

Debugging Distroless Images

Distroless images have no shell at all. Ephemeral containers are the primary debugging strategy:

# Debug a distroless application container
kubectl debug -it my-pod --image=busybox:1.36 -- sh

# Inspect the filesystem of the target container
kubectl debug -it my-pod --image=alpine --target=app -- ls /proc/1/root/app

Creating a Copy for Debugging

You can also create a copy of a pod with a different image or added containers:

# Copy the pod and replace its command
kubectl debug my-pod -it --copy-to=debug-pod --image=ubuntu -- bash

# Copy with shared process namespace enabled
kubectl debug my-pod -it --copy-to=debug-pod --share-processes

The original pod continues running undisturbed while you investigate the copy.

Node-Level Debugging

kubectl debug node/my-node -it --image=ubuntu

This creates a pod on the target node with host filesystem access at /host, useful for inspecting node-level issues.