Debug containers with nsenter.

A process can be started so that is confined in a single namespace. For example one can start tcpdump from the host OS inside the container's namespace. This gives us the great ability to bring our own tooling inside a container without having all these tools actually available in the container's image.

This is great for scratch containers!

Behold the awesomeness

Enter all namespaces

Entering all (-a) namespaces and execute ps ax. For this to work, the command ps must be available inside the mnt namespace.

(minikube:default)> sudo nsenter -t 8511 -a ps ax
  PID TTY      STAT   TIME COMMAND
    1 pts/0    Ss+    0:02 /bin/bash
  967 ?        R+     0:00 ps ax
(minikube:default)>

Only in the net namespace

So what if you want to use tools not available in the container on a namespace? Linux got you covered, here we attach to the "network" namespace and run the tcpdump command.

ps ax |grep "<process from container>"
sudo nsenter -t <pid> --net tcpdump -nni eth0

or

# lo and eth0 only live in a docker container.
(minikube:default)> sudo nsenter -t 859 --net ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
24: eth0@if25: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default 
    link/ether 02:42:ac:11:00:03 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 172.17.0.3/16 brd 172.17.255.255 scope global eth0
       valid_lft forever preferred_lft forever
(minikube:default)>