Break a link with the wrong MTU and then find it

applied · 50 min · Objective 1.4

Task

Create an MTU mismatch deliberately, observe the characteristic symptom — small things work, large things hang — and then diagnose it with ping the way you would in the field. This fault is extremely common behind VPNs and tunnels, and it is nearly impossible to guess if you have never seen it.

Steps

  1. Confirm the healthy state: from Host A, ping -c 2 192.168.20.10 and a curl of a large file from Host B both succeed.
  2. On the router, lower the MTU on the interface facing network 2: sudo ip link set eth1 mtu 1200. Leave both hosts at 1500.
  3. From Host A, test small traffic: ping -c 2 192.168.20.10. It still works. Now test large traffic: fetch a file of a few megabytes from Host B web server. It stalls, or transfers a little and then hangs.
  4. Diagnose it the way you would without knowing the cause. Use ping with the do-not-fragment bit and a payload size, bisecting until you find the largest that gets through: ping -c 1 -M do -s 1472 192.168.20.10, then 1172, then narrow down.
  5. Add 28 to the largest working payload size — 20 bytes of IP header plus 8 of ICMP — to recover the path MTU. Confirm it matches what you set. Then put the interface back with sudo ip link set eth1 mtu 1500.

Verify

ping -c 1 -M do -s 1472 192.168.20.10; echo "exit $?"
ping -c 1 -M do -s 1172 192.168.20.10; echo "exit $?"
ip link show eth1 | grep -o "mtu [0-9]*"

With the router at MTU 1200, the 1472-byte ping must fail and the 1172-byte ping must succeed. 1172 plus 28 is 1200, which is the path MTU you just measured without being told it.

Notes

The symptom pattern is the thing to memorise: ping works, SSH connects and then freezes, web pages load headers and hang, small files transfer and large ones do not. Anything that suggests the network is up but big things break should make you think MTU before anything else.

The reason it survives in the wild is that the ICMP message which should fix it automatically — path MTU discovery — is very often blocked by a firewall. That is why an MTU problem behind a tunnel is usually fixed by clamping the MSS rather than by fixing the ICMP filtering.