Find the bottleneck with iperf and RTT

applied · 55 min · Objective 5.4

Task

Prove that a link rated at a gigabit does not deliver a gigabit for a single TCP stream over a long path, and find where a path's throughput is actually limited. You will measure achieved throughput with iperf, watch it collapse as round-trip time rises, and see why the answer is often window scaling rather than more bandwidth.

Steps

  1. Measure the clean path. Start iperf3 -s on Host B and run iperf3 -c 192.168.20.10 -t 10 on Host A. Record the throughput — this is your reference, close to the link's capacity on a short, clean path.
  2. Add round-trip delay to simulate distance: sudo tc qdisc add dev eth0 root netem delay 100ms. Re-run iperf. A single TCP stream now achieves far less, because the sender can have only a window's worth of data unacknowledged and spends most of its time waiting. This is the long fat network problem — the bottleneck is the window, not the wire.
  3. Prove it is the window and not the bandwidth. Run several streams in parallel: iperf3 -c 192.168.20.10 -P 8 -t 10. Aggregate throughput climbs well above the single-stream figure over the same impaired link — more bandwidth was never the fix.
  4. Now create a real capacity bottleneck: sudo tc qdisc change dev eth0 root netem rate 20mbit. iperf now tops out near 20 Mbit regardless of streams — this is a hard ceiling, the point that limits the whole path.
  5. Remove the shaping (sudo tc qdisc del dev eth0 root) and confirm throughput returns to the reference figure.

Verify

iperf3 -c 192.168.20.10 -t 5 | grep -E "sender|receiver"
iperf3 -c 192.168.20.10 -P 8 -t 5 | grep -E "SUM.*receiver"
tc qdisc show dev eth0
ping -c 5 -q 192.168.20.10 | tail -2

The two iperf figures are the deliverable: single-stream throughput far below the aggregate of eight parallel streams on the same link is the long-fat-network effect proven — the limit was the TCP window, not capacity. When the rate ceiling is in place, both figures converge near that ceiling, which is what a true bottleneck looks like. The ping RTT ties the throughput number to the path.

Notes

The practical lesson: when someone says "we have a gigabit link and only get 400 Mbps", that may be entirely normal, and the useful question is what the path and the protocol are, not what the interface is rated at. Bandwidth is a rating; throughput is what you achieve; and the gap is where the diagnosis lives — protocol overhead, congestion, errors, the devices at each end, and the round-trip time.

Finding the bottleneck is the whole job, because performance is governed by the single limiting point and upgrading anything else changes nothing. Work along the path checking utilisation at each hop, and remember the bottleneck may not be in the network at all — a slow application server produces exactly the same "it is slow" complaint. iperf between two internal hosts is how you measure a specific link instead of the internet, where the test server's own load muddies the result.