Troubleshoot Linux Packet Loss on Servers: Commands & Fixes

Practical, command-first guide for sysadmins and network engineers to diagnose and fix packet loss on Linux servers. Work top-down from physical links to application layers, capture evidence, and test one change at a time so fixes are reversible.

Table of Contents

Quick diagnostic checklist

Before changing settings, gather facts: timestamps, affected IPs, interfaces, direction of loss, and whether the issue is continuous or intermittent. Useful initial commands:

ip -brief addr — interface state and addresses
ip -s link show dev eth0 — tx/rx packet and error counters
dmesg --ctime | tail -n 200 — kernel messages since boot
ss -s — socket statistics

Verify physical NICs and drivers

Start with link-level checks to rule out cabling, SFPs, or switch port issues.

ethtool eth0 — link speed, duplex, and driver/version fields
ethtool -S eth0 — NIC-specific statistics (CRC, RX errors, collisions)
ip -s link show dev eth0 — kernel-level rx/tx drops and errors
dmesg | grep -i -E "eth|link|nic|e1000|ixgbe|mlx" — driver messages

If you see CRC/rxe length errors or link flaps, replace cables, swap switch ports, or test with a different NIC. When a driver update coincides with new loss symptoms, consider rolling back to the vendor-stable driver and collect logs for vendor support.

Basic packet loss tests: ping, mtr, and directionality

Compare tests from client-to-server and server-to-client to determine directionality.

ping -c 200 -i 0.2 10.0.0.5 — sustained ICMP test
mtr -r -c 100 --report-cycles 10 10.0.0.5 — historical hop-by-hop loss and latency
Run tests from multiple locations to distinguish host vs network issues.

If loss appears only on the last hop, focus on the destination host (NIC, queues, CPU). If mid-path loss shows on intermediate hops, open a ticket with network operations including timestamps and mtr output.

MTU and fragmentation checks

Mismatched MTU or broken Path MTU Discovery often drops large packets. Check MTU on all local interfaces and tunnels:

ip link show — MTU per interface
ip route show — MTU may be set on routes
Test maximum non-fragmenting payload: ping -M do -s 1472 target and adjust size down until successful.

For tunnels, VXLAN, GRE, or VPNs, calculate overhead and standardize MTU across endpoints (e.g., set physical MTU 1500, reduce tunnel MTU to 1450). Also check for devices dropping ‘DF’ flagged packets.

Bonding, bridging, and offload interactions

Check bonding/bridge mode and ensure switch configuration (LACP/ad‑hoc) matches server settings:

cat /proc/net/bonding/bond0 — bonding mode and member status
bridge link show — bridge port state

Offloads can mask packet issues or cause corruption with virtual switches or buggy drivers. Temporarily disable offloads for testing:

ethtool -K eth0 gro off gso off tso off
Re-enable after testing: ethtool -K eth0 gro on gso on tso on

Queues, QoS, and tc debugging

Host-level drops often occur when transmit/receive queues or qdiscs drop traffic under load.

cat /proc/net/dev — device counters including drops
tc -s qdisc show dev eth0 — show per-qdisc statistics and drops
ss -i dst_ip:port — socket info and retransmit statistics

Consider increasing txqueuelen (ip link set dev eth0 txqueuelen 10000) temporarily or tune qdiscs to prioritize critical flows. When using eBPF or complex tc filters, disable them briefly to isolate the cause.

Firewall, routing, and conntrack

Stateful firewalls and full conntrack tables can cause dropped or reset connections.

conntrack -S — current conntrack counters
conntrack -L | wc -l — active entries count
iptables -L -v -n or nft list ruleset — rule hit counters

Temporarily increase net.netfilter.nf_conntrack_max if tables are full, and inspect rules for accidental DROP chains or rate-limited rules. Also validate asymmetric routing: packet checks should include source-based routes and return paths.

Packet captures and correlation

When basic checks don’t reveal the cause, capture packets to prove loss and where it occurs.

tcpdump -i eth0 -w /tmp/capture.pcap host 10.0.0.5 and host 10.0.0.6
Use two-sided captures (client and server) to compare sequence numbers and see where retransmissions or drops occur
For high-volume environments, use hardware timestamping where available: tcpdump -j adaptername ... or libpcap options supported by the NIC driver

Monitoring, alerting, and long-term fixes

Implement continuous metrics and alerting for interface errors, drops, and queue utilization. Capture softirq/CPU usage during incidents to detect interrupt saturation:

watch -n 1 cat /proc/softirqs
Collect per-CPU interrupts: cat /proc/interrupts
Integrate with Prometheus or other monitoring to record node_network_receive_errs and node_network_transmit_errs

Plan long-term fixes: scheduled firmware/driver updates, standardized MTU across overlays, documented bonding and switch configs, and automation for alert-driven remediation.

Remediation checklist

Confirm physical link and replace suspect cables/SFPs
Match switch and server LACP/bonding modes
Standardize MTU across path and adjust tunnel MTU
Disable individual offloads to identify driver issues and apply vendor patches
Tune txqueuelen/qdiscs or implement QoS prioritization for critical flows
Increase conntrack limits temporarily and tighten firewall rules to avoid accidental drops
Collect and attach packet captures and ethtool/tc outputs for vendor escalation

FAQ

Q: Packet loss only under high load — what to check first?
A: Check CPU softirq/steering, interrupt distribution across CPUs, tx/rx queue drops, and offload behavior. Use cat /proc/interrupts, top/htop, and tc -s qdisc.

Q: How to tell if packet loss is host-side?
A: Run identical tests from multiple clients and the server. If only traffic terminating on the host shows loss and host counters (proc/net/dev or ethtool -S) increment, the issue is on the host.

Conclusion

Troubleshooting Linux packet loss on servers is systematic: establish scope, capture data, and rule out layers from physical to application. Use the commands above to gather evidence, apply one change at a time, and retain captures and logs for rollback and vendor escalation. Proper monitoring and standardization (MTU, bonding, QoS) prevent recurring incidents.