Networking Essentials for DevOps

TCP/IP & the Layers That Matter

18 min Lesson 1 of 30

TCP/IP & the Layers That Matter

Every production incident involving connectivity — a pod that cannot reach its database, a load balancer timing out, a microservice throwing Connection refused — traces back to one of four conceptual layers. Before you reach for tcpdump or open a firewall rule, you need a precise mental model of where in the stack the failure lives. That model is the TCP/IP four-layer stack.

Why four layers, not seven? The OSI model has seven layers and is invaluable for vendors selling hardware. The TCP/IP model collapses those into four that map directly to the kernel, the socket API, and the tools you run every day. In production, nobody debugs "Layer 6 — Presentation"; they debug TLS handshakes, which is Layer 4 in TCP/IP terms.

The Four Layers

Each layer adds a header (and sometimes a trailer) to the payload handed down from the layer above. This process is called encapsulation. On the receiving end, each layer strips its own header and passes the remainder up — decapsulation.

The TCP/IP four-layer model. Each layer wraps the layer above inside its own header — building a frame by the time it hits the wire.

Layer 1 — Network Access (Link Layer)

This layer deals with physical transmission between two directly connected nodes — the NIC driver, Ethernet frames, MAC addresses, and VLANs. In cloud environments the "physical" medium is virtual: VXLAN tunnels (used by Kubernetes CNI plugins like Flannel and Calico) wrap IP packets inside UDP datagrams so overlay networks span multiple physical hosts.

DevOps engineers rarely configure this layer directly, but its failure modes surface constantly: MTU mismatches cause mysterious packet drops that look like application timeouts. Check MTU with:

# Show MTU for all interfaces
ip link show

# Confirm effective path MTU to a host (ICMP probe)
tracepath -n 10.0.0.1

# In Kubernetes: pods default to host MTU minus tunnel overhead
# Flannel VXLAN subtracts 50 bytes; if host MTU is 1500, set pod MTU to 1450
# In flannel ConfigMap:
#   "Backend": {"Type": "vxlan", "MTU": 1450}

Layer 2 — Internet (Network Layer)

IP is the universal glue. Every packet carries a source IP, a destination IP, a TTL (Time To Live, decremented by each router hop), and a protocol field (6 = TCP, 17 = UDP, 1 = ICMP). Routing decisions happen here — the kernel consults its routing table and either delivers locally or forwards to a gateway.

# Print the local routing table
ip route show

# Example output:
# default via 10.0.0.1 dev eth0
# 10.0.0.0/24 dev eth0 proto kernel scope link src 10.0.0.42
# 172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1

# Trace hops to a destination (uses TTL=1,2,3... probes)
traceroute -n api.internal.example.com

# Quick ICMP reachability check
ping -c 4 -W 2 10.0.0.1

Production tip — TTL as a diagnostic signal. An ICMP "Time Exceeded" response (TTL expired) tells you where in the path packets are dying. If traceroute stops at hop 5, a firewall or black-hole route exists there. If TTL reaches the destination but connections still fail, the problem is at Layer 3 (Transport) or above.

Layer 3 — Transport

Transport adds end-to-end delivery semantics. The two dominant protocols:

TCP — connection-oriented, reliable, ordered. Uses a three-way handshake (SYN / SYN-ACK / ACK) before data flows. Retransmits lost segments. Used by HTTP/1.1, HTTP/2, gRPC, PostgreSQL, Redis.
UDP — connectionless, unreliable, low-latency. No handshake, no retransmit. Used by DNS lookups, QUIC/HTTP/3, VoIP, game servers, and some monitoring agents.

Ports live at this layer. A socket is identified by the four-tuple: (src IP, src port, dst IP, dst port). The OS uses this tuple to demultiplex incoming segments to the right process. Ephemeral ports (typically 32768–60999 on Linux) are assigned by the kernel for outbound connections.

# Show all listening TCP sockets and the owning process
ss -tlnp

# Show established connections to port 5432 (Postgres)
ss -tnp state established '( dport = :5432 )'

# Check the ephemeral port range on your kernel
cat /proc/sys/net/ipv4/ip_local_port_range
# Typical output: 32768   60999

# Raise it for high-throughput servers (add to /etc/sysctl.conf)
# net.ipv4.ip_local_port_range = 1024 65535

Layer 4 — Application

Everything above TCP/UDP is the application layer: HTTP, TLS (treated as an application-level concern in this model), DNS, SSH, gRPC, and your own microservices. This is where request routing, authentication, and business logic live.

Where do DevOps problems live? Empirically, the distribution in production is roughly: ~50% of connectivity issues are Layer 3/4 (wrong firewall rule, missing security group, port not listening), ~30% are Layer 2 routing/NAT misconfigurations, ~15% are DNS failures (Layer 4 application issues that look like Layer 2), and ~5% are genuine Layer 1 MTU or NIC driver problems. The layered model gives you a systematic triage order: start from Layer 1 up, or from Layer 4 down — depending on the symptom.

Encapsulation in Action: A Real HTTP Request

When your application calls POST https://api.example.com/orders, here is what actually happens at each layer on the sending machine:

Application: HTTP library serialises headers + JSON body into bytes.
Transport: Kernel TCP stack wraps in a TCP segment with src/dst ports, sequence number, flags.
Internet: IP layer prepends the IP header with src/dst IP addresses and TTL.
Network Access: NIC driver wraps in an Ethernet frame with src/dst MAC addresses and sends on the wire (or VXLAN tunnel in cloud).

On the receiving server each layer peels its header, validates checksums, and passes the payload up. The final payload — the HTTP bytes — lands in the server process's socket buffer.

Capturing Packets Across Layers

# Capture on a specific interface; -n skips DNS resolution; -v shows TTL/protocol
tcpdump -i eth0 -n -v 'host 10.0.0.42 and port 443'

# Decode the TCP handshake (SYN, SYN-ACK, ACK)
tcpdump -i eth0 -n 'tcp[tcpflags] & (tcp-syn|tcp-ack) != 0 and host 10.0.0.42'

# Write a pcap for later analysis in Wireshark
tcpdump -i eth0 -n -w /tmp/capture.pcap 'host 10.0.0.42'

Production caution: Running tcpdump on a busy interface with no filter can generate hundreds of MB of output per second and spike CPU on the host. Always add a host or port filter, use -c <count> to limit packet capture, and write to a file with -w rather than printing to the terminal. On Kubernetes, use kubectl debug with an ephemeral container running tcpdump — never install tools directly in production containers.

Key Takeaways

The TCP/IP model has four layers; each adds a header during encapsulation and removes it during decapsulation.
Layer 1 (Network Access): physical delivery, MAC addresses, MTU — VXLAN in cloud/Kubernetes overlays.
Layer 2 (Internet): IP addresses, routing tables, TTL, ICMP — where ip route and traceroute operate.
Layer 3 (Transport): TCP (reliable) vs UDP (low-latency), ports, the four-tuple socket identifier — where ss and firewall rules operate.
Layer 4 (Application): HTTP, DNS, TLS, gRPC — your code and most of your config.
Triage incidents by layer: confirm physical reachability first (ping), then routing (traceroute), then port reachability (telnet/nc), then application protocol (curl -v).