K
Systems KeepaliveA Kordish publication
Calibration record

Fast internet. Full inbox. Nothing moves.

A photo upload can stall on a perfectly usable connection. The receiver needs room to accept more bytes. More bandwidth cannot negotiate with a full tray.

8 minTCP/IPUpdated
  • tcp
  • flow-control
  • linux
  • networking

Everyday use

One photo. Several invisible agreements.

Stick character uploads a photo while tiny couriers carry envelopes behind the laptop. Visual metaphor for a TCP byte stream.
Couriers represent bytes, not literal packets or a promise that the application saved the photo.

You send a photo and expect a link back. When an upload uses TCP, the connection delivers an ordered stream of bytes to the receiving application. Usually the application keeps reading and everything feels immediate. You never need to notice the flow control quietly keeping the receiver from being overwhelmed.

How it works

The other end gets a vote.

A courier waits at a receiver desk with limited tray space and a stop gesture. Visual metaphor for receiver flow control.
The advertised window limits incoming bytes; an acknowledgement is not an application completion receipt.

The receiver advertises how much more data it can accept. Think of a desk with limited inbox space: the sender must respect the available room. Acknowledging bytes and making room for more are different things. An established connection therefore tells you very little about whether an upload is making useful progress.

Failure

The network is up. Your upload is stuck.

A receiver clerk is tangled in a filing cabinet beside a full inbox while the courier waits. Visual metaphor for a blocked receiving application.
The cabinet represents a downstream dependency. Not every slow upload follows this chain.

Suppose the receiving application waits on slow storage and stops reading. Its socket buffer fills, the receive window can reach zero, and the sender pauses new data. TCP can keep probing without completing the upload. The service remains connected; the person still has no shareable photo link. Those are entirely compatible facts.

Recovery

Make room at the receiving end.

A clerk clears the filing blockage, drains the inbox and waves the courier onward. Visual metaphor for restored receiver progress.
Confirm acknowledged bytes and application completion resume; an open connection alone is insufficient.

Check receiver-window and queue evidence before changing the network. If the application is blocked, restore its ability to read or reduce incoming work while its dependency recovers. Buffers can absorb a brief pause; they cannot store an endless one. A resumable upload can spare the user a restart when the service supports it.

Operator notebook · mechanism, safe drill, commands and diagnostic signals

Core mechanism

The receiver controls how far the sender may advance

The TCP receive window is receiver-advertised flow control: it identifies the range of sequence space the receiver is currently willing to accept.

Each ACK carries a window value. The sender may advance new data only inside that advertised range, after applying the window scale negotiated during the handshake. If the receiving application drains its socket slowly, buffered data consumes the window and the advertised value shrinks.

A zero window stops new data without proving packet loss. The sender enters persist behavior and sends occasional probes so a lost window update cannot deadlock the connection. Congestion control remains a separate limiter: the congestion window protects the path, while the receive window protects the peer's buffers.

Operational consequence
A transfer can have queued outbound bytes without a growing retransmission count. A persist timer or zero advertised window points toward the receiving application or its socket capacity before it points toward the path.
Boundary
Flow-control evidence identifies the side applying backpressure. It does not explain why that application stopped reading; scheduler delay, downstream blocking, garbage collection, and deliberate rate limiting remain candidates.

Rust removal

Sequence numbers count bytes

TCP sequence numbers track octets in the stream rather than packets on the wire. Segmentation offload can make host captures show large logical segments, while the NIC or a later hop emits smaller packets. Retransmission can also repacketize the same sequence range differently.

SYN and FIN each consume one sequence number. A pure ACK consumes none. Packet counts are useful for several questions, but they cannot establish byte-stream progress without the sequence and acknowledgment ranges.

Still matters: Comparing packet counts across capture points can manufacture loss when offload changed segmentation. Compare acknowledged byte ranges before opening a ticket against the network team.

Break it safely

Distinguish path loss from a receiver stall

Goal: Inspect one slow established TCP connection and classify its limiting evidence as receiver flow control, retransmission, local application queuing, or unresolved.

Assumptions

  • Run on Linux with a recent iproute2 package and permission to inspect the target socket.
  • Use a connection that is already slow; every command is read-only.
  • Socket state is network-namespace local. For a containerized process, inspect from that process's network namespace.
  • Treat host-wide counters as corroboration because unrelated connections also increment them.

Procedure

  1. Record the connection's local and peer address plus port. Keep numeric output so name resolution cannot add noise.
  2. Take two socket snapshots several seconds apart while the transfer should be making progress.
  3. Compare Send-Q, timer state, bytes_acked, bytes_retrans, retrans, rtt, cwnd, and snd_wnd when the kernel exposes it.
  4. Check whether relevant host counters advance during the same interval.
  5. If the process is in another network namespace, repeat the socket inspection there before concluding the connection is absent.
  6. Classify the evidence before changing buffers, congestion control, firewall rules, or timeouts.
Inspect matching established sockets twicebash
peer='203.0.113.20'
port='443'

for sample in 1 2; do
  date --iso-8601=seconds
  ss -tinmo state established \
    "( dst $peer and dport = :$port )"
  test "$sample" -eq 2 || sleep 5
done

Replace the documentation address and port. The timer named persist is the zero-window probe timer. snd_wnd is especially useful when present, but iproute2 and kernel versions expose different field sets.

Read relevant host TCP countersbash
nstat -az | grep -E \
  '^(TcpRetransSegs|TcpExtTCPTimeouts|TcpExtTCPLossProbes|TcpExtTCPAbortOnTimeout|TcpExtTCPZeroWindowDrop)[[:space:]]'

Capture this output at both ends of the observation interval and subtract. A host-wide delta supports a hypothesis; it does not attribute the event to your socket.

Inspect from a process network namespacebash
pid=12345
nsenter --target "$pid" --net -- \
  ss -tinmo state established

Replace the PID with the target process. Namespace entry may require elevated privilege, but the socket query does not mutate network state.

Model to carry

Every sender answers to two governors

For new data, TCP is constrained by the receiver's advertised window and the sender's congestion window. The tighter governor determines how much unacknowledged sequence space can be in flight.

The receive window reports the peer's ability to buffer more bytes. The congestion window represents the sender's estimate of what the path can carry safely. A fast path cannot overrule a full receiver, and an empty receiver cannot overrule a congested path.

Application buffers surround both governors. Data can wait before TCP accepts it, inside the send queue, in flight, inside the peer receive queue, or after the peer reads it. Locate the queue that is growing before tuning a window.

Boundary
The usable flight size also depends on bytes already outstanding and implementation details such as pacing. min(cwnd, rwnd) is a diagnostic model rather than a complete sender algorithm.
Failure mode
Treating every throughput collapse as packet loss leads to buffer inflation and timeout changes while the receiver remains blocked. The connection gets more memory and keeps the same bottleneck.

Tooling weather

AWS changed a firewall default that could drop TCP control traffic

On June 22, 2026, AWS Network Firewall changed the default stateful action for newly created policies to Application drop established (server-directed only). AWS says the prior bidirectional default could silently drop legitimate server-to-client window updates, keep-alives, and resets.

Why it matters: New policies receive the safer default, but existing policies do not become evidence-free. Audit their stateful default actions before diagnosing intermittent stalls. AWS documents a separate consideration for policies retaining bidirectional behavior to handle fragmented post-quantum TLS handshakes.

Signal sources1 source
  1. AWS Network Firewall updates default drop action for improved connection reliability

    Amazon Web Services · Published 2026-06-22 · Accessed 2026-09-02

Check your bearings

Three checks before the pager gets a vote

Source ledger4 sources
  1. RFC 9293: Transmission Control Protocol (TCP)

    RFC Editor · Accessed 2026-09-02

  2. ss(8) — Linux manual page

    Linux man-pages project · Accessed 2026-09-02

  3. tcp(7) — Linux manual page

    Linux man-pages project · Accessed 2026-09-02

  4. AWS Network Firewall updates default drop action for improved connection reliability

    Amazon Web Services · Published 2026-06-22 · Accessed 2026-09-02