Traffic Engineering at Scale: Request Paths, Load Balancing, Caching, and Resilient Delivery
Torres, Miguel
Synopsis "Traffic Engineering at Scale: Request Paths, Load Balancing, Caching, and Resilient Delivery"
A request does not simply reach a backend. It crosses names, routes, networks, connections, proxies, caches, queues, and admission controls - and every layer can add state, latency, or failure. Traffic Engineering at Scale follows the complete request path from a user device to useful application capacity. Written for backend and infrastructure-minded engineers, it explains how DNS, anycast, Layer 4 and Layer 7 load balancing, gateways, service meshes, CDNs, health checks, connection pools, retries, and overload controls work together as one production system. Rather than presenting infrastructure as a catalog of products, the book asks operational questions. What happens while DNS answers remain cached? Which component owns a retry? How does a load balancer react when topology changes? Can a regional drain move new requests without breaking existing connections? When does a cache protect the origin, and when does it amplify failure? Inside, you will learn how to: - Trace requests across resolver, network, proxy, cache, and origin boundaries - Compare Layer 4 and Layer 7 balancing and their different state models - Reason about anycast, geo-routing, failover timelines, and stale DNS answers - Design cache keys, freshness policies, shielding, and origin protection - Manage keep-alive, pooling, multiplexing, draining, and connection churn - Coordinate deadlines, retry budgets, rate limits, and load shedding - Test multi-region traffic shifts, overload behavior, and recovery paths Concrete failure timelines, public engineering reports, open-source data planes, runnable labs, and review checklists connect each mechanism to real operational consequences. The goal is not to memorize a canonical stack. It is to understand who owns state at every hop and how the entire path changes under stress. Traffic Engineering at Scale gives engineers the vocabulary and methods needed to design request delivery that remains observable, controlled, and useful when traffic and infrastructure refuse to behave politely.
