COMPUTER MEMORY, CACHES, AND PERFORMANCE: Understand How Data Movement Shapes Program Speed
HALSTEAD, CORIN
Synopsis "COMPUTER MEMORY, CACHES, AND PERFORMANCE: Understand How Data Movement Shapes Program Speed"
Why does faster hardware sometimes fail to make software faster? Why can adding more threads stop improving throughput? And why can a small change in data layout suddenly transform application performance? Modern processors can perform extraordinary amounts of computation, but computation is only part of the performance story. Before useful work can happen, instructions and data must move through registers, caches, translation structures, main memory, interconnects, and sometimes accelerator memory. When that movement becomes inefficient, even powerful hardware can spend valuable time waiting. Computer Memory, Caches, and Performance gives developers, systems engineers, students, and performance professionals a practical framework for understanding how data movement shapes real-world software speed. Beginning with latency, bandwidth, locality, and parallelism, Corin Halstead explains how modern memory hierarchies behave and why working-set size can create sudden performance cliffs. You will learn how CPU caches actually use lines, sets, associativity, replacement, prefetching, and write policies—and why seemingly independent variables can interfere through false sharing and coherence traffic. The book then moves deeper into DRAM channels, banks, memory controllers, bandwidth saturation, virtual memory, TLBs, page-table walks, page faults, huge pages, NUMA placement, thread affinity, and multicore scalability. But this is not simply a hardware reference. The emphasis remains on better software decisions. Discover how data layout, compact representations, loop order, tiling, batching, allocation patterns, copy avoidance, and ownership design can reduce unnecessary movement and improve useful work per byte. Dedicated chapters show how to measure these effects using hardware performance counters, profiles, traces, size sweeps, scaling experiments, and roofline-style reasoning. The same principles are then applied to web services, databases, analytics, scientific computing, cloud systems, GPUs, HBM, unified memory, and modern accelerator workloads. With practical examples, case studies, professional checklists, troubleshooting patterns, a 30-day learning sequence, and a complete Memory Performance Investigation Playbook, this book helps you replace performance guesswork with evidence. Follow the data, identify the real bottleneck, and make optimization decisions you can measure and defend.
