Cache AMAT Calculator

Modern CPUs access memory through a hierarchy of caches, each faster but smaller than the next. When a requested value is found at a level, it is a hit. When it is missing, the processor must look at the next (slower) level.

The AMAT formula for a two-level cache:

AMAT = L1 hit time + L1 miss rate x (L2 hit time + L2 miss rate x main memory time)

Every L1 miss incurs the full L2 hit time. Every L2 miss incurs the full main memory latency. Because main memory is 100-300x slower than L1 cache, even a small L2 miss rate has a large impact on average performance.

Typical values in modern desktop CPUs:

• L1 cache hit time: 1-4 clock cycles (~0.3-1.5 ns at 3 GHz)
• L2 cache hit time: 10-20 clock cycles (~3-7 ns)
• Main memory (DRAM): 60-100 ns

L1 miss rates for typical workloads range from 1% (tight loops with good locality) to 10% (pointer-chasing, large working sets). L2 miss rates are usually 0.1%-1%.

Why cache miss rate matters more than hit rate. A 99% hit rate sounds excellent, but 1% of accesses still going to main memory can dominate total runtime if memory latency is high. This is called the memory wall and has driven CPU design for decades.

Reducing AMAT is one of the central goals of cache-aware programming: improving data locality, avoiding cache thrashing with large strides, and keeping working sets small enough to fit in L1 or L2. Algorithms like matrix blocking and loop interchange exist specifically to improve cache behavior.