Memory latency matters. As we can see, accessing memory cause 65ns to 106ns in Nehalem arch.
This figure is from paper by Daniel Molka, “Memory Performance and Cache Coherency Effects on an Intel Nehalem Multiprocessor System”, Intl Conf on Parallel Architectures and Compilatoin Techniques, 2009.
