> relative to the CPU speed memory is ridiculously slow

Latency from cpu to memory sure.

Memory frequency has caught up though, so you have have more bandwidth than any CPU can deal with.