Where does hard water come from?

Where does hard water come from? As rain water falls it is naturally soft. However, as water makes its way through the ground and into our water ways, it picks up minerals like chalk, lime, calcium, and magnesium.