Yes the point of impact will change a little each time you mess with the stock/action relationship. Think about barrel harmonics, the barrel is like a tuning fork that is held in a vice. Each time you shift the fork in the vice and relock it, it will produce a different pitch. When people hand load, we are striving for consistancy to make sure that each shot creates the same velocity, and vibration so the bullet leaves the barrel at the same vibration each time. This is not the most scientific explination, but I hope it clears things up a bit
I used to re-load but now I "hand-load".
-- Well, at least I try