Always 3-4"? I would start by looking at the gun - not the ammo. Make sure everything is clean, check the barrel for free float, check stock screws. Make sure mounts are tight and scope is all secure. If it is a very lightweight barrel, try a pressure point at the end of the stock to see if that helps (not a permanent solution, but helps diagnose). Have a gunsmith check it out if necessary.
If the gun looks good and you still question the reloads, try a box of Federal Premiums to see what they can do. Should be less than 2 inches. If not, sell the gun.