I don't think there is any limit.
If a rifle is shooting 6" groups at 100 yards, it's probably not the load. Probably something loose, extremely poor bedding, or the barrel is shot out. Or maybe, just maybe, you are using long heavy bullets which cannot be stabilized well.
Do you have...