Not exactly sure what you've got going on here, but it's worth saying that if you're going to be doing any kind of accuracy shooting, you need a rock-solid platform. Whether that's a table, prone on the ground or whatever, rock-solid is the key term. Given that I do not have a concrete bench from which to shoot, I generally do my accuracy shooting prone with a harris bipod on the front and a heavy rear bag.
Also, 100 yard groups are not a great indicator of accuracy. You need to be testing at 200 or 300 to truly determine whether a load is accurate or not.
I'm not gonna shoot here. I'm gonna shoot waaaaaaaay over there!