THis is relatively common with heavy for caliber VLD bullets. You are seeing the bullet go to sleep if you will as range increases. This means the bullet is fighting off the rotational effects of the rifling withing the first 100 yards or so of being fired.
The rifling makes the bullet spin around the axial center of the bore which often times is not the same as the bullets center of gravity as such, when the bullet is released by the rifling, the bullet is trying to transition from rotating around the axial center of the bore to rotating around the center of gravity of the bullet. This causes a wobble, if you will until the bullet "goes to sleep" and spins true around its center of gravity.
This is why the groups are larger as far as MOA measurement at closer range then you see it at longer range, just as you are seeing.
The longer the bullet the more you see this and often at longer ranges. The faster the bullet is rotated, again, on average the farther you will see this.
You often see this if rotation is marginal.
Again, there are many, MANY very good loads that are passed up on because many feel that if its not a 1/2 moa load at 100 yards, there is no need for any testing at long range. With conventional bullets, that may have some merit but with a heavy for caliber VLD type bullet that is often not the case at all.
So if your getting good, consistantly shaped groups and your velocity spreads are tight, your much wiser to test at long range and often times a much better result will show up then expected.
It could also be a result of paralax adjustment. If your PA is adjusted well for 300 yards but not at 100 yards, your groups will be larger if your eye position is not consistant at the closer ranges. You see this with non PA scopes or lower quality scopes or scopes that the shooter does not adjust PA prior to shooting groups.