Re: 111 LRH Loads
That's a fairly light barrel with a big cartridge intended for hunting. So, it'll heat quickly.
You might try 3 shot groups with plenty of time to cool.
When you have tight groups with the 5th shot being a flier, it's often the shooter's fault.
Also, using the same die unadjusted with the 165gr vs 180gr isn't the same seating depth since the ogive has a slightly different shape. Even if it was the same shape/depth, the velocity won't be the same and the bullet won't exit the muzzle at the same point in the vibration/whip cycle of the barrel.
Hence, you probably do need to shoot a few groups at various seating depths to find out which one it prefers.