I don't shoot Factory 300roy, but I do help a buddy load for his Accumark. I think you may have your best luck with the heaviest bullet weights you can find for this chambering from what I'm seeing. Others on the forum have mentioned the heavy bullet preference about their 300roy rifles too... Even the 180's you mentioned may be on the light side for really good accuracy...