I've like the 'Micro-Hunters' from Browning in 20" barrel lengths. That includes the A-Bolts & newer X-Bolts. Another I am fond of is the Mod 70 'Classic Compact', which has a 20" barrel. I have these in 284 Win, 7mm-08, & 243.
For your chambering I'd consider 243, or maybe even the 22-250, especially as a reloader. The term 'Varmint rifle' can vary widely with individuals, type of hunting, location, & intended game. I consider 500 yards to be somewhat specialized shooting, I'd rather be well inside that range. Oh yes, I've read about the great shots at longer ranges on the net. I also seem to see a fair amount of wounded deer in my travels. One year we dealt with 5 wounded/dead/lost deer from neighbors on the smallish IL deer lease.
Of course nothing wrong with 800+ yard prairie dog shooting.
