Put a good recoil pad or brake on your rifle and run it like a 300 win. Where you shoot your critters controls meat loss, not the cartridge. The bullet has some effect on marginal shots, but why is the bullet there unless you shot the critter there to begin with?
That said, you can use a lower charge weight powder and slightly lighter bullet weight to drop recoil. Since you seem stuck on 180 barnes I'd look at something in the 4350 class and on the lower end of the charge range.
I'd actually have done the '06 myself if I was recoil sensitive as it's only 200-300 fps down from the 300 win when both are loaded to potential until the bullet weight gets better than 180 grains. Download a 300 and you are at '06 ballistics with more expensive brass and more powder burned.