You will lose some velocity with the short barrel, but unless you hunt at very long range it probably will not make a great deal of difference.
I have a Winchester Model 70 with a 24 inch barrel, and I bought a Ruger Mark II with a 22 inch barrel to use for elk hunting in Wy. When I shot the very same factory load in these two rifle I found that I did not lose a great deal of velocity. This is what I found.
Federal Factory Load 300 WSM
, 180 grain solid base Nosler bullet.
Average 3 shots 3055 ft/sec with the 24 inch barrel. 3017 ft/sec with the 22 inch barrel. I know that this would not be true for all guns but it will help you see that you may not lose as much as you might think.