I'm not super familiar with either round but I'd say 22mag. They aren't very loud and with the heavier bullets I'd say 40gr or so you should be able to kill the heck out of them inside 100yd. I've shot the 17hmr a few times and it's loud. It's also moving much faster, I'm a hornady fan and they've got a couple new loads for the 22mag which look promising, 45hr ftx and the 35gr vmax. "I may or may not" have seen a couple deer killed with the 22wmr that dropped like a sack of taters...lol
I still feel the 17 is a lil light for anything over ground hog size but that's just my .02 The 22mag has been around for a long time and has earned a pretty good rep for killing yotes, foxes and cats up close. I'll just about guarantee any hardware store you go in will have them on the shelf too. I've got several pradator hunting dvds and a few of the guys are using a 22mag, up close looks to kill the just as dead as anything else and I doubt the animals could tell the diff. I would still try to make head shots on them but that's me like I said not too familiar with either round but if a 22lr that killed the fox snatching chickens from the neighbors coop a 22mag will be all the better. Good luck and happy hunting let us know what you decide and post some pics, we all love gun porn...lol