Tytalus
You didn't mention birds so I'll not address shotguns. Were I in your situation today (and when I was younger I was), I would have 3 rifles in the following calibers:
1. 22LR (mostly for cheap shooting practice but they will kill a pest or two)
2. 22-250 Rem (cheap, fun to shoot, accurate and will handle up to deer where legal)
And when you want to put that practice you had with number 1 and 2 to use, you drag out number 3 (with a good muzzle brake attached).
3. 338 RUM shooting Barnes TTSXs. With those bullets this rifle will kill anything you will ever find in Canada and only damage it enough to kill it. You can as they say "eat the bullet hole". I shot a small whitetail with this cartridge to test the theory and it didn't hurt a bit of meat. The deer wobbled for 20 yards and fell over. The bigger the animal the harder it gets hit as these bullets usually don't stop in the animal. If it is 3 feet thick it goes through 3 feet shedding energy the whole time. If it is 1 foot thick it goes through 1 foot shedding energy the whole time - but less total energy is dumped than for an animal that is 3 feet thick. Good luck.