Power

When hunting small game, a 22lr or a 20 gauge shotgun work best. A 12 gauge works but is better for larger of the small game like ducks or geese. A crossbow or bow work but are difficult to hit

When hunting big game a minimum of a 223 rem is needed. usually I go with a 308 win but a .30-06 works well too. Crossbows and bows are great here but need a lot more skill. 

My firearms that I know are good:

1. Tikka T3X, 308 win - ~$1,100 + $800 scope
2. Franchi Affinity 3 - 12 gauge ~ $1,500
3. Derya TM22 - 22lr ~ $500 (including accessories and case)

Cost & Accessories

For larger caliber rifles like a 308 win, the cost of the rifle won't matter much in the long run compared to the ammunition. A good price for a pack of 20 rounds is ~$60 CAD and you can easily blow through a couple of those when you go to the shooting range. The people at the store will try to sell you a very expensive scope and they are partially right. For deer hunting and casual use anything good quality will work, it'll still be something in the $400+ range though.

For a 12 gauge the ammunition is much cheaper and it's so easy to aim so you don't need much practice. A pack of 250 target shells costs ~120. A 20 gauge would be cheaper and lighter too. The 22lr is the cheapest with 100 rounds being ~$15

Accessories are a must and I'd go with the premium but not rip off options. A good cleaning kit will save you time and effort. 

A locker or safe is also a very good investment and don't trust those listings like “10 gun cabinet” it won't even fit 3 or 4. Seeing it in person first is the best way to go.