I called the border patrol last week. Agent i spoke to stated i could bring them back for personal use.... suggested a reasonable amount. Maybe 2000 primers and 8 pounds of powder. Be sure to declare it on way back. Most likely will do vehicle inspection.
I have not confirmed you can actually buy powder and primers in Canada as a US resident yet.
I would call the crossing you will be crossing at.
To buy primers and powder here in Canada all you need to do is provide you name and address. Most retailers don't check ID so Donald Duck has bought a ton of powder over the years. I don't know what your immport restrictions but there are no export restrictions from Canada to the U.S. that I am aware of. You cannot mail powder or primers here, they have to go via Canpar, Fedex etc. Canpar is reasonable cost, Fedex and the rest, not so much.