I don't think you would need anywhere near $50 k for tooling. Like I said, you can probably get a nice south bend and enough tooling to do muzzle brakes, crowns, contours and the likes for under $10k, and that might get you a mill as well. You just need to get a big enough lathe to thread the barrel between centers if at all possible. Some would say to run a steady rest, but the chances of error start climbing. It's one thing to goof on your own gun, but customers are less forgiving.
The other plus thing is with the lathe and mill, they are not just for gunsmithing. They are capable of making all kinds of parts for your car hobby. Then all you need to do is get into backyard metal casting and you'll find yourself out of time to get bored