I have everything from inexpensive factory to expensive factory and very high end customs. Top of the line Factory, like Weatherby MKV, Sako, Sauer, Mauser, Blaser, and semi-factory/custom shop like Purdy, Holland and Holland, and Taylor are great rifles, shoot well and have great customer service, while being slightly to a lot cheaper than customs. They also get a LOT more of their original price back than a custom will. Cheaper factory you tend to get what you pay for but Tikka and Vanguard/Howas have been consistently excellent rifles for pretty small dollars.
Mass produced semi customs like Fierce, Christianson etc I have zero use for. They are way over priced for what they typically deliver.
Building customs is fun if you have the budget to do it right and a top notch builder. They also tend to shoot great. Expect to keep it for life or take a big bath on resale.