Mandrels from turning can leave lots of ridges and burrs
I polish the inside of the necks with steel wool but use a Lee Zip Trim, easier to get a tight fit on the steel wool and hold the case without hand fatigue
By leaving a larger diameter of steel wool you can also take the burrs off the inside and outside neck chamfers.
Does it make a difference? Who knows? I use the Lee Collet so I don't have any lube in the neck and the necks come out pretty clean