What distance are you shooting?
How are you loading? Single shot loading, magazine fed or something else?
In my hunting rifles, whether custom or factory and within 600mtrs, I don't load any closer than .010", even my small varmint cals. Dust, dirt and other anomalies that can booger up a rifle probably will in my experience.
The Brian Litz bullet jump test has always proven itself to me, the only time I jam Bullets or kiss the rifling is with dedicated target guns. This is sometimes necessary, but not always.
Anyway, .020" is as good as any other measurement to start testing at.
Cheers.