I never had much luck doing it that way either.
Seems like sometimes the bullet will "stick" in the rifling and COAL measurement will be longer because we pull it out of the case some when we pull the bolt back.
Seems like sometimes the ejector plunger isn't getting compressed fully, so the COAL measurement is shorter than it should be.
The ejector plunger can cause big issues when trying to do this, as can the extractor IMO.
I tried it by sooting the bullet with a candle, marking it with ink, split neck cases, neck turned minimal tension cases, thick neck bushing sized maximum tension cases, you name it...........inconsistant results, just like you're seeing.
I believe the best way to get it (if you want to use a real bullet and case and bolt is to remove the plunger and extractor, close the bolt, open the bolt and gently push the bullet and case back out with a cleaning rod. But that's too **** much work, especially since we need to check it often as throats get eroded from shooting.
So, long story short; I ended up getting an RCBS Precision Mic which has worked wonderfully for all my factory chambered rifles for many years now. Key Tip: if you use one, make sure the screw is as tight as you can get it with your fingers........I mean really tight. It's got to fully compress the ejector and slip past the extractor to get consistant accurate measurements. You want to feel alot of resistance when you push the bolt forward and compress the guage.
Hornady makes a tool, Sinclair makes a tool, as probably do others also. I use the sinclair model and comparators on the custom wildcat chambers now. Never have used the Hornady one. Regardless of which one you use/get, taking a reading off the ogive/bearing surface junction is the preferred method, rather than from the tip. You can seat two different type bullets to the exact same OAL (base to tip) and they will be different distances from the rifling because of the shape of the bullet (Ogive length, Bearing surface length, ect).
Not an expert on this by any means, but through trial and error I found a system that works well for me. Hopefully that helps save you some frustration.