I should have mentioned that the flat areas inside the ports is where the wrench would go.
But I agree with geargrinder... if it's shooting that well, I wouldn't touch a thing.
We're getting 2700 fps (just over) with 300 SMK's and 89.3 grains of Retumbo (Lapua brass)... Maybe now that the barrel is broken in, you can try the Retumbo again... when the lands dress forward a bit (at the rearmost edges), pressures drop. You're using Lapua brass, I'm assuming...
I've pushed the MV to 2800 with ~93 grains of Retumbo, but at the expense of stretched primer pockets... and 89.3 shoots the best anyway...
Those tool marks you're seeing must not be harming accuracy to any significant degree. Who knows, you could get a new perfect barrel and it wouldn't shoot as well as what you have.
I think if the barrel was over-sized at the very end, you wouldn't be getting the accuracy you're getting... so I wouldn't suspect that.