I understand it does not seem follow the "equal and opposite" reaction law of physics, but it does. In order to understand the principal that "Wild Rose" brought up, you have to know what a "closed loop system" is in physics.
Until SOMETHING moves external to the barrel, the bullet, barrel, case......... are in a closed loop. When the bullet leaves the barrel, the closed loop is OPENED and all the recoil factors begin. As Wild Rose pointed out there is a tiny amount of air in the bore that is being pushed out the end of the bore in front of the bullet and there would be the tiny amount of expansion of the barrel steel. Those are EXTERNAL and keep the system from being a total "closed loop", however the effect of those things are so small as to not matter.
It has been 30 years since college physics for me so I am a little rusty here, but if you are interested in understanding, look up "closed loop systems" in a physics text.
After all that........ I will still maintain my statement that the recoil of the rifle does not affect it's accuracy, only how we react to it.
It's been fun, and a good discussion, and will admit I certainly could be wrong. Not trying to be the resident physics expert here because I'm not.