I am only citing one possible issue.
Well you can feel the difference in a good spring and buffer did you buy a kit with the spring and buffer included? From who? Case in point, I built an upper for a buddy, all BCM, LMT, and DD parts. He supplied the lower, it was built local and who's parts kit they used in anyone's guess. After about a month he shoots me a text and says is short stroking etc. We set up a training date and I threw another one of my lowers in my box just in case. It was apparent to me the issue the first time I pulled the trigger, I handed him the rifle and told him fire one round and concentrate on how the buffer and spring feel as the weapon recoils, Now shoot mine, feel the difference? YES, swapped the lower and finished shooting. He replaced the spring and buffer with BCM H buffer and spring no more issues.
Your buffer should be stamped H something or A something, if its not you have a junk buffer, the spring well who knows? The buffers all come in different weights, H, buffer being the standard for the M4 series of rifles, however the current trend is to run a heavy buffer to mitigate an issue known as "bolt bounce". It has a very, very small effect on accuracy, but combined with other issues it is a factor non the less.
I will suggest a check list and some dime drills.
1. Make sure your buffer tube is strait with the receiver.
2. Check to see if your castle nut is properly staked to prevent it from loosening.
3. Get a new spring and buffer, I would suggest calling Vltor and asking their opinion (know what BCG you have).
4. Check your crown for burs, use a cotton ball see if anything get stuck there
5. Check gas tube alignment
6. Try a box of good ammo, federal match, black hills match, in a bullet weight between 69 and 77g.
7. Also, this may be out of your control and I don't know how the for end you have is installed is attached. But look up the manufactures torque specs and check them to.
Lastly "dime drills" this is what they teach in boot camp and looking at your rifle i'm not sure its practical with that much barrel. But you can modify the drill, have someone else balance a dime on the round part of your barrel while you practice your trigger pull while aiming down the sights. The object is for the dime to remain in place as your dry fire the weapon.
Lastly, this sucks and parts are not parts you have some good and some unknown, do what you can and good luck
Jon