I do not know how long you have been shooting and do not wish to be too simple but my first impression would be that you have a stock weld/ paralax problem. That is why I listed those possiblilites first. It is very easy to get 0.5 MOA error with bad stock weld and bad paralax together. You can often get by with one problem or the other but when you have both simultaneously things go down the sewer in a hurry. Many cheap scope are set for paralax at 100yds and when you move out from there the problem becomes severe. If your scope does not have paralax adjsutment, then you msut have a very consistent shooting mechanics to get your eye in the exact same place and alignment each time. Tack this much error onto the wind and mirage effects at 500 and you have your answer. If you already know all of this then I apologize for going on and on and on about it. You should not expect to hold the same MOA at 100 yds all the way to 500yds but you should be doing better than you are.
Secondly, I would make very sure that the barrel is very clean.
To follow up on the bullet runout possibility you can do a search of this site for "run out" or "concentricity". Basically it means the loaded cartridge is not in line with the barrel and the bullet starts down the barrel out of alignment. This can be caused by an out of alignment chamber (my problem with one
or it can be caused by your reloading process of either bending the brass (usually the neck) or seating the bullet off alignment. Take several loaded cartridges and roll them across the table and see if the tip of the bullet wobbles around. If it wobbles on the table it has "runout" from center.
If you are new to relaoding there are some common beginner mistakes that will create a problem in excessive runout. First, if you have bottomed out your decapping rod then it is probably bent and will bend all of your case necks when the expander ball comes out of the case. Secondly, if you did not lube the inside of the case necks and the decapping rod is off center it will bend the case necks.
I will go back to my first suggestion. Next time you go to the range shoot at 100yds and then at 500 yds. If you get good groups at 100 and bad at 500 then this is not a reloading issue, it is a scope/eyeball issue.
These are jsut my opinions. Take what you want and throw the rest in the garbage.