I'd try different bullets, specifically the 39gr Sierra Blitzking. I've had good results with that bullet, 24.8gr Reloder 10x, Federal 205M primers, Hornady Brass, seated at 2.26". I have also had good results with H335, 26.7gr.
Have a look at Home of 204Ruger.com and RugerHunting.com!
, under the loads. There is also a good forum on reloading where many people have posted a variety of loads - lots of Savage shooters. There aren't too many people shooting the heavy bullets in the .204 from what I have read, and most of the time when people have trouble finding a good load, the solution seems to involve switching bullets more often than not.