H870 will get you the most FPS while still being very accurate, but it's dirty discontinued powder :( Mine liked 74gr H870. Both 264 Win Mags I had liked 64gr RL25. That put me right around 3100 FPS with a 140gr.
I found both AA8700 & US869 to be too slow. 77gr was max & only gave me around 3080 FPS.
I am hearing a lot have good luck with Retumbo. My rifle didn't seem to like it. 65gr was max in mine.
The 264 Win Mag likes slow powders. I'd like to see what 50BMG or the new RL50 would be like?
A 264 Win Mag seems to be very picky, but once you find the sweet spot, they are a very fast & accurate round.
I ended up opening up my 264 Win Mag to a 6.5mm-300 Win Mag. I get 3275 FPS from 79gr H870 & a 24.5" barrel. Gotta love that 6.5mm BC.