I know some time has passed since you started this thread. Just thought of something else that might be helpful. In my experience with the .280AI I've never been able to get good accuracy out of Nosler 280AI brass. I've always had to fire form my own from Rem brass. Others may have their own experience. But I've found to get my rifle under .75 groups I needed to fire form.
In my 280AI, I have used H4834SC, IMR4350, H1000, IMR7828SSC, Retumbo
Bullets have used are, 140gr accubond, 160gr accubond, 162gr A-Max, 168gr VLD berger, 168gr classic and 168gr long range accubond.
My current pet load is with 168gr long range accubond and Retumbo powder with Fed 215M primers.
What kind of velocity are you getting with Retumbo and the 168's? I was going to test H1000 and Retumbo next.