Your 12 twist will easily stabilize the 178 AMAX, if you're a fan of Hornady bullets... I shoot the 178's in 30-06 and .308, and they'll cake-walk to 1000 yards with as little as 2500 fps MV.
Since it's hard to find a range to shoot 1000 yards, it would seem to me that you'd rather go with a sure thing--the heavier bullet--than hope for making enough velocity to get the 168's to 1000 yards, something you can't be nearly as sure of.
There are some "go to" .308 recipes with the 178's... you can work up to 44.2 grains of Varget for a likely 2600 fps from your 20" barrel... and 42.0 grains of IMR 4895 is a great recipe that works in most .308's, making around 2550 from your rifle, estimating of course. This is in Winchester brass, by the way...
You really won't need to spend a lot of time and material making the heavier bullets work. Click the link in my signature line for an understanding of the OCW loading process, and you'll hopefully see what I'm talking about. In less than 20 shots, you'll be pretty much there if you follow the directions.
For tighter extreme spreads, the larger case fill of the 44.2 grain Varget charge will serve you better.
Work up to these charges, of course. I shouldn't have to mention that--it should go without saying--but I've had some encounters with self appointed mall cops lately that may perhaps expect me to state the obvious anyway.
If you're really set on the 168's, stick with the AMAX... JBM says they'll make it to 1000+ with lower velocities as well. Play around with JBM a bit for some more info... http://www.jbmballistics.com/cgi-bin/jbmtraj-5.1.cgi