In my opinion you are incorrectly assuming that the higher impacts are are due to the load being hotter when in fact it's probably due to higher BC values that are creating a flatter trajectory.ok I have loaded ammo at 200'and at 5000 they shoot high about 2" at 100 m and the handloads are very hot up here... you need to drop your powder charges 1.5 grains so you will not have flattened primers.. from 200 to 6000 elevation.. I am sure that is why the 416 rigby is loaded at 40 000 cup instead of 50 000 cups because in africa it is high elevation.. I think that elevation has more effect than tempature.. another thing if it is light rain you velocity will go up as much as 75 fps..
Temperature has a greater impact on trajectory. Higher altitudes usually result in lower temperatures. Higher altitudes will also cause the pressure to drop.
All of those changes at higher altitudes will produce higher BC values.