The two biggest changes to ballistics is temperature and elevation. Elevation determines the distance that you can shoot carrying enough energy. The higher you go, the farther you can shoot.
Temperature effects ballistics a little more in my opinion. Depending on the powder you can see 100+fps variable (more prone in ball powders) that will definitely change your original zero. Keeping them warm will help with keeping them as close to a summer time ballistics. Just make sure you squeeze it off as quick as you can. There are some powders that are a little less effected by the temps. A lot of the Hodgdon extreme powders (Varget, H series, IMR4064) are suppose to be able to handle temperature change. I know that Snipers shooting M118LR expect 2700fps in the summer and 2650 in the winter. They have this stuff logged and can adjust accordingly.
So to wrap this up, if you want to store them in your pocket, its not going to hurt anything. Make sure it is an internal pocket, the only problem I see, is that if you have to make a quick shot you will need to load a round and by that time you may have missed an opportunity. My suggestion would be to get out and see what it is really doing in the winter months and record your results to adjust accordingly.
Tank