jarhead, perked my curiosity, so I ran out to my shop and pulled a manual off the shelf, Speer #11. On page 472, under the heading 30'06, it lists a bunch of manufacturers that use barrels with a 1 in 10" twist, including the Ruger #1 and Model 77. In fact, only Husqvarna and the Browning Model 78 use a 1 in 12" twist. Of course, that was in 1986, some things may have changed, since then....but not your Model 77 built in 1978.
Additionally, these gentlemen are correct about requiring a slightly faster twist, with heavier bullets, or as mentioned, increasing the velocity also increases the RPM and the stability for those bullets running on the ragged edge, accuracy wise.
Also, and I'm sure the previous respondents are aware, although you may not be, that the actual problem is the length of the bullet, rather than the weight. Since most jacketed bullets have a lead core, it amounts to the same thing, excepting solid copper bullets and design features like a boat tail or the VLD ogives. Those bullets may weigh the same as a shorter bullet, however, they require a faster twist, again, for different reasons like a shorter bearing surface relative to the rifling.
Basically, your rifle should shoot Factory 180 grain bullets every bit as accurately as the 165s. If you are happy with the 165 weight, look no further, but there is probably something else going on, as to why your groups were worse, with the 180s. Because you do not handload, you can't try different seating deapths, which is the first thing I'd do? You might check the pressure on the forearm, there is a remote possibility that the harmonics are screwed up when you are shooting the heavier bullets? Next best solution is to try different brands, or get into handloading; which is the smart thing to do, regardless.
Good luck, LB