I'd sell it and get a .243.
But .30-06 is just such a cool round
How about a compromise? .25-06 or .270 win Which in case you didn't know, are the .30-06's younger siblings, which more or less, use the .30-06 case necked down to .25 cal and .277 cal, respectively.
Generally, they have less recoil and are flatter shooting than the .30-06 and they still have plenty of punch. The .270 can take the same game as the .30-06 with the right load, and they are pretty much equal on power. The .30-06 has heavier bullets, but you said yourself you wont be reloading, not that you'd need heavy bullets for the game you intend on hunting.
130 grain bullets in the .270 is more than enough for deer and hogs. You will likely get nice exit wounds, depending on bullet construction, you choose. If you don't think 130's are enough (trust me they're more than enough)
You've got 140's and 150's in factory ammo. The 150 grain bullets in .277 cal have comparable sectional densities to 185 grain bullets in .308 cal. Not that you'd need anything that heavy. I've seen ethical, DRT shots on Elk with 130's from a .270 win.