It's definitely both. No you don't need to spent 10, 8, or even 5 grand necessarily to get an accurate rig, but you do have to start off with quality hardware or it's going to get very aggravating.
Many years ago a friend of mine told me something that still makes good sense. "Be willing to spend at least as much on quality glass as you are a quality rifle".
There's some wisdom there.
More often than not you can buy any of the major branded rifles off the rack, and find one that with some tuning is capable of 1 MOA performance and do so reasonably cost wise.
However if you spend a thousand on a rifle and having it accurized then stick a 200.00 piece of Chinese crap on top of it, you will never reach the potential accuracy that he rifle itself is capable of.
Read through this thread.
For all you people thinking of getting started but are intimidated by the dollars...
You can get a good read on what a lot of us poor boys have done to put nice budget rigs together.
Be willing to shop the classified section here for both rifles and glass as you will find some very good buys.
Ebay is a good choice as well for quality used glass.