I think I'd have to second Timber338. The safest bet is, of course, to camp far away. However, I've had several similar hunt camp experiences. We've hunted one area here in CO for almost 30 years and seen and killed many elk within 3/4 mile of our tent and vehicles. My last three bulls fit that profile as well. I know where to find them and don't feel the need to spend the extra hour "commuting" - but I camp pretty quietly, and I'm probably closer to 3/4 mile from them vs. 200 yards. As Timber338 said, they seem to measure security more by cover than distance, and if their "safe place" still feels safe. The surest way to get them to move is find their bedding area and kick them out of there repeatedly (if they return after the 1st).
Hard to say if you will mess up the hunting - if that is a definite travel route and your camp makes them feel more exposed, they will at least probably go more nocturnal on you. Some depends on how you camp also - if it sounds like a rock star party, you probably should allow more space. Although, there were a few years when some newbies made camp a lot louder than usual, and we still saw elk - they had lots of cover and just had to give the camp a few hundred yards berth. In many places they are used to chainsaws and people cutting wood, etc..
When bowhunting in AZ for several years, I basically camped in the middle of them. They still had their "safe places" undisturbed, but they fed and bugled all around us, within 20 yards in the night. My wife tagged along just to camp and thought it was the coolest thing ever. All I had to do was sneak out of my bag and I was stalking elk within 70 yards of the tent come morning. I even came back to camp around 9 am once, where my wife and another couple were just camping and busy talking and making breakfast and was shocked to see elk feeding less than 70 yards away from them on the other side of a grove of small pines. Go figure.
Again, more space is probably better, but it's not always the deal-breaker you think it would be in the right situation. In fact, at times, semi-close is better. 1) You don't make a lot of noise/commotion getting into and out of the area, 2) you get to sleep in
, and 3) sometimes, especially when "claiming a spot" on the mountain with vehicles and tents etc. it can discourage other hunters from hunting as much nearby or, if you aren't there, from them moving in and camping right there where you want to hunt (so it can be the lesser of two evils at times).
If you get real lucky, you can get one of those reclining camp chairs from Cabelas and sit in camp quietly drinking your coffee and shoot something from your easy chair