I am one of those people, but mainly because I prefer having a nice clean first and foremost. One thing to consider is if your amp can be pushed into overdrive with the volume maxed and the tubes cooking. You can get a nice distortion tone if you use your distortion/overdrive pedal as more of a volume boost with the distortion on the low side and volume maxed (with your amp's volume set on the edge of clean) to push the amp into natural overdrive.
Some days I would love to be in a band where I could strip my rig down to a 335-type guitar, tube screamer, and a vintage Deluxe Reverb. That is tone heaven!! But in my current situation, for both volume and versatility reasons, I play through a 4 channel amp with both pedals and rack effects.
Depends....if you play more clean or more dirty! I don't think you should have to to compromise on either.