I have some crossover in my wardrobe too. I think that's fine, but I hear ya on the collars. Some of mine look great with a tie or buttoned up, but then worn casually get a little unruly. I like slimmer collars for casual wear and even button collars.
Are you using collar stays to keep them weighed down?