It's not a question of dirtiness, really (though it really is.) Why take a shower and then put something against your skin that has all of yesterday's dirtiness on it? A dress shirt should be crisp and fresh. Dress shirts start to look "worn" by the evening, let alone a second day of wearing. When I put on a shirt in the morning, I want it to feel fresh and crisp and completely un-wrinkled and unworn, not smelling slightly like yesterday's cologne or looking slightly droopy and less sharp than a clean, ironed one.