This is a good point and I agree, but I was more responding to the "comfort" part of the argument:
"It'd make it a lot easier to get dressed in the morning, and I think it'd finally get me feeling really comfortable in my clothes"
Getting dressed in the morning is pretty much knowing which parts of your wardrobe mesh together. Most of us have specific elements that make up outfits and wouldn't work with certain other pieces. When you understand how each item works - through learned intuition or trial and error - you become comfortable in your clothes.
wire.artist aka yoda of yohji doesn't even look in mirror just picks the ensemble and heads out the door
i will one day achieve that level of zen