I don't see why spread looks bad in a casual context. You're just saying that because you relate to spread as formal but that's not aesthetically true I think. Something like black being formal or at least striking is connotations as well as the nature of the color. Plain fabrics being formal and texture being more casual has good reasons. Spread? I don't see why it's more formal than a point collar for instance at being worn tieless.