A few thoughts.
Noone is a lesser person for wearing black. It goes with everything and vice-versa (in theory at least). Most of all, it goes with other black things. It can look good, and the suggestion that this is only on women is a bit silly I think.
However, it should not be worn with a tie, for the very important reason that one's tie should always be darker than one's shirt, I feel. And this is not possible.
As to its casual-wear applications, look to Ralph Lauren of late. Much of it may not be to individual members' tastes, but it does not look abhorrent.
Finally, I agree that colouring is an issue. I have fair hair and very pale skin so it can look quite nice. Were I otherwise, I think less so.