That goes back to the whole idea behind "satorial rules." They're shorthand for when people either (a) Have lacking taste in clothing, or (b) Simply haven't had a chance for the trial and error necessary to dress better. In both cases, we want to save them from an obvious mistake, and there isn't time to explain why it's not a good idea (or we want to end a debate prematurely). Often the rules are foolish (e.g. "No Brown in Town") but look into why they were first laid out and it makes a little more sense (e.g. I can see a father telling his young son, "Brown shoes require a little bit more thought as far as matching to trousers, so if you're just wearing navy and gray in town to your appointment in an hour just wear black shoes now, we don't have time to discuss the theory"). They're simplifications, and very few simplifications tell the whole truth. But if you outline the why, there's usually a helpful truth in there
I always assumed "no brown in town" referred to brown cloth, not brown leather.
i.e. no tweed or otherwise country clothes in the city