Manton also called dimples gay. But it was circa 2007 when I suppose you could get away with that. http://www.styleforum.net/t/33066/conservative-business-dress/
(OT - he was a flaming asshole in this thread. Some hilarious stuff for we fashion nerds).
I'm with whoever said it is more work to NOT have a dimple. I think Manton double crossed himself on this one.
So if there was ever a rule to break, I'll break this one 99% of the time, cause I think it looks better. But to each his own.