Fabrics that are soft but not too soft and substantial but not too substantial. Especially thick, stiff or hard fabrics make for a bad button down collar. I think in the USA a lot of times when men want a shirt to wear without a tie, they get a button down. Ive seen them work with Madras, Seersucker, linen, Bamboo/linen, poplin, all weights of oxford, even those brushed cotton flannels and viyella wool/cotton mixes, maybe even some fine twills. I would imagine that once you get into 160s cotton or very heavy twills they start to be a waste of time, even making doing up the buttons a problem.