I think you can wear shantung in winter, but you need more rustic fabrics to go with it. The smooth worsted that Noodles is wearing doesn't work well with it. You can also rock wool in the summer if the color is summery and the weave isn't too chunky. GMMcL's tie would work with a fresco suit.
Of course, the real question is why anyone would want to wear shantung at all...
Agreed (to the first paragraph, anyway).
What's your opinion on flannel trousers worn with a smooth worsted blazer?