I am actually. But I like that big B&W plaid for a summer jacket. I actually have something like it in half linen, half silk. The slubs save it from looking cheesy and the linen saves it from wearing too hot.
Just do fresco. Closest thing you'll get to looking like flannel. The lightest or second lightest gray Minnis are perfect. Slightly varigated color and will wear reasonably cool. Be sure to specify unlined.