Scratchy: Are you in the habit of wearing an unlined tweed jacket without a shirt? If not, then why would scratchiness be an issue?
Rustic patterns: Those checks can certainly be done in the anachronist's style, but they also go nicely with denim and other more casual fabrics. In that sense, these fabrics may be more "modern" than the sleek suiting we supposedly disdain.
Weight: Do you run hot? I am comfortable in my office wearing my winter-weight jackets.