Funny enough when I wrote this I said to myself, "gus will post a pic of him wearing this to disprove me".
I think some people (like you) pull this off, although I think you wear sportcoats and jeans sans tie, but when you add a tie to it, you need to be Andy Warhol (a well known, kooky artist) our Ralph Lauren ( a well known designer with some kooky personal outfits). When you are simply Claghorn in South Korea, you look bad.
Growing up in the NYC area in the 60's and being influenced by my friends older brothers coming home from prep schools and Ivy league colleges, we did the combination of jeans, tweed, corduroy or blue blazers and tie. It was during the transition from a more formal society dress code to the late 60's "anything goes". We still had to wear a tie to dances, etc., but we could wear jeans now for the first time. It was comfortable yet you were also part of 'the club" so to speak. In the late 70's when I used to run into Andy Warhol at the Museum of American Folk Art events. We were about the only two guys wearing BDOC, blue blazers, rep ties under 70 years old. However, we were the only two guys wearing Levis.
I still like the look today, with a tie. Especially knit silk or cashmere. Like a lot of great sportswear, I think it has a true all-American pedigree.