Uch, makes me want to do a book. But it's too hot to do what I'd do for a Murakami novel (which would be to bring TheChair into the middle of a field--Texas has plenty of those--and take my picture there, trying to capture a sense of isolation). There are some good Japanese novels out there that really play to the sense of isolation felt by many salarymen. If someone had a black suit, a salaryman costume would be awesome.
Nobody is out to get you or is holding you to any double standard. Promise.
The button pulling that is on almost all your fits is much more extreme than the button pulling on SML's jacket here. The jacket also doesn't appear too tight elsewhere either. Since SML regularly posts jackets which show no evidence of being too tight, we give him the benefit of the doubt: the jacket fits and it's just the pulling you get do to the way you stand.
An inductive approach is really the best way. My process is the same as yours. I suspect that's true for most of us. It may be a form of post hoc rationalization, but given how deeply buried our preferences our, when something (an item or combination or anything else) is continuously received, it's a good way to develop guidelines.