Im kinda in the opposite side of that, I also like to balance my closet but nowadays Im semi-retired and just do some minimal work from home so I rarely go out of the house these days, and I do mean rarely, iirc last time I was out longer than 1hr was over a week ago, forealz.
So I tend to just lounge around the house most of the time in sweatpants or ogs, birkenstock sandals, tshirt & cardigan, but as soon as I have to go out I tend to want to compensate and do a polar opposite fit, so even if Im just going out for whatevs to a store or something I'm in some type of blazer fit, and the rare occasion now when I have a work meeting or something work related I'm in a suit by default, not necessarily a full cm suit+tie fit everytime but at the least I do a casual suit fit
In echoing with the above posts...
I work a lot... I tend to buy things that I can wear there and still show off a bit of style. I do research in surgery so I could wear scrubs every day, but I really like to put a clothing wall between watching cases in the OR and being at my desk. I really hate putzing around in scrubs all day.
I can get away with Dark wash jeans, so I've been wearing a pair lately. Otherwise, I try buying things that I can wear at the office and still wear out of the house when I go out on the weekends.
I like some SWD stuff, but I don't see the point of throwing a lot of money at something I can wear for only 2 days a week, if even...
Man, I'm in a similar situation. I work online from home and I only go out on the weekend. I'm currently rebuilding my wardrobe and there are so many pieces that I want but I'm on the fence about it because I know I'd only wear them no more than a few times a year. Maybe I should just be like Hirsh and gain satisfaction by just owning things and wearing them at home.