I'm going to jump in in defense of having a lot of clothes, especially if they don't all "go well" together.
Think of it like having a bunch of records I guess, or even just a bunch of music in general. You have the workhorse stuff that you're happy with wearing twice a week for a few years. It works in most weather, most seasons, and it's not only what you're most comfortable in but it most likely fits some sort of expectations of your persona even. This varies for everyone but it might be something like your jeans, boots, gats, your ToJ or 5-zip, whatever. It brings you a certain happiness and peace of mind, but mostly it just works year round with any mood, and it will always be enjoyable for those few times a week you bust it out. For me that might be like a Neil Young album or Boards of Canada or something. Can go over a decade playing the same stuff and it's still a-ok.
But then you have some other things that you barely wear all year, but are definitely worth just hanging onto because when the time is right, nothing beats them. It could be something like a Beach Boys album. No way you'd want to listen to that regularly, but when everything syncs up and the time is right it's impossible to beat. It's not something uncommon or even close to a #grail or whatever, and it might be easy to just want to get rid of because you barely use it, but when it all comes together there is nothing better. For us SWD guys this might be a good suit, or it might be Regis' 5-zip he wears for a week a year, or it might be a raincoat. It might be a ratty old graphic tee that you wear to one show a year or one party a year.
Lastly, you've got that thing you really struggled to find, now find yourself struggling to wear. These are the things that fill up our closets and make us wonder weekly why we don't just spend the effort putting them up on B&S so that somebody else can enjoy them. But there was a reason you bought it at one point, and there is a reason you haven't actually sold it (and it's not laziness). It's because when the sun and the moon and the stars all align, and the weather is just right and you've got just the right occasion and inspiration strikes with you what to pair it with and you do, you feel like you can take on the world (and then subsequently take a fit pic and post it everywhere instead). It might feel more "challenging" to you but that's only because you're letting it get the best of you. This might be something like a certain Venetian Snares record for me, where I'm so happy to own it, even if I only play it 3 times a year these days. It's just that those 3 times, when I'm all alone and in just the right mood that I can enjoy it as much as I did on the first few listens. It won't get played when friends are over, or my parents, or really around anyone but my girlfriend, but it's still very special to me even if it collects dust most of the time.
It's my opinion that you'll start having the most fun with clothes when you make these ones your workhorses instead, and I have a feeling that's definitely what happened with a lot of people that have gone deeper and deeper into more esoteric fashions. This could be like Shah's polka dot pajama robe, or something like that. (Which I'm still mildly annoyed he sold, for these reasons entirely).
You might think I'll say something like the Tim Coppens rug jacket for me, but that thing gets a ton of casual wear and it's more like that boiled wool baggy Sage de Cret suit that I've only gotten a few wears out of. My excuse is that this is entirely because of weather reasons and not for lack of trying :-P
So buy more clothes, and wear more clothes! Wear your rare grails as you would beaters and get some life into them!
Warning: Spoiler! (Click to show)