Mr. Pollock, if I remember correctly, you are a size 10. If you need unload some JM handmades or other assortment of shoes, I am around.
Slightly more serious, I think what you are doing is fine -- just wear your things in places that is not work or court. Who cares that only 5% of people dress up for Broadway these days? You are still not alone. I also find that the best restaurants still command higher level of dress. A sunday dinner at Bouley showed 85% of the men in jackets. Also, if you'd like, you can do the charity circuit and wear nice clothes at the various dinners and benefits or take on other hobbies that enable you to dress well.
Aside from all that, what's wrong with not wearing your various clothing more than once or twice? Sure, it's an existential crisis for the clothes themselves (their made-purpose not being fulfilled), but that should not be a problem for you. You obviously like the variety and the ability to pick from a large permutation of things on a whim. Sure it may be excessive, but so what? Hobbies are about excesses, really. If you empirically enjoy having the variety of items you own, you shouldn't let society's norms get you down that much.