I used to do that, too, but I eventually realized that I had stopped wearing my high-waisted slacks with belts because they wouldn't stay put. Since they're cut to be worn suspended, those slacks would drop down to my hips no matter how much I'd tighten my belt. I'd spend half the day hitching them up and the other half of the day with a droopy crotch and with my cuffs pooling around my ankles.
A few months ago I surrendered to gravity and removed the belt loops from all my high-waisted slacks and accepted the fact that I'd have to own some slacks for wearing with belts and some for wearing with braces. I figured that since you wear so much bespoke clothing you would do the same.