I thought OP's predicament was pretty clear. So far the one place where he has found sells clothes that fit him is American Eagle. He wants to do better and naturally the step up from American Eagle IS the likes of J Crew and Brooks Brothers, against which I, however, had to warn him because they don't exactly have his size. Also, not trying to be snarky but I thought it's funny you dismissed Brooks Brothers and J Crew from the grand scheme of things and then came right back to Uniqlo :)
I have a friend who's around OP's size and I know how hard it is for him to find clothes. We dug up the whole mall and went from American Eagle to Club Monaco, leaving empty-handed but for a few items he found at AE. If the OP does live on the East Coast, it's worth it to pay Uniqlo a trip. Otherwise he will just have to make do with what he can incidentally find or budget for tailoring.
The reason why that makes sense is because BB and Jcrew is absolute garbage for the price. I know people look at the retail price and the marketing and they feel like they stepped up to something nice, but it's not. The only time it's remotely buyable is at 40-60% off and even then it's rarely worth it.
Uniqlo can be brought up because of the value proposition. The stuff gives incredible value. Is the stuff high end? Hell no. But is it worth it for the price? Yes. Especially if it fits you and it likely will if you are shorter and slimmer.