You're mixing up two things here: the width of your foot, especially the forefoot, and the height of your instep (the top of your foot, where the laces go).
As the 9.5E is fitting you OK across the toes, your issue isn't a "wide foot" (at least, not "crazy wide"). It seems to me like this is your proper size. If in doubt, print out AE's size guide - I found it very accurate for size and width. Like you, I find some flexibility in trading off width vs length e.g. I am basically a 10E in Allen Edmonds, but a 10.5D can work too. In your case, I suspect you might be a 9EE/EEE and you're just about getting away with the 9.5E accommodating your forefoot.
Anyway, whether 9.5E or 9EE is really your size, I can see why you've also been getting 3E sizes or sizing up: with a derby (blucher), or something similarly unstructured in the upper, like a monk or even an adelaide, the bigger shoe has allowed the upper to stretch over your high instep. But what you have here is an oxford, with lots of fixed seams in the upper, and even if you get a wider or longer fitting, it still won't allow that extra leather to stretch up over the top of your foot. I actually have two pairs of shoes, both in wide fittings, in different sizes on the same last from the same maker: the first was a derby, the second I had to size up because it was an oxford and the bottom of the facing was too tight over my foot.
In general, I also find the lasts from AE to be shallow in the instep in general. Not a problem to the degree you have, but noticeable compared to most English makers. It does vary a bit across the different lasts so it's worth going into an AE store and learning if there's a last that fits you (I have several pairs, but only on two different lasts so I'm not the person to advise). Getting super-wide or bigger shoes will eventually accommodate the high instep, but end up fitting badly across your toes and probably loose in the heel too. It's not the answer.
I suspect that you have two options: either try a different maker that does shoes with higher insteps - there are plenty of English, Italian, Spanish and other makers in that price range. Or if it must be AE, you might be more comfortable sticking with bluchers, even though they're less formal. The "open" lacing allows the tongue/vamp to move up and accommodate your instep, and even though the sides of the facing will still be a little further apart, they will be straight and it won't look so uncomfortable and wrong.