Hey all. I am 5'5" and slim with a fit and semi-athletic build. I usually wear a size 29 or 30 waist trousers, depending on the brand and I like my clothes to look tailored and slim but not skin tight.
I've been experimenting a bit with casual shirts and oxfords and have had good results with the following:
J Crew Slim Fit (XS) - I believe these are designed for someone who is around 5’8” so they required some tailoring - sleeve length and shirt length. They fit very well after that. These are probably the slimmest of all of the shirts listed, so watch for this. The arm holes are higher than typical shirts so they work well for a short guy. The collar height on most of these shirts is sort of a medium height and seems to work well for me. A couple of others had a higher collar that I don’t like as much. My only beef with the J Crew shirts is that the button spacing down the front of the shirt is proportioned for a taller guy so, after shortening the shirt a bit, the bottom button lands closer to the bottom of the shirt. Though, this isn't a problem for shirts that I tuck in. www.jcrew.com
Peter Manning Slim (Size 2) - Bought online. I can’t imagine it getting much better than this, right off the bat, with the exception of Jimmy Au. These are almost perfect in length and overall fit but a slightly looser fit than what I prefer so I took mine in for some very minor tailoring around the torso to make them more slim. The collar height is a tad bit larger than the J Crew shirts which is okay. The spacing of the buttons down the front is perfect for a guy my height and just looks right. Arm holes, pocket placement, etc all great. Highly recommended. www.petermanningnyc.com
Brooks Brothers Black Fleece (XS) - Bought online. Similar to J Crew but baggier in the sleeves and torso. A couple of these were great after some minor tailoring (overall length, sleeve length and sleeve width) but others took quite a bit of darting in the back area to remove some billowy sack-like bagginess. This might be okay for someone who likes a looser fit but I had to spend quite a bit of money getting these to look as good as the other brands, which required much less tailoring. Collar height is higher than the J Crew and Peter Manning... probably the highest that I would ever want to go. The button spacing isn’t ideal for a short guy - once I shorten the length of these, the last button looks too close to the bottom of the shirt. When I’m not tucking in, I will usually leave the bottom button unbuttoned so it looks more natural. www.brooksbrothers.com
Jimmy Au's (XS) - Bought in-store. The closest to being perfect right off the rack. Very similar to Peter Manning in that the proportions are all designed with a short guy in mind. The button spacing, arm holes, and pocket placement are all great. Collar height seems perfectly proportional for me. They did some minor tailoring for me right on the spot - just a little bit of shortening and a tiny bit of slimming around the torso. These are probably my best fitting casual shirts at the moment. Again, highly recommended. www.jimmyaus.com
Hugh and Crye (Short Skinny) - Bought online. Out of the box, these probably would fit a slim 5’7” guy perfectly. The fit falls somewhere between the J Crew XS and the Black Fleece shirts. I needed to get the shirt length and sleeve length shortened a bit. The shoulder and torso fit is great. The collar height falls somewhere between Black Fleece and Jimmy Au - totally fine. Arm holes are in a good spot for a short guy and the button spacing isn’t perfect but, after some tailoring, the shirts look more natural than the Black Fleece and J Crew shirts. www.hughandcrye.com
Salt Valley - Urban Outfitters (XS) - Like the J Crew shirts, the Salt Valley shirts fit well and look very good after a tiny bit of tailoring on the sleeve length and overall length. I've taken some out of the sides on a few of these too. Arm holes, button spacing, pocket placement, collar height, etc are all very good. In fact, I think these shirts look more "right" than the J Crew or Brooks Brothers, similar to the Jimmy Au and Peter Manning, but Salt Valley's quality isn't quite there with the other brands I've listed and they offer flannel western shirts as well as patterned spring/summer casual lightweight shirts. I don't think they do basic oxfords, etc. www.urbanoutfitters.com
I hope this is helpful. I'll continue to post my findings.