I have the same thing! And it "migrates" around about an inch area too. I just go with it and make a joke out of it if anyone notices. I think things like that are much better if they're not made into a big deal.
If you can't find them in your specific length I'd say buy medium and get them altered. The alterations aren't hard either, if you want to invest in a sewing machine you can do it in about a minute to every shirt you own.