To my untrained eye, it looks like there is excess material in the 50 shoulders, while the 48 sits much better. Also, you might want to consider the 48 because the sleeves will need less work than the 50. Because of the working buttonholes, I don't think a tailor will be able to shorten them all the way up because the sleeve end will sit way too close - if not directly on - the first button.
Others might want to chime in, but my bet is that the 48 might be the way to go!
Thanks, thats a good point about the buttonholes. I'll try and take it to a tailor over the weekend and see how much the sleeves can shorted.
The jacket is the Havana Navy Plain http://eu.suitsupply.com/en_AU/jackets/havana-navy-plain/C757.html?start=1&cgid=Jackets&prefn1=collection&prefv1=Spring%2FSummer
I go to the gym quite regularly so that might play a part in that. I'm thinking this might also be causing the shoulder bump in the 48.