Fit is an interesting subject. Because it involves so many groups of thought. It can be, and sometimes needs to be, divided up into two or more groups. These groups are certainly intermingled, which adds to the confusion. Inlays are used to adjust any of these fits. Armholes need to be in there correct place to be comfortable and functional. Change the style and adjustments need to to be made, or it doesn't fit. Personality also plays a role of better fit. That is three. How many more reasons are there that deals with the armhole fit?
As far as tailors touching sometimes they can feel if the garment isn't fitting. Where as, customers new to a higher standard of clothing wouldn't know, and think this is strange with crazy thouhts rushing through their mind. By touching a tailor can tell how tight or loose the cloth is, and which direction the stress on the cloth is coming from. So, these are a few legitimate reasons. How many more reasons are there? And, not saying that some tailors touch for other reasons, because, well, there's always some of them.