Thanks for your observations, mimo. Being a long time lurker I know your disdain for all things denim. I like blue jeans so we won't be able to agree on this.
I agree with you on the jacket. While I don't think it should be let out (I still plan to lose my X-mas girth), I think it's too short and the shoulders only fit when I do an unnatural robopose without moving. I will have to let it go soon which saddens me because the fabric is so nice.
Why not just wear it unbuttoned with denim, if you like that look. This might "hide" some of the issues mentioned by Mimo. I think a nice pair of lighter gray flannels or donegal tweed trousers would be great.
Do the divots in the shoulder disappear when you wear it without a sweater? I ask because as I understand the cause of divots (from the tailor's thread), it's putting something too large for the arm hole which causes the distortion and hence divot (I hope that is right). That might help your shoulder fit.
So this would be a sort of experiment: what happens when you remove the sweater (of course, you arm/shoulder might still be too big). I find that when I wear v-necks under some of my close fitting blazers, divots appear.