The shirts you describe fit me the same way.
I have a big neck relative to chest/waist and I find the BB ESF to be big and poofy.
J. Crew Slim fit is ok.
Nordstrom's Calibrate is pretty good but now that I look, they don't seem to have no-iron. I thought they had Calibrate "trim fit" & "smart-care" but can't find it on their site.