The shirt can help with that. I'd consider a linen-cotton blend or a cotton voile. They're both dressy shirtings, but run cooler than standard cotton weaves like poplin or pinpoint. And in the case of a 50-50 linen-cotton, it'll wick sweat away too.
If you're going to do a vest, linen is a good idea.