Wear the cummerbund or vest. Its function is to prevent the white of the shirt from peaking out below the jacket buttoning point, as well as hiding the pant waist if the jacket is unbuttoned. It prevents cutting the body in half visually, which helps the wearer look taller. If you don't want to wear a cummerbund, you can always go with a double-breasted jacket.
Thank you for the replies. I wanted to go for a less traditional black tie look (if that's not an oxymoron). I think any color cummerbund other than black makes the wearer look like a waiter. As the bow tie typically matches the cummerbund, though, this would limit me to a black tie. Would a black cummerbund with a black and silver paisley bow tie be alright, even though it wouldn't match the cummerbund, strictly speaking?
Please avoid trying to do "less traditional black tie." This usually ends up terrible, with the wearer looking like he is going to the prom. I wore a silver paisley cummerbund and bow tie to my high school prom, BTW. So few people actually do black tie properly that, if you do, you will end up looking distinctive.