For serious, I haven't kicked around Downtown, Beacon Hill, North End, or South End too much.
Towne - Not bad usually an older crowd (I mean older than 25), it has a divorcee vibe to me. Food and drinks aren't bad.
Sonsei - One of the be seen and be heard places, sort of an institution for cocktails.
Eastern Standard, Hathorne, and Island Creek Oyster House are all right next to each other. They are all interchangeable with slight variations. I prefer Hawthorne, a little less busy and has less douches. Eastern Standard has a fucking awesome spirits list. If you are into spirits neat this is a great place.
Depending on what you are looking for, and where you are staying, I can be more helpful.