Traditionally, a flat cap is more informal. But these days I would be comfortable wearing a nice wool one -- from a real hat maker, not one of the cheap ones at Nordstrom -- with a suit and coat. It looks a bit rakish, and since you take it off as soon as you're indoors anyway, there's no harm in it.
That said, I have always wanted a shtremel.
