What about viewing hoodies as something other than outerwear? For example, there are a few hoodies out there which resemble sweaters more than gym wear. Many are even made out of wool or lightweight cotton knits. I love the concept of layering using a cardigan, but rarely wear cardigans since they frequently look too fussy and old man like. But a nice knitted hoodie can serve roughly the same purpose without it feeling like you're wearing your outerwear indoors.