Try to find shallower v-necks. You can still wear them with a buttonup shirt and optionally a tie, but assuming you're in reasonable shape, i.e. your chest isn't flat to sunken (I'm borderline here IMO) you can also wear them with nothing showing underneath.
I also really like my henley/shawl-ish collar sweater, and 1/4 to full zip turtlenecks. I find they look good with a short collared polo shirt popped underneath. I don't mind crewnecks - have a few of those. I hate mocknecks, the kind that only come halfway up the neck - they look very sportscastery to me and are usually loose fitting, yuck. Zip hoodie sweaters would be great, but non-zip ones are far too warm for me.
And polo collar sweaters have always been good. I had a Polo RL polo collar long sleeve sweater that I wore until it fell apart. Lambswool, I would just let the collar do whatever, button it to the top and pop it on cold days. I ended up cutting thumb holes in the sleeves by the time it was on its way out.
I'm not a fan of cardigans for me, but they can look good IMO. I saw a dude wearing a tracksuit or something and thought it was this, but it wasn't - anyway I thought it would look really, really cool (and I would wear it) to have a thin cashmere or lambswool black cardigan with a slim shawl collar with white edging the pockets, cuffs, hems and lapels. If a designer wants to do this, just send me one free please. If you do it with a zipper, use a double zip so I can open the bottom a bit to avoid the gut look. Thx.