I like knits, so I have no desire to discourage folks from wearing them. But I question the sense of obligation. Once you're in denim, you've made the call on formality. In that case, I'd say, add a knit if you think it adds to the effect, but not because of what you're doing. Here, I'd like the laid-back vibe that would accompany an open collar.