This style does not work across the board for all your needs. It looks more casual than dressy. You need to focus on how it is worn most and pick a style and color.
1) Dressed up with suit and tie: lose the patch pockets and use charcoal or navy. Navy is dressier. Camel doesn't look right at night over a suit to me. It's been done but I don't like it. Camel will never be as dressy as charcoal or navy with a suit.
Make up the coat as is and wear it with 2 & 3. You could pick a neutral oatmeal/taupe color or medium brown. Charcoal if you need the multi purpose color, it will work with the casual usage better than navy but don't get patch pockets. Get flaps, no patch. Navy is too dressy to wear with options 2 & 3.
2) Dressed down with jeans and polo
3) Sport coat (with or without tie) and slacks in a "professorly" stye (earth tones,tweed, checks, etc)
If you change from patch pockets you could add turn back cuffs on the sleeve and or a pleated back with sewn down or loose belt for extra detail. This coat has an inverted pleat on the back and a kick pleat vent.