for the jacket, i'm thinking camel or any medium to dark warm brown.
consider wearing a solid cream or aqua colored shirt instead of olive, or patterned shirts with brighter colors if you're into that sort of thing.
black shorts are acceptable if they complement your coloring, and can be part of a great outfit. for example, add blue suede shoes of some sort (penny loafers, gucci loafers, bucks, etc.) and a pure white linen shirt with the sleeves rolled up.