That shouldn't happen on a properly cut jacket. Everything would scale accordingly.
How grey is that jacket? As a blue jacket, it's perfect fine. But grey? With khakis? That's no good. Anyway, no matter the exact color, I would not wear this jacket with khakis on the grounds that the finish is far too refined for pants so rough and casual.
For buttons, go with brown horn, not silver or MOP. Far more versatile and unrestricted by season.
It's right down the middle between gray and blue -- I'd say the picture is pretty accurate as far as the color goes, though I had to mess with it on the computer because the light in this room is a mixture of tungsten and flourescent.
Are you categorically opposed to gray jackets with khaki trousers? Even gray tweed? I can see where you're coming from in terms of the textures, though it doesn't bug me much.
Would brown horn be enough contrast? My concern is that the jacket lacks enough detailing to separate it from an orphaned suit jacket -- and I'd like to make it work, since I don't have much to spend and this fits well.
Thanks very much -- I've been defaulting to cream silk when I don't know what square to wear, thanks (I think) to a post of yours in the good taste thread.