For your purposes, the NMWA you liked is a pretty solid choice--that'll be your loud pattern. I really like brown as an odd jacket, and generally in winter weights with texture (flannel/tweed), but since you want a bit more year round versatility, maybe a brown pick and pick. I do think linen is wonderful, so for the last, linen navy or linen in a lighter blue (since you already have two blues.
As far as shirts, 2-3 white shirts with hefty spread collars (my preference, but it depends on your face shape and size...as long as the collar points fall beneath your lapels. A blue OCBD and maybe a white. Two solid light blues, one with a super small scale pattern for visual interest. Maybe two striped.
Pants, definitely mid gray flannel. Two other grays in medium weight, perhaps twill from Howard Yount. One or two browns (one flannel if two). One summer weight light gray.