It is hard to tell by the pic if it is a Summer or Winter fabric but never the less I would start by getting either charcoal or oxford grey tropical wool or flannel. I would add a classic tan garbadine or tropical. If it is a winter fabric, I would get a nice twill or covert cloth. Cordoroys are a nice transition fabric in a pinwale. Khakis for the dress down look with a dressier polo without logo. And if you have to, a pair of undistressed jeans in a dark wash with a bengal stripe button down shirt.
And do not button both buttons as your pic suggests you might do!