I always wear double cuffs with a jacket so they are staying as are the shoes so that leaves the suit itself.
As regards colour for the suit I don't like blue with brown and brown is too matchy so maybe a lighter grey would be more suited?
I also thought tweed and brogueing were a good match?
Tweed is a casual material and you have three formal things going on here -- French cuff shirt, oxfords, and the color charcoal. If you want to keep the formal elements, then you'd be better off with a more formal suiting, but then prob have to ask where are you going to wear this formal suit and its formal accompaniments.