They're limited. Plain gray would be better, because then if a patterned jacket made its way into your wardrobe, they would work. Those would be okay with a thin sweater, no tie, or with a navy blazer. You couldn't wear them with tweed, or a patterned jacket.
I second Mimo's advice. RTC has a good point, but navy trousers are tough with a lot of jackets.