"That's it"? That's a really, really important one.
You can wear mid gray trousers every day and it won't be noticed. They're not particularly loud, and certainly not loud enough where they'll stand out if you wear them several days in a row.
Medium gray is a conservative color, and THE default odd trouser for most men. If they own nothing else, they own a blue blazer and gray trousers. We're not talking a neon green with red checks here, though I'm sure somebody on this forum owns a pair of those.