Thats my thought. The fuller pants don't look wrong in an of themselves, but paired with a relatively slim jacket and sleek shoes emphasizes their fullness.
F*ck that. Armani made an entire design career out of fuller pants with a fitted jacket. It also happened to make Richard Gere's career. Greg, for you it works and you look damn good. Rock on man. And don't slim it down.