I have a pair if cavalry twill on the way.
Should I just open a open a suiting book and pick something with a bit of variegation in the color?
You could try a grey tweed with some surface itnerest. Have a look at the Sherry Tweed book; it's primarily a jacketing book I believe, but since you have lots of pairs of trousers already, I don't think you'll have any wear issues. Towards the end of the bunch there are some interesting flecked/mottled harder-wearing (at least, from the feel) donegal-ish cloths. There are several different greys there, all of which are interesting. I have a pair of blue-grey trousers from that section (wore them yesterday, in fact). Only got them last year, so can't comment on wear. I really like the Sherry Tweed book; lots of fun stuff, and all at a versatile 11oz (give or take) weight.
I was just thumbing through the old edition the other day (found an ideal fabric for the suit I want; sadly no longer in production) and planning on having a look through the newer bunch (from last year's collection, I think) soon.
EDIT: google has found a kind tailor who has uploaded all the Sherry Tweed swatches to his site: http://deoost.nl/hollandandsherry-fabrics-stoffen/sherry-tweed-hs1188/
Do look at it in person (and talk to your tailor) before using it for trousering. If you didn't already have plenty of other options (meaning they won't suffer from frequent/excessive wear) I probably wouldn't have mentioned this book.