I have a pair of lime green trousers and a pair of Kelly green shorts. With both I wear white OCBDs. I'll tie a navy or cream colored sweater around my shoulders if the weather calls for it. I'll wear a blue blazer and simple tie with the pants if it's a more formal occasion. Otherwise keep it simple. Yellow and pink will make it look like you're trying too hard. Orange and green makes me think of Autumn.
The whole point to wearing trousers with some pop is to make them stand out. Keep everything around them simple and you'll make way more of an impact than if you have competing stand-out colors.