I have a navy Dolce suit with a lightish grayish chalk stripe, and was thinking about pairing it with a pair of gray slacks. Right now, the pants on this suit have gotten too big (I lost a bit of weight since I got it), and I have to have my tailor fix them. The jacket is still perfect though. Is it wrong to match a suit jacket with other pants once (no extra "wear", and a more casual shirt for casual Friday? Or does this always look wrong?