This kind of thing is pretty common in more 'rugged' casual suit combinations. JLibourel is no doubt refering to standard CBD-type suits when he says that these don't really go - in that combination I would agree that you'd be better off with a more conventional type of business shoe. It's not that it would be completely beyond the pale, but just other shoes would be much better. But with a something a bit rougher and tougher (tweeds, twills, or other kinds of more casual fabrics and cuts), I don't see any problem.
Better to have a dressy shoe with jeans than a clunky shoe with a suit. I'll wear brown half-brogues with pretty much anything.