I think the belt, shoes and 5-pocket style pants bring fit to a level of casualness that no tie will ever look right. Change the shoes and belt and I think the tie could work. Although to your other point, I have decided that for me (given my age, environment, the city I live in, etc) a tie will never look cohesive without a sportcoat at a minimum. YMMV.
So since a little bit of extra fabric doesn't change the price, including fabric swatches shouldn't be a big deal to make it seem like a better value for customers. I have already ordered shorts at full price, so I'm not just cheaping out.
@luxire - if I could make a suggestion regarding shorts. They are currently priced the same as full length trousers which makes it poorer value. Can you consider either offering different pricing for shorts or include a free book of fabric swatches with a shorts order?