I think the belt, shoes and 5-pocket style pants bring fit to a level of casualness that no tie will ever look right. Change the shoes and belt and I think the tie could work. Although to your other point, I have decided that for me (given my age, environment, the city I live in, etc) a tie will never look cohesive without a sportcoat at a minimum. YMMV.
So since a little bit of extra fabric doesn't change the price, including fabric swatches shouldn't be a big deal to make it seem like a better value for customers. I have already ordered shorts at full price, so I'm not just cheaping out.
@luxire - if I could make a suggestion regarding shorts. They are currently priced the same as full length trousers which makes it poorer value. Can you consider either offering different pricing for shorts or include a free book of fabric swatches with a shorts order?
I just received a pair of shorts this week. I'll post them when I get a chance, but they turned out excellent. The only mistake was that I asked for no crease to be ironed into them but it had a crease anyway.... oh well. They are this fabric (which is excellent - sort of feels like a summer moleskin):