Well you probably lost a customer with the Value Village jacket. You never know the circumstances. I had one guy come in with some cheap, dirty thrift store jacket and did a similar thing. We got to talking and he just got a massive promotion at work, along with a $3,000 bonus that he wanted to spend on clothes for his new office position. I didn't necessarily critique his jacket out loud (it was too big), but had him try on a few different sizes (off the hanger, no easily visible size tag) in front of a three-way mirror and we found his true size. Obviously I had to tell him that he was a 40L instead of a 44L at some point, but once he saw what a well-fitting jacket looks like on him, it wasn't such a blow.
Other than that story, I don't think you're being too much of a douchebag at all. When a customer is trying to return stuff obviously worn or even destroyed (and outside of the return policy), you have to be defensive and blunt.
Speaking of, I had a woman get mad at me today because I wouldn't exchange her 6-year-old son's urine-stained sheets and comforter set. She didn't even wash them before bagging them up and taking them to the store. Seriously wtf?