I'm not an eBay buyer/seller, so my thoughts come with that caveat. But I fail to see a huge problem here. Did he take your money without delivering the product? Did he sell something that was inaccurately represented?
My advice: leave negative feedback and be done with it.
I really have to agree. This obviously sucks, but it's really not that bad.
The seller is wrong, having said that he can refund your money and take his chances with ebay if he wants. Why not split the difference with him if you really want the scarf...maybe not the right thing to do since he should have known better, but if you leave him negative feedback get ready for him to do the same, and that is not worth it IMO.