I'll chime in to say that I get more returns on shoes than anything else. Other stuff can be altered, not so for shoes.
out of about 60 pair sold in 4th quarter of last year, I got 5 back.
Also as you point out, a lot of guys don't know their size. I had a guy return a pair of shoes because I showed the inside length at 11" and his foot measured a hair under 11". He didn't realize that you have to add a half inch for toe room.
I advertise a 15% restocking fee, but only charge it if I paid for original shipping or feel the person is being a dick.