I think another reason why people might get mad is cuz they probably are like, WTF THIS GUY THINKS IM GAAAAAY ARRRRGGGGHH
See, that's a dumb rationale. It's no diffferent than a fat, ugly girl checking you out. She knows you're not attracted to her, but she's still going to look. I remember this guy I used to work with who would freak out if a gay guy checked him out, then he'd rush home and start working out obsessively, trying to bulk up. I'm like,"right, because there are no big, muscular gay guys out there!"