I have similar experience. I worked for 18 months in an office job in which my father was the VP of the company. The owner was a no show kind of guy, so my father ran the day to days. I got hired against my father's will, because he was afraid of accusations of nepotism. The way that I got past it was by working my ass off day after day after day. I was generally the first person to be at the office, the one to take the shortest lunches, and the last one to leave. I'd say that for the first six months, I took a lot of shit from people because of my father. People accusing me of being lazy and getting promotions only because of my father. Then they started to see that I was working my ass off. That there was a reason I was getting promoted after 6 months and they weren't after 2 years, and that was reason was that I cared about my job and wasn't just passing the time from 7 to 6 (while they worked from 8 til 5, often showing up late or leaving early). I was productive, hard working, and had a knack for the job and good leadership skills. I didn't listen to what they said, and I put my head down and kept working. Eventually, the talk went away, and I made some good friends that I still keep in contact with at the company.
Basically, if you want these people to get over whatever they have a problem with, you need to show them that you care about the company and are willing to earn everything you get. Unless, of course, you are just riding on your father's success, in which case you shouldn't be upset when they talk about it, because it's true.