Legal passing, either by going into the oncoming lane on a two lane road or passing in either lane on a four lane road, or on a divided highway....all legal, they always speed up!
I only am in the left lane to pass, or alternately, when the density of traffic necessitates it. If I am in the process of passing four cars with a semi trailer at the front, I will be getting over to the right lane as soon as I reach a time when I can maintain my speed in that lane without getting over again. Impatient people are going to whip into the right lane, always without a turn signal, and try to pass between me, a car and a semi in front of them. It's incredibly dangerous and illegal where I am.
I guess it's just impatience in general; I look several cars down the road when I drive so I know what's coming up. When people whip around me, they aren't seeing that they'll just have to slow down anyway because of the car in front of me. They just have to play Fast and Furious and be dangerous on the road. People who you are passing on a 2-lane highway that speed up when you're on the oncoming lane are dangerous and irresponsible; I slow down when I'm being passed in those situations so people can get over quickly.
And for the record, whenever I am in the left lane and see a car approaching at a higher speed behind me, I try as soon as possible to get into the right lane and let them pass me.