"Ideological" - epithet hurled by ignorant partisan dullards at the principled, in lieu of cogent conversation.
Trump seems to be one of the things in life that I see eye to eye with you about
Gib has made posts that have at turns disparaged Trump for not having an ideology and disparaged libertarians for having one. He's also admitted that he'll be voting for a candidate whose record he has no intent of defending. To paraphrase the great film noir "Out Of The Past", he is a leaf that the wind blows from one gutter to another.
For all sorts of reasons, but the big two are:
1. Crazy people call themselves libertarian when they're not
2. When you believe all government is inherently wrong/evil/whatever (such as ancaps), it is kind of hard to find any common ground
More generally, people who are libertarian tend to be frustrated and feel marginalized by the system. I mean, just go over to Reason and read the comments. Most of the commenters are talking about refusing to vote for Gary Johnson this year because he's not a real libertarian. For the first time since Perot, there is a shot of a third party getting double digits in the popular vote, and they're refusing to vote for him. Johnson wouldn't be my first choice, and he's not really very libertarian, but he's actually the candidate with the most governing experience actually running.
Libertarians seemingly refuse to take advice from the axiom "don't let perfect be the enemy of good." Rather than try to walk back big government slowly, they want to immediately dismantle it, which is unpalatable for most. Why does government keep growing and getting more intrusive? Because the two major parties slowly increase size and scope in their own ways. We didn't get Obamacare before Medicare, we didn't go straight to 90% marginal income tax rates, the interstate tolls in Chicago were "temporary," etc etc.
Also, if my military career had worked out, I would have gone into military intelligence.
The other way to look at it is this. Most people are either partisans or not really either. Imagine being a ideologue and imagine the frustration of dealing with the vast majority of people who are essentially "team blue" or "team red." The positions of the parties change over time, and as a result, people just change their views because that is the way the team is going. As an ideologue, that's hard to handle.
I don't necessarily think libertarianism has anything to do with whiteness. Agreed that democracy is at odds with with libertarianism because government will always work to increase its power, and people love "free" crap.