Veganism is more or less a 1st world phenomenon. Most humans, especially in the past, did not have the luxury to choose what they could eat. They ate what they could get and if they got access to meat and animal products they ate it because it has high nutritional and caloric value. Even the vegetarian Indians who don't eat meat foe religious purposes still have eggs, milk, etc.
It feels disconnected with the human struggle.
In addition, it's sort of meaningless in the grand scheme of things. OK. You don't eat meat to protect domesticated cows. In reality, those cows would not exist in the first place. So really, you're advocating to eliminate the species of domesticated cattle.
In addition, our modern society requires massive amounts of energy which is often generated by fossil fuels. Even if a society uses 100% solar, they're importing products from countries like China that burn coal.
So you're pumping out carbon emissions that will inevitably result in mass extinctions anyway. It seems like a meaningless protest against the inevitable. You say let's exterminate the cows to save them from suffering on one hand and with the other drive to work talking on your iPhone with the A/C turned up- contributing to the destruction of infinitely more animals.
The only real way to stop is for everyone to give up every modern luxury and live in a log cabin in the woods. And for the vast majority of the population to die off.
It just feels like pissing into the void but doing so with moral superiority.
Having said all that, I empathize with many vegans. But those are some thoughts on why people may look down on vegans.