The poor do not have the ability to manipulate mass quantities of people, the rich do. Because they are rich and have the resources and connections to accomplish this. They buy politicians and manipulate to cause the conditions you describe. If they were not rich, they could not do that.
Blaming the poor for being manipulated is bad faith. And victim blaming is not an effective rallying strategy.
It's a bad faith argument and a strawman. They don't actually think it's reasonable for anyone to do that or think the other person is suggesting that. They are setting a person up as a hypocrite despite that obviously being an insufficient and inefficient solution to the housing crisis.