There is a concern among right wing people that by placing more and more people in government housing and programs, they will vote more and more in favor of state and federal power from promise of funding and/or more personal support, which will place people in powerful positions that see no issue with eroding and diluting the power of the US's constitutional freedoms (btw I don't see individual freedom as bollocks or a direct reason for higher prices on housing, see the local government building regulations that for economic, environmental, nimbyism, but mostly to retain prices of housing for existing home owners or ensure a luxury condo is still profitable after it's construction). You are absolutely correct that the powers that be despise the 4th amendment along with most of the bill of rights that are designed to protect individuals from government by limiting it's ability to intervene. Even today you can see otherwise innocuous groups of people socially being labeled as "domestic terrorists" in the US for opposing restrictions on a whole array of government laws and programs that they see as overreaching and infringing on their constitutional rights.