Err…no one wants their neighborhood to suddenly have a bunch of new unknown people in it, right? You don’t have to be a nimby to be concerned about who you live next to. It’s not a “misconception” in that sense, especially if moving is impossible because of rent control, as you mention.
Perhaps we need better protections around dealing with insufferable neighbors and other neighborhood changes’ negative impacts, and people will object less?
I believe you, on the other hand, you’re one data point, and apartment buildings have stricter rules about acceptable activities and behavior (as I propose above).
Even in the biggest cities, apartment buildings often have board that approve sales and/or new residents.
...until your neighbors are all renters, at which point the tide turns?