I think you have it exactly backwards the cost goes down due to the area being a less desirable location allowing the poor to afford the houses there. If you analyzed the situation better it is extremely likely that there is an underlying cause that isn't poor people moving in. The economic situation in Detroit has been shit and getting shittier for years now leading to the people who are more capable and therefore able to command a higher salary to tend to move elsewhere.
A house that a doctor and a lawyer are competing to buy is going to be go for more than the one an auto mechanic and a fireman are competing for.
It is a little bit of both, at least when it comes to maintaining a local public school’s rating. People will oppose lower income housing to prevent the children of lower income people from attending the same schools, which might cause the school’s rating to go down. Which would then be downward pressure on the price of the land.
Every real estate website has school ratings for each house listing because it is such a sought after amenity.
A house that a doctor and a lawyer are competing to buy is going to be go for more than the one an auto mechanic and a fireman are competing for.