Are the windows and walls of the said house also made of steel? In that video, why are the armed personnel so obsessed with entering through the door?

Because real life isn't an action movie, and police would rather try to ram the door for a while and hope it budges, instead of instantly going for rappelling from the rooftop or for using a ladder. Especially given how relatively uncommon steel core doors in apartments are in practice.

One guy climbing through a shard-lined window is easy to mow down; the first cop that dies halfway through will block access to the rest. It also disarms the guy with the riot shield.

Opening the door lets a group of them quickly file in at once to swarm the occupants.