> Losing TOTP is easy. Lose your phone and it's gone.

That is the main point of it. That's why it is called a second factor.

> It means game over for a regular person.

It just means you have to go to the nearest branch.