I made a point about this previously and unfortunately, your situation is exactly the reason why SMS authentication should be avoided, since these sort of attacks are now becoming common. [0]

[0] https://news.ycombinator.com/item?id=27311641