I am really curious to know why? And don't invoke "what about your kids, mothers, fathers?". "Solipsism" (first time I have heard of this, so thank you) does not make me feel any less worse when something happens to them.
The parent was making a joke, but one way to answer your question might be:
You feel bad because you assume they - like you - feel something when you mistreat them, and you assume this at a very deep level that might not, itself, feel like anything to you.
While some humour was intended, I do actually grapple with this question. The answer that rings truest for me, especially as a newly minted father, is Vonnegut’s: “A purpose of life, no matter who is controlling it, is to love whoever is around to be loved.”
