X windows client server model can seem intuitively backwards. The applications you run on remote servers are clients of the x server (that renders the windows) you are running locally.
It's a relic from a different era: You had a dumb terminal that could only display stuff, and several servers that each could run a piece of software you like.
Also, the networks were (supposedly) less dangerous security wise, so people worried less about things like authentication and access control back then.
X is celebrating 25 years or so these days: And yet, modern clients can mostly work with 25 year old clients, and modern servers can definitely accept connections from 25 year old clients. And it's very usable.
It's a marvel of engineering, even if it is showing its age.
Still, it'd be a good idea to tunnel it through an outbound SSH connection (which is generally the preferred method of starting up the X client app in the first place).