You're right. Thinking about this more, I think perhaps there's an association between verbal communication difficulty and intelligence.
I've personally experienced situations where people seem to think I cannot understand some simple concept when I am not a native speaker. Just an an anecdote, though
I've caught myself on the giving end of that. I'd think "Oh, they don't speak clear English, therefore they can't be smart." But that is obviously completely wrong. I don't know why my brain goes that direction, or what I can do to stop it.
It happens to everyone, and the best you can do is to try to be aware of it.
It happened to me often when interacting with Chinese CS professors during college. I'd constantly have to remind myself that they were much more knowledgeable than I was despite having subpar communication skills.
I've personally experienced situations where people seem to think I cannot understand some simple concept when I am not a native speaker. Just an an anecdote, though