Isn't racism judging someone's character by the colour of their skin? The joke is poor, but pointing out that pale skinned people are pale or dark skinned people are dark doesn't seem to be making any judgement on their character.