The first step is to convert from the RGB color space to something more perceptual, like Lab. I'm sure there's a standard for comparing the color similarity of two images but I'm having trouble remembering the name - too early in the morning I guess.