Comparing objective and subjective error measures for color constancy

We compare an objective and a subjective performance measure for color constancy algorithms. Eight hyper-spectral images were rendered under a neutral reference illuminant and four chromatic illuminants (Red, Green, Yellow, Blue). The scenes rendered under the chromatic illuminants were color corrected by 5 color constancy algorithms that are based on zero-, first- and second-order image statistics. The angular error is used as the objective performance measure for color constancy. It estimates the chromatic mismatch between the true and estimated illuminant vector in RGB space. A subjective performance measure was derived from a psychophysical experiment involving paired comparisons of the color corrected images shown on a calibrated monitor. Eight subjects indicated their preference with respect to color reproduction when comparing the two images (i.e. color constancy algorithms) against the reference image (the same scene under neutral illumination). Our results indicate a large negative correlation (-0.9 on average) between the objective and subjective color constancy measures. The data suggests the possibility for further improvement of the correlation between the two types of performance measures.