precalculate_color() had a optimization that avoided duplicated
conversion for YUV formats. This optimization did not take into
consideration YUV444, YUV555, YUV565 or limited range quantization.
This patch keeps the optimization, but fixes the wrong handling.