I think it might be possible to infer the colours of some objects based on how old black and white film worked because each type had different spectral sensitivity and probably has a signature that can be determined automatically. Once you know the film you might have a shot at inferring what colour that car is for example.