See, I would think this might only be true at very high speeds, such as around Special Stage Route X for instance. Because a car with cleaned & polished bodywork is going to have a very slightly better aerodynamic profile. Around most tracks, this very slight increase of aerodynamic drag isn't going to make any difference, though.
The ultimate test would be to actually drive a dirty, high-powered car around Test Track X. Then do the same drive after Car Wash has been performed. I'd do this, but so far I don't have anything worthy of a drive around SSX.
The only game so far in which I've noticed dirty cars REALLY looking dirty is the original GT. I have an '88 Silvia in that game which looks as though scratching your fingernails on its bodywork will make an annoying SCREEEE sound.