I liked the analogy that, in a first person shooter, someone shoots you in the head, you are DEAD...
Personally, I think we spend FAR too much time wanting damage to LOOK accurate, and not nearly enough wanting it to affect the driving physics. But any driving game that doesn't penalize overly aggressive driving, is de facto encouraging that. And all we will end up with is Shift all over again. Rammers, bashers and track cutters. That's not what GT is all about, is it?
It is the graphical representation of damage that is the chokepoint here. If damage didn't LOOK very much (if anything at all) but you had a 'damage meter' like a 'health' meter in a FPS, and as your damage accumulates, you slow down, lose traction, steering input, whatever quite noticeably, you could start to drive realistically again.
The way to encourage clean racing is not to balkanize the players into clean clubs just so they can get away from the idiots that will exploit the ability to bash without consequences. It is to make the idiots suffer tough enough consequences for that idiotic behavior that they don't WANT to do it any more! In fact, making crashes LOOK realistic is probably the way to guarantee that we will see a LOT more of them!
Making damage ONLY if you can get it to LOOK good (realistic) misses the point. Make damage affect the car, even if you can't SEE it, and clean driving will become the norm (and will tax the PS3 a lot less, making it more likely that it CAN be made to behave correctly), not the exception...