I've talked to many "Real" Gamers (Those who specialize in RPGs, Platformers, First-Person-shooters, etc.) about this issue, and the one answer I get every time?
License. Tests.
Why? well, to run certian races, you need a certian license. Which wouldn't be a problem, if the tests weren't so difficult for the typical button-masher to master!!! even such gamers as "Gamemaster" Anthony Bault, (someone who beat Sonic 2 at a demo station in a store) tend to shy away from the subject. and alot of it's finesse; the license tests are more difficult than many of the actual races! Why? you have to run with that same car everyone else does, with all of it's glaring flaws, unlike in the main game, where you can drive pretty much what you want.
I imagine a few people have picked up a copy of GT3 or GT4, and ended up never playing it, as they never really got "Good" at it, their skill set being more oriented towards pushing buttons in quick succession and moving analog joysticks in a skilled and precise fashion. (something I never got the hang of.)
My younger brother is shaping up to be one of these guys. He's beaten Kingdom Hearts II in about a month's time. still terrible at GT. Right now, MMORPGs are getting pretty big: you see them in about every crime drama or sitcom nowadays. Hell, there was a Toyota Tacoma commercial not to long ago about that subject.
It's just a different skill set.