Firstly
Firstly, I have to say, do not judge anothers performance by the replay. Your replay firstly, will not show other drivers smoothly, what may look like them jittering about in a turn or even a straight. Your entity takes priority to any other entity while the game records the data. As such, everyone will look kinda jittery. Secondly, your replay I'll most likely not look exactly the same as someone elses, stewards can back that up. Add to that the fact that usually the input being recorded is either full on, full off with a controller, and the issue gets even further convoluted. Kinda like how someone who is lagging can go from being smashed in a wall 5 cars behind you to all of a sudden taking the same space you are in and blasting you off the track.
That said, it's all in the fine controls. Accurately modulating the brake and gas with the X and square buttons just isn't going to happen, minuet adjustments in a turn are not going to be as smooth or small using a joy stick as they will with a wheel. I know personally, there is nothing I do better with the DS3 than my wheel. Even drifting is easier with a wheel for me now. I can't imagine there is anyone worth their salt on the track that will actually be faster on the track with a controller over the wheel. They might be faster than you, but I don't think they will be faster than themselves given the same amount of experience with both.