I think its just your tolerance for lag. The lag that occurs using xlink is consistent. The Frame rate is consistent enough that I can race and drift as I could with the normal frame rate. I have to adjust my driving a bit but because it is consistent frame rates, driving inputs are predictable if you know what I mean, you will just have to adjust for it. Sounds to me that since you say you are a hardcore gamer, you may not like it. But have you tried it yet? If you have two skilled drivers racing each other, its a lot of fun.
Pretty much all the online games I've played for the PS2 lag the same way. I've gotten used to it because I play a lot of them. (Madden, Nba live, NCAA College Footbal, ESPN Football, etc...)