There are a number of factors which affect online connections and there's not enough information in your post to really diagnose what's happening. But I can tell you P2P networks and the PSN are a bit sensitive. I don't think it has anything to do with GT5 itself. I've played at least 8~10 other racing games and have had similar issues with all of them.
This past weekend, I hosted an organized online event and we had people from the US, Canada, England, Ireland, Western Europe and Eastern Europe, all in the same lounge for 2+ hours of solid action. During that entire time, nobody was disconnected and lag was minimal. But in order to ensure this level of quality, certain steps are required.
As the host, I switched off my wireless network, bypassed my router and plugged my PS directly into my modem. NAT1 connection. The most common form of dropped connection in an online lobby is NAT negotiation failure. And it may even be somebody ELSE causing you to drop. Which is why, if somebody else is in the same online room who has a PS3 running behind a complex firewall, it can disrupt other players as well. Make sure (System Settings from XMB) that you have a NAT1 or NAT2 connection. Even so, a NAT 3 connection can sometimes be erroneously reported as a NAT2 connection. If you have no option to bypass your router, check the FAQ on port forwarding. If done correctly, it's just as effective.
The 2nd thing I did before opening my lounge was to clear the cache (from the options menu), restart the PS3 and reload the game. It sounds like overkill and it's a PITA for normal play. But I believe it really does work.
The 3rd thing I did was ask EVERYBODY else joining that race to do the same. Obviously I knew everybody else who would be racing and such a scenario is simply not possible when joining random online lobbies. But I just mean to say, under the right conditions, it's possible to have a stable online session.
It also helps that I as the host had a strong, stable connection. Testing it on the PS3 (using speedtest.net) I had a 12ms ping to my nearest local server and between 85ms and 130 ms to various overseas locations. Your ping speed is probably more important than your actual bandwidth. If you have 5 down and 1 up it's probably more than sufficient.