As
Joe_Ald said above, you can create a private "My Lounge" that only your PSN friends can see and join. You can set the initial settings of the room from the "Options" cog off of the GT Life screen.
Once inside, you can change tracks, limit cars, etc, just like you would in a public lobby, but only your PSN friends can join. So, just make sure that the people that you want in the lounge are actually on your PSN friends list. I know that sounds pretty obvious, but I just wanted to make sure that you knew.
Once you have started the lounge from the "Community" section, a small circle will appear next to your name in the Community section and then, your friends can just click on that little yellow circle and join directly to your private lounge. 👍