I've been doing the entire career on expert, I'm not very far in, near the end of year 2, I believe.
My suggestion is to mess with the controls until the clutch is on RB. It boggles my mind that they'd default it to the side with the brake... in real life you take your foot off the gas to clutch, and you have to in the game as well or you damage the transmission, so remapping it makes all the difference. You also need to clutch to downshift while braking, which is pretty much impossible with the default controls.
I can shift crazy fast in Forza, and it feels right. It's the most fun I've ever had playing a racing game since Rally Cross (PS1) because of the clutch. The only downside is that once you get the clutch on RB the "A" button changes to swap camera, and sometimes you accidentally swap your camera.