I own the Ultimate Collection and I can confirm that it contains most (but not all, more information below) of the DLC on disc. Keep in mind that it is installed via a second disc, so if you don’t have enough storage (or worse, the game doesn’t come with Disc 2 at all, which it might since you’ll probably buy it used) you won’t be able to get everything.
There is some content that isn’t on disc though: The Ultimate Collection comes with an exclusive ‘Stig’s Garage’ car pack (Koenigsegg CCX, Lexus LFA, Mercedes SLS AMG) that requires a download code. Chances are you’ll be buying a used copy and the code has already been used.
The VIP pack (five cars) and the Community Choice Classics pack (ten cars) aren’t included in the Ultimate Collection at all, and you can’t get them anymore because the DLC was removed from the store in 2013.
Forza 4 is a more complete Forza experience in my opinion; it has pretty much all of Forza 3’s DLC by default plus more. Shame it didn’t get the ‘ultimate edition’ treatment like Forza 3 did.