Even with a console with a thousand TFLOPS of horsepower, you're unfortunately still going to be restricted by what a team of people can create in a 2-3 year dev cycle. It's why The Crew has a gigantic map, as you say, that's mostly empty.
That applies to the cars too, I'm sure. Forza has generally gone for quantity, over the ability to extensively customize ALL of them.
I'll bet the inevitable Forza Horizon 5 has a bigger map than any previous game, but I doubt it'll be The Crew-sized, because they clearly pride themselves on the detail of their environments, whereas The Crew's map goes for quantity over quality. Plus if they keep up the seasons thing, they have to create four variations on the same basic terrain.
I'm excited to see what Forza Horizon 5 looks like on a new console (with no Xbox One S anchor around their legs), and I kinda hope that Microsoft give them a ton of time to really revolutionize their formula - maybe even another extra year, for a four year dev cycle, instead of the usual two. But I'm guessing it'll depend on how Turn 10 improve on the Forza engine for next-gen.