I'm pretty much worthless with a pad nowadays as I've been wheel-only since GT3.
I think it's largely the same thing as when you switch to a wheel: at first you're terrible, after a day or so you get to where you can handle it, after a week or so you start being more consistent, after a couple weeks or a month or so you get fully 'up to speed.' Of course, if you don't completely stop using one or the other for a long time, you won't lose nearly as much of the technique/muscle memory.
The problem is, when you switch from a pad to a wheel, the sheer fun of using a wheel keeps you coming back until you finally get properly comfortable with it. When you switch from a wheel to a pad, the sheer silliness of it keeps trying to make you quit instead of soldiering through.
I pretty much leave the pad for screwing around while not entirely in full contact with my senses. Or for having a game at a friends, which often amounts to the same thing(and usually with a less unforgiving game).