The reason tracks like North Wilkesboro and Rockingham went away is because the track owners got greedy. The owners wanted to expand out to larger tracks in untapped markets away from the rural southeast, so that they could build larger tracks with more seats, to sell more tickets at a higher cost per ticket. The race dates from Rockingham (60k seats) and North Wilkesboro (40k seats) went to Texas (137k seats), New Hampshire (96k seats), and California (92k seats).
That worked at first, because of the newness and novelty of having NASCAR Cup series races in places they had never been before. However, it alienated a large portion of the core fan base in the rural southeast. The fans felt abandoned by NASCAR when all of the local tracks started losing Sprint Cup race dates and closing. Once the newness wore off in the new larger markets, the track attendance started falling, and attendance went down everywhere.