Like others have said, it comes down to what you are exposed to at a young age, whether that's at school or from a parents/older siblings interests. I can link back all the sports i have an interest in now to how i was introduced to them at a young age. At school i competitively played rugby union, did athletics, swam, played tennis, badminton and squash and basketball. I follow, or are happy to watch all of those sports now. I also follow to a lesser extent rugby league and NFL football - mainly because they're similar games to rugby union and as a rugby team we used to practice them to an extent too. I never got into my country's most popular sports, football and cricket, because i did athletics during the summer instead of cricket, and we didn't play football at our school. My father, uncle and grandfather all raced cars, so i was always going to end up following motorsport.
That's not to say that there are sports that i won't watch just because i've never played them - i've never boxed, skied, bob-sleighed or horse trialed but will still watch them - i just don't have the same connection to them.