The SVT Contour looks nice and I've heard otherwise that its a good car, albeit styled like a bar of soap.
I would suggest looking on Craigslist in Detroit and Lansing to augment your current searches (if you haven't already). An Infiniti G20 is a great little car equivalent to the early 1990's Sentra SE-R (with sr20de) and can be had for a couple thousand if you look carefully.
A clean civic is never a bad platform to start off with, as it will last you forever. However, you always have to watch out for the ricers (

) and there's only so much modification you can reasonably do. Hondas hold their resale value well though, so you'll probably be looking at $3000 + for a mid to late '90's version with a manual.
F-body pricing is attractive but the insurance isn't. The decent Firebirds within your price range are 3.4 L V6's with automatics (

) and later models with the Series II 3.8 V6 are usually more expensive or taken care of poorly. The third gen Firebird is hard to find in good condition and without much rust.
If you're into controversial cars, you could always go for a Pontiac Fiero. Sits very low and handles decently (the 1988 models more so) but controllably. They can be had for next to nothing and will not have exterior rust (fiberglass body). Trouble is that the seating position can be a little bit uncomfortable and has (muscle) power locks, windows, doors, etc.
See here for an example.
Good luck!