Don't be confused by other resolutions: HDTV has a relatively limited number of very specific resolutions (I won't go into them here, you can look them up). Using a lower-resolution source-image and expanding (scaling) the that image to a greater number of pixels for your display will always, always, produce a noticeably lower picture quality than if your input resolution and display resolution match exactly. Some systems are better than others, and some people are pickier about this, so your mileage will vary. The alternative, inputing a higher resolution and allowing your display to reduce the resolution is preferable, but still isn't as good as if there is an exact match.
So, I would run at your PS3 at 1080p if you can. If not, you'll have to choose to either expand the 1280x720 (720p) signal to 1440x900 and reduce the quality of the image, or to set your display not to expand the image but instead produce a black-border around a display of 1280x720 actual pixels which will look better but be slightly smaller).
Hope that helps!