Temporal reprojection is a technique that tracks the position of pixels over time and predicts where they will be in future. These “history pixels” are combined with freshly rendered pixels to form a higher-resolution new frame. This is what KILLZONE SHADOW FALL uses in multiplayer.
So, in a bit more detail, this is what we need for this technique:
- We keep track of three images of “history pixels” sized 960x1080
- The current frame
- The past frame
- And the past-past frame
- For each pixel we store its color and its motion vector – i.e. the direction of the pixel on-screen
- We also store a full 1080p, “previous frame” which we use to improve anti-aliasing
- Then we have to reconstruct every odd pixel in the frame:
- We track every pixel back to the previous frame and two frames ago, by using its motion vectors
- By looking at how this pixel moved in the past, we determine its “predictability”
- Most pixels are very predictable, so we use reconstruction from a past frame to serve as the odd pixel
- If the pixel is not very predictable, we pick the best value from neighbors in the current frame
On occasion the prediction fails and locally pixels become blurry, or thin vertical lines appear. However, most of the time the prediction works well and the image is identical to a normal 1080p image. We then increase sub-pixel anti-aliasing using our 1080p “previous frame” and motion vectors, further improving the image quality.
The temporal reprojection technique gave subjectively similar results and it makes certain parts of the rendering process faster. This reduces controller lag and increases responsiveness, which improves the KILLZONE SHADOW FALL multiplayer experience.
