The max file format is used by a piece of hardware called the MaxDrive. Once you've bought a MaxDrive, you get some software and a USB memory stick to transfer the max file first from your PC to the memory stick and then onto the PS2. You can then view the max file through GT4, in the replay theatre (but only if the replay came from a game of the same region as yours (e.g. if you're using an NTSC copy of GT4 the max has to come from an NTSC game too)).
You didn't think that little 40KB max file was really a compressed video file to be played as any other video file, did you?