Gran Turismo Sophy: Sony AI x Polyphony Digital

[GTalaune]

Hopefully a full and definitive integration in gt8 at launch.

Hope they do some more test in 7 but it's probably gonna launch on 8, be kinda broken and get fixed as time goes on

 

[sonnyboychris]

From Hypothesis to Reality: The GT Sophy Team Explains the Evolution of the Breakthrough AI Agent – Sony AI

Since its inception in 2020, Sony AI has been committed to enhancing human imagination and creativity through the accele...

ai.sony

This interview was published in January and the developers mention endurance races with dynamic weather as one of their goals for Sophy.

I hope we learn some new things as they sure showcase it on the World Tour events.

Thank you for sharing this, that sounds hopeful!

 

[SUBZERO-8K]

It looks like Sony AI is simultaneously working on another AI model (not Sophy) potentially for use in Gran Turismo.

Essentially it appears to be trained using "vision" such as what we see when playing the game, rather than being fed input data (like slip angle and ratio, wheel load, etc) the way Sophy is. It has currently only been trained in time trial settings, but it sounds like they plan to train it in more of a race setting in the future.

Full details in this Traxion.gg article.

 

[Famine]

It looks like Sony AI is simultaneously working on another AI model (not Sophy) potentially for use in Gran Turismo.

Basically this is news from 2021:

Polyphony Digital Helps New AI Deep Learning Study Using Gran Turismo

A new paper posted to the arXiv research repository has demonstrated the possibility of teaching AI how to race in Gran Turismo using only the same visual cues as human drivers. The research, conducted by Sony AI -- a recently created subsidiary of Sony to study applications for AI -- follows up

www.gtplanet.net

The research predates Sophy and Sony AI, but became part of Sony AI when that formed. I saw the most recent paper was published in August, and it had expanded the track selection to three rather than just the original one, and while still better than the best human players it wasn't quite as quick as Sophy.

 

[SUBZERO-8K]

Basically this is news from 2021:

Polyphony Digital Helps New AI Deep Learning Study Using Gran Turismo

A new paper posted to the arXiv research repository has demonstrated the possibility of teaching AI how to race in Gran Turismo using only the same visual cues as human drivers. The research, conducted by Sony AI -- a recently created subsidiary of Sony to study applications for AI -- follows up

www.gtplanet.net

The research predates Sophy and Sony AI, but became part of Sony AI when that formed. I saw the most recent paper was published in August, and it had expanded the track selection to three rather than just the original one, and while still better than the best human players it wasn't quite as quick as Sophy.

Interesting, thanks. The article made it seem like it was a new interview. Cool to see in the paper summary that it's already being trained with weather on those three tracks.

 

[Famine]

Interesting, thanks. The article made it seem like it was a new interview. Cool to see in the paper summary that it's already being trained with weather on those three tracks.

Well, the paper itself is definitely new (well, August), and I think they were at some AI awards event so Dr. Subramanian might have said something at it, it just looks like the same vision-based RL agent from 2020 with more development rather than a new addition alongside Sophy.

 

[happycorey]

That article seems quite a bit different than anything I'd seen. At least through the screen shots it was provided so it's got to be somewhat new.

The discussion about feeding it visible information vs precise track data is a warm welcome for what else is in store for the new AI in GT though

 

[Famine]

That article seems quite a bit different than anything I'd seen. At least through the screen shots it was provided so it's got to be somewhat new.

The discussion about feeding it visible information vs precise track data is a warm welcome for what else is in store for the new AI in GT though

Here's the pre-Sony AI paper, with the AI trained on visual cues:

https://rpg.ifi.uzh.ch/docs/RAL21_Fuchs.pdf

 

[happycorey]

This is the information I was talking about. Outlined in this paper here.

https://arxiv.org/pdf/2406.12563

 

[Famine]

This is the information I was talking about. Outlined in this paper here.

https://arxiv.org/pdf/2406.12563

Yes, that's the paper presented at the conference in August; it even cites the 2021 Fuchs paper in the Related Work section (naturally )

As far as I can tell, the difference between the 2021 version of the vision-based reinforcement learning agent and the 2024 one is that it dispenses with what the paper calls "forward-looking" or "global features" - knowledge of the course layout is given as an example - and only uses local information (what it can see right now), so that the agent is mimicking human proprioceptive behaviours.

 

[Bornhall]

Fascinated by the technology, but must admit that it is a slippery slope when this matures enough that anybody at home can use it – camera pointed at screen, device hooked up to a controller… Equals basically auto grinding credits at one end, and the end of competitive online time trials on the other? Pros and cons, for sure.

 

[Meythia]

Fascinated by the technology, but must admit that it is a slippery slope when this matures enough that anybody at home can use it – camera pointed at screen, device hooked up to a controller… Equals basically auto grinding credits at one end, and the end of competitive online time trials on the other? Pros and cons, for sure.

You could already do that.
Sophy - with other cars on the track at the same time - is a step further than time trialing.
Also Sophy doesnt suffer from reaction and input delay as any user-solution would, because user can not run his AI-Agent direclty on the console but requires external devices to achieve the same.

 

[jonoholland29]

Fascinated by the technology, but must admit that it is a slippery slope when this matures enough that anybody at home can use it – camera pointed at screen, device hooked up to a controller… Equals basically auto grinding credits at one end, and the end of competitive online time trials on the other? Pros and cons, for sure.

Someone has already done this with Lego Technic… I’ll see if I can find the YouTube link

Here it is:

 

[Bornhall]

I wasn’t saying it was impossible for the auto-grinding, that’s kind of irrelevant in my opinion. I was more going after the machine learning AI of a home user setting time trial lap times that a human driver have difficulty beating. That would be a conundrum to solve. How do you know if someone is as good as their lap times claim to be? Just taking the edge cases into consideration is all 😄👍🏻

 

[Meythia]

I wasn’t saying it was impossible for the auto-grinding, that’s kind of irrelevant in my opinion. I was more going after the machine learning AI of a home user setting time trial lap times that a human driver have difficulty beating. That would be a conundrum to solve. How do you know if someone is as good as their lap times claim to be? Just taking the edge cases into consideration is all 😄👍🏻

You dont know unless they show evidence of them driving (as long as it doesnt look toooooo perfect).
TAS (tool assisted speedrunning) is a thing in Trackmania and theoretically could be applied to all racing games that are deterministic (which is the case for GT7 time trialing, as weather and wind dont change, nor does fuel or tyrewear).
The only thing (in my opinion) preventing this is GT7 being a console game, which causes reaction and input delay to every external device (as said before), as you cant run your ML AI on the console