OpenSource AI Race Engineer for Gran Turismo 7

  • Thread starter baruta
  • 11 comments
  • 1,408 views
4
United States
United States
Hey everyone! I’m excited to share a little project I’ve been working on: an AI “race engineer” for GT7, built entirely with open‑source tools and the Gran Turismo telemetry app (https://pypi.org/project/gt-telem/). If you like racing with the HUD off, especially in VR, this might be just for you! I think this project has legs, but lots of room for improvements, so all feedback is welcome! And if anyone is keen to help out and contribute to the GitHub, please feel free to do so!

Why this exists

- There’s already a couple beta open‑source engineers for live online races, but I mostly race against Sophy/do weekly challenges

- GT‑telem currently only exposes your own car’s data, so this is to get information of your own vehicle or the general race

- I wanted real‑time voice feedback without cluttering my screen (and since I play in VR mainly, without the HUD)

- I wanted to make a tool that did not use pre-scripted/pre-recorded responses, but rather genAI, and with telemetry data, frontier models are great at processing it and not hallucinating responses

Costs

- Uses OpenAI audio & chat calls under the hood.

- Roughly $0.01–$0.02 per full race.

Try it yourself

Admittedly, this is still very Python-centric, and at some point it'd be great to move to a simple executable folks could run. But if this is of interest to you, please check it out! Happy to iterate on issues that I'm sure will come up in the initial testing phase. Here's the repo: https://github.com/baruta1/gt7-engineer

---

## Features

### 1. Voice‑activated Q&A


- You join a Discord voice channel & type !engineer to summon your AI engineer. This makes it easy to then cast Discord to your PS5 so that you can use your audio/microphone from the PSVR, or you can play with your computer right next to you.

- Once the race starts you hear:

“Engineer here — radio check, good luck out there!”

- Then, at any time, you can say “Radio…” followed by any question:

- “Radio, how many laps left?”

- “Radio, what’s my tyre temperature?”

- “Radio, how long have I been racing?”

- the beauty of using GenAI is that you can ask things any way you want and about anything that data is there for to the engineer!

- The bot transcribes your question, grabs the latest telemetry, and uses GPT‑4o to reply via TTS. You'll need to have a .env file with your OpenAI credentials as mentioned in the README.

### 2. Automatic race updates

- Lap summaries (every lap except Lap 1 and the final lap): one‑liner summarizing position, fuel, pace, or tyre wear.

- Fuel warnings at 50%, 20%, and 10% remaining

- Position change alerts (overtakes or lost places) after Lap 2, with a 45 s cooldown

### 3. Customizable

- Replace driver_name in the main code (GT7_Radio_GenAI.py) so it uses your name on the radio

---

## Getting Started

  1. Clone the repo
    1. git clone https://github.com/baruta1/gt7-engineer.git
    2. cd gt7-engineer
  2. Install dependencies
  3. Configure tokens
    1. Add your Discord Bot Token & OpenAI API Key to a .env file
  4. Run the GUI launcher(shows an image & real‑time logs)
    1. python gui_launcher.py
  5. Join the engineer in Discord:
    1. Join a voice channel
    2. Run !engineer
    3. The radio should type Waiting for the Race to Start in Discord
  6. Start a race and use your engineer!

Call for Contributors​

I’m new to sim‑racing and open‑source collaboration. If you’re a Python/Discord/AI enthusiast (or just a fellow racer) and want to:

  • Improve latency
  • Add more telemetry hooks (weather, opponents, pit‑stop strategy)
  • Polish the GUI or packaging
  • Swap in a cheaper/faster TTS engine
…please jump in! Issues, PRs, and ideas are very welcome!!
 
Last edited:
Top notch project! Great idea (the hardest part imo)! The execution sound sensible too, at least to get a working prototype. This parts takes quite some ingenuity too (using discord to avoid a lot of heavy lifting on the coding side for example). Well done!

I doubt that I'm a good enough software developer to be useful here and also haven't worked with Python in years but I might take a look at the repo and maybe at least spam some ideas for improvement or added features and let others do the heavy lifting.

I would guess that making using the thing as simple as possible should be priority number one. Setting it up sounds a bit tedious unfortunately and might be too big a hurdle for some, especially those not that tech savvy / not that interested in tinkering / those that not yet have Discord and /or ChatGPT accounts.

Pipe dream and probably completely unrealistic would be something hosted remotely that could be accessed directly from within the PlayStation UI (are there PlayStation apps? From the PS store?) without any additional configuration necessary. Well Polyphony paying you for the idea and putting it directly in the game might also be nice but let's not get ahead of ourselves. But then again Sophy started similarly as some dudes research project if I recall correctly.

More realistically step one would be packaging as a standalone exe and the equivalent for MacOS (nonidea what things are called there), step 2 no need for discord, step 3 no need for paid LLM (no idea how long till proper TTS and a LLM model capable of what the engineer does will be free and how to integrate it without user needing to add own (unpaid) account anyways but who knows)

Also integration into the controller speakers (and thusly headphones via audio jack) would be great.

I would also be interested in hearing from users what useful info not mentioned by you here they could reliably get out of it and also where they didn't get something they would have expected to find.
 
Demo Video showcasing all of the features I have programmed in the github:

This was captured with my microphone, so the quality isn't great. It sounds much better when audio is direct into the PSVR2. This is just a demo, and I hope to improve several aspects of the engineer. Please feel free to let me know if you have any suggestions or feedback!
 
Last edited:
Top notch project! Great idea (the hardest part imo)! The execution sound sensible too, at least to get a working prototype. This parts takes quite some ingenuity too (using discord to avoid a lot of heavy lifting on the coding side for example). Well done!

I doubt that I'm a good enough software developer to be useful here and also haven't worked with Python in years but I might take a look at the repo and maybe at least spam some ideas for improvement or added features and let others do the heavy lifting.

I would guess that making using the thing as simple as possible should be priority number one. Setting it up sounds a bit tedious unfortunately and might be too big a hurdle for some, especially those not that tech savvy / not that interested in tinkering / those that not yet have Discord and /or ChatGPT accounts.

Pipe dream and probably completely unrealistic would be something hosted remotely that could be accessed directly from within the PlayStation UI (are there PlayStation apps? From the PS store?) without any additional configuration necessary. Well Polyphony paying you for the idea and putting it directly in the game might also be nice but let's not get ahead of ourselves. But then again Sophy started similarly as some dudes research project if I recall correctly.

More realistically step one would be packaging as a standalone exe and the equivalent for MacOS (nonidea what things are called there), step 2 no need for discord, step 3 no need for paid LLM (no idea how long till proper TTS and a LLM model capable of what the engineer does will be free and how to integrate it without user needing to add own (unpaid) account anyways but who knows)

Also integration into the controller speakers (and thusly headphones via audio jack) would be great.

I would also be interested in hearing from users what useful info not mentioned by you here they could reliably get out of it and also where they didn't get something they would have expected to find.
Thank you for the feedback! I agree that as of now it is quite clunky with the setup, and that the first thing to work on is making installation/trials as seamless as possible. I think I could get around the OpenAI token perhaps using a local LLM (which do not require tokens), but I am puzzled for how to pick up live audio from the PS5 and feed that into my computer for processing without Discord. If anyone knows of Playstation UIs that can facilitate this, please let me know. The three steps you laid out seem great. I'll aim to work on that!
 
I don't have much time to help but cheers, this is a great project which would greatly enhance the experience ! Good luck for the next steps !
 
Thank you for the feedback! I agree that as of now it is quite clunky with the setup, and that the first thing to work on is making installation/trials as seamless as possible. I think I could get around the OpenAI token perhaps using a local LLM (which do not require tokens), but I am puzzled for how to pick up live audio from the PS5 and feed that into my computer for processing without Discord. If anyone knows of Playstation UIs that can facilitate this, please let me know. The three steps you laid out seem great. I'll aim to work on that!
Yes local open sources LLMs might be powerful enough to get the job done nowadays but I'm not too knowledgeable on what is and is not possible especially when it comes to self hosted stuff running on a consumer grade machine, i.e. not in the cloud in some insane datacenter.

What exactly do you mean grabbing live audio from the PS5? Do you mean the player's voice spoken into a headset connected to the PS5? You might need access to the PS developer kit or whatever they offer to (probably vetted) developers, which might be hard to get. But maybe there are open APIs that aren't really secret but maybe hard to find info on. But honestly I doubt it. But then again there must be something because how else do games transfer voice chat between players?
 
@baruta, which python version is required? I'm getting a lot of errors when installing the requirements.txt complaining of python version.

UPDATE: TTS is not compatible with python 3.12

Thanks!!!
 
Last edited:
Thanks all for the suggestions. I looked into potential open source remote play APIs. It looks like Chiaki might be one way I can get the microphone from the controller into an audio file on my computer (https://sr.ht/~thestr4ng3r/chiaki/). But then the issue I have is how do I play the audio onto the controller, does anyone know if Chiaki can do that? More generally, does anyone have experience with Chiaki and getting it to record audio files?

@Seiitiro it does look like TTS v0.26.0 works with Python 3.12. At leas tthat is what I have been using (TTS(model_name="tts_models/en/vctk/vits")).
 
I am of no help with these questions and don't have the time and energy to do research or contribute in any other way.

But I just wanted to say that I love how much effort you put into this and how quickly you get ahead. You got this. This thing will be great. Once it is out of the alpha stage and people can readily use and (most importantly) it delivers what the users want (I believe that's possible, definitely when relying on paid online LLMs) then word will get around.

I hear from many VR users that they love the immersion but hate the limitations. This gaps that divide.
 
… For the first time I also used the pit engineer voice in the Ezio app to give me tyre temp info. Just nice to have a reminder when tyres are cold, and potentially when they overheat. …
Sounds like somebody has already done something like this.
 
Back