Audio flag warnings in GT7, here’s how…

  • Thread starter Thread starter Nightster
  • 9 comments
  • 1,850 views
Nightster

Nightster

Messages
41
United States
United States
My set up uses a PS4 and Remote Play.

So basically, I wrote a script for autohotkeys (runs on windows) that looks on screen (PS Renote Play on PC) for a specific color in a specific pixel range. If it finds it, it triggers an audio file I recorded. This is just an example of a blue flag. I used the blue flag as an example because it’s easy to recreate on the game.

With that said, I am currently working on modifying the code to check for a yellow flag also.

this is fabulous since polyphony does not give us this telemetry in game output

Let me know your thoughts. I can share the process and code. My script is compiled into an EXE file and it just runs in the background.

 
smart. I do wish PD would see these types of tools users are developing and realize that a crew chief style audio cues system implemented in the game would really enhance the experience, especially for VR users who are missing HUD information. for me personally I find generally external tools are too cumbersome to use for a console experience, I will just play a PC sim at that point, but I do understand many people use them to enhance their GT7 experience so love to see things like this.
 
notarjy
smart. I do wish PD would see these types of tools users are developing and realize that a crew chief style audio cues system implemented in the game would really enhance the experience, especially for VR users who are missing HUD information. for me personally I find generally external tools are too cumbersome to use for a console experience, I will just play a PC sim at that point, but I do understand many people use them to enhance their GT7 experience so love to see things like this.
Click to expand...
Thanks, just exploring what some 3rd party applications are capable of. I think more telemetry from the game is a must…

I’m currently using SIMhub screen capture to create a 3 TV set up for the PS4. Have a 50” and 2 32” turned on their side
 
OK, this thing is starting to take on an entire screen engineer project, lol. I have yellow and blue flag audio warnings working.

Next I plan on coding for Best Lap, Penalties and Radar proximity(using red indicators that come up on the hud with newer cars). This is all triggered by screen color so it’s not perfect.

Just audio warnings. Such as you incurred a penalty, car left or car, right. That was your best lap, blue flag, caution flag out.

I’ll try to post a video of everything working whenever I finish coding and I can create each scenario in the game
 
I’ve taken this project even deeper. Did not really plan on that when I started, but the more ideas I come up with the more I want the automation 😆

I’ve started looking at just single pixels for more accuracy now. This is not all tested, but it should work. 😉 a lot of it is tested. But I am reporting current lap every minute. Reporting yellow and blue flags. Reporting card left or car right. Reporting best lap and fastest lap from everyone, but those need a little more logic before they’re accurate. Also looking at the fuel gauge to determine percentage based on pixel colors again.

Then I thought to myself why not try to call some command line OCR program to try to capture what I can from the game screen. So I am using a combination of third-party products to make that happen. I use ‘Nircmd’ to grab the game screen from PS remote play. Then I use ‘tesseract’ to read the image and covert what text it can. Then I parse what usable data I can use. For example, I can see current laps and total laps. That allows me to use that information to report audibly. So I’m still going through the file to see what I can actually code for.

But as I was coding for current lap and total laps, I thought wouldn’t it be nice if I could use AI to tell me what lap I need to pit .and possibly use AI for other race strategy based on current conditions. So based on laps remaining, fuel remaining and fuel percentage used per lap I could ask AI what lap I need to put on. So I coded that and it’s working.

I may just put the math in my program to handle that computation, but now that I have established a command line script driven ChatGPT interface with audible responses. I’m sure I can come up with some ideas for that. It’s not a wide open interface for ChatGPT. And by that I mean, since this is all code running in the background, the questions have to be pre-determined and variables fed to ChatGPT. And the call to ChatGPT on comes from the screen engineer program that calls it when certain conditions are met.

Making progress, but the more I add the more dependencies are required

So far I have downloaded and installed the following

AutoHotKeys
Tesseract
Nircmd
ChatGPT
 
Last edited:
@Nightster this all sounds amazing.

As soon as I read that you use pixels for the flags, the first thing I thought of was tire wear could have an audible feedback for 50%, 25% 10%, something along those lines.

It would be great to see a weather radar in one of these consumer made apps, but don't think that is possible from data pulled from the game 🤷🏻‍♂️

Well done, looks like you've made a great project, looking forward to see it all in action.
 
P-Courty
@Nightster this all sounds amazing.

As soon as I read that you use pixels for the flags, the first thing I thought of was tire wear could have an audible feedback for 50%, 25% 10%, something along those lines.

It would be great to see a weather radar in one of these consumer made apps, but don't think that is possible from data pulled from the game 🤷🏻‍♂️

Well done, looks like you've made a great project, looking forward to see it all in action.
Click to expand...
Tire wear is definitely doable, based on the red color. I’ll add that on for sure
Thanks
 
Well turns out doing a screen grab, processing to text, reading the text and computing it takes a lot more time than I expected. There’s no way the PC can keep up with the game. At least not my PC.

So even though I coded a lot of it, I can’t include that code.

Still deciding on using AI for pit strategy as that logic was driven by the screen grab. There are some other third-party products out there that might be able to feed me the information required to get AI involved. More to come on that.

With all that said the color pixel, checking on the screen is going nicely. I believe I’ve got blue flags and yellow flags worked out. I believe I’ve got tire wear worked out. And now I’m working on fuel consumption.

For each event that occurs the engineer audibly reply’s .

More to come
 
Back