Autonomous Driver outperforms

[Voodoovaj]

My buddy sent me this. SUPER interesting.

 

[breeminator]

The link is in the youtube description, but in case people wouldn't spot it there, the paper can be read here:

https://arxiv.org/pdf/2008.07971.pdf

 

[meekrab]

Going fast around a circuit is neat. How's it perform in traffic?

 

[breeminator]

Having looked at the paper, it looks like it had access to some sort of unpublished API for getting data from the game and providing its inputs, rather than using a camera to observe the screen and having to physically control a wheel and pedals. I'd expect it to lose some of its advantage over humans if forced to interact via the screen and wheel+pedals, as it would introduce errors due to e.g.
- limited screen and camera resolution affecting the accuracy of distance measurements
- unpredictable small variations in timing throughout the chain
- errors in inputs due to e.g. potentiometer noise, and force feedback affecting the wheel position

I'm sure it could still outperform any human, but it wouldn't be able to so consistently push so close to the limits if it had to use a camera to observe the screen and had to physically apply the inputs (with its physical ability limited to that of a human rather than having such a strong grip on the wheel that the force feedback doesn't affect the wheel position).

 

[Leggacy]

Looks interesting but remember this is using an off board computer to control one AI, you need to squash all that code into the game and then make it control 20 individual cars all at the same time. It’s very interesting but I’m not sure we’re quite there yet. I am wondering, can this sort of thing be handled by a GPU?

I am also wonder how this AI would perform on a track with other cars, does it have any racing etiquette? Will it occasionally make mistakes?

 

[breeminator]

I am also wonder how this AI would perform on a track with other cars, does it have any racing etiquette? Will it occasionally make mistakes?

From the paper:
"Limitations of the present work include a) the restriction to single-player time trial races without other cars on the same track, and b) the constrained applicability of the learned control policies to a single track / car combination. We intend to address a) in future work by extending the observation space to allow the perception of other cars and modify the reward to disincentivize unfair driving behavior."

 

[meekrab]

Looks interesting but remember this is using an off board computer to control one AI, you need to squash all that code into the game and then make it control 20 individual cars all at the same time.

Also the goal of the Gran Turismo AI has never been and will never be to be as fast as possible, it's only meant to give a relatively easy to overcome challenge.

 

[Leggacy]

Also the goal of the Gran Turismo AI has never been and will never be to be as fast as possible, it's only meant to give a relatively easy to overcome challenge.

It’s a start though, if the A.I can learn from your driving and pace it’s self against the player, it could make off line racing more interesting.

 

[Fennario]

Any bets on how long until someone reverse engineers and distributes the "Ultimate Qualifying" hack for GTS and/or if one is already out in the wild? In order to cloak its use, imagine there would be the ability to add some "mistakes" or randomness to achieve a specific time vs. the absolute best (eg. just a few 100ths faster than the best time currently on the leaderboard instead of beating it by 10 seconds).

For fun, you could pretend to drive like my kids did when they were in car seats with the fake steering wheel. Can even save the electicity cost for your wheel by not having to plug it in/power it up.

 

[waukazoo]

My buddy sent me this. SUPER interesting.

Sadly, that will be the ruin of racing games.

 

[LeGeNd-1]

Fascinating stuff. I wonder who is the reference driver they used i.e. would the AI still be faster against the ultra elite drivers like Hizal/Fraga for example. The computations are probably too resource intensive for this to be used in the actual game itself though.

The team behind the paper should apply for Roborace. I bet they'd do very well

EDIT: Just read the pdf. Interesting that the AI took a tighter entry line at Tokyo's final hairpin and does a double apex, whereas the elite player took a later entry and stays tighter all the way through. I always feel a double apex is faster here, but the lack of peripheral vision to look "around" the turn always makes it inconsistent and difficult to nail the second apex. No such problem for the AI, and probably the human could get closer if we have triple screen/VR.

 

[Fennario]

Going to reach into my back pocket and go quantum... At the end of the day it is a "traveling salesman" problem (NP-complete) which is why it was interesting for the paper's authors. The program/computer needs to go through every permutation of how to take each corner and straight with all of the speed variables, then plug each of those outputs into the following corner/straight, and so on. Very easy for a computer to do at first; however, as you add more corners, elevation, gearing, straights, braking, etc. the number of possible combinations quickly escalates into the trillions and beyond, becoming very difficult/impossible for traditional computers to resolve. As such, solving problems like this without having to resort to time consuming brute force attacks are at the forefront of quantum computing (similar in many ways to cryptography).

 

[meekrab]

Going to reach into my back pocket and go quantum... At the end of the day it is a "traveling salesman" problem (NP-complete) which is why it was interesting for the paper's authors. The program/computer needs to go through every permutation of how to take each corner and straight with all of the speed variables, then plug each of those outputs into the following corner/straight, and so on. Very easy for a computer to do at first; however, as you add more corners, elevation, gearing, straights, braking, etc. the number of possible combinations quickly escalates into the trillions and beyond, becoming very difficult/impossible for traditional computers to resolve. As such, solving problems like this without having to resort to time consuming brute force attacks are at the forefront of quantum computing (similar in many ways to cryptography).

It's only a travelling salesman problem if you decide to brute force it, which would be the worst possible way to build a functioning driving program.

 

[Deadpool]

It’s going to be awhile before a computer can challenge a human in this regard. I think a long while actually.

Sure, for a single time trial lap. We can make a computer go faster. But that’s easy. Now try and make it do a 24hr race at The Ring with a field of 200+ cars. The situational awareness, feel for tires/changes through a race. That’s the hard part.

 

[Fennario]

It's only a travelling salesman problem if you decide to brute force it, which would be the worst possible way to build a functioning driving program.

If you want to achieve the "fastest" time/optimal solution then it is a NP-Complete problem. If you only want to make the AI achieve a "fast" time then there are other ways to get there. Remember, the NP-Complete designation is b/c it takes less time for the computer to verify the solution than it took for it to originally calculate/arrive at the solution.

 

[meekrab]

If you want to achieve the "fastest" time/optimal solution then it is a NP-Complete problem. If you only want to make the AI achieve a "fast" time then there are other ways to get there. Remember, the NP-Complete designation is b/c it takes less time for the computer to verify the solution than it took for it to originally calculate/arrive at the solution.

How would you even build a verification function for a fastest lap?

 

[Fennario]

Verification is just running the ai output lap and comparing against other times. Same as traveling salesman. That’s why it’s a beast of a problem.

 

[kungtotte_]

Sure, for a single time trial lap. We can make a computer go faster. But that’s easy. Now try and make it do a 24hr race at The Ring with a field of 200+ cars. The situational awareness, feel for tires/changes through a race. That’s the hard part.

Eh, some of those things are going to be easier for a computer to handle given their higher sensitivity.

It's going to be able to accurately measure exactly how effective the brakes are, how much the tires slip due to wear, etc. and be able to perfectly hit brake points and turn-in points the same way each lap which means it'll be able to calculate how to adjust for worn tyres and so forth.

Situational awareness down the track is going to be worse though, that's true. It won't be able to look far down to the next corner through traffic and see the spin that happened there and prepare for it. It won't be able to judge that the cars going two-wide for the next corner aren't going to make it. It won't take personalities into account, e.g. "the #23 car always goes for the aggressive inside pass".

Humans are better at that stuff. Humans are also better at making worse-is-better decisions like staying out one or two laps on worn tyres and running suboptimal laps on a gamble to come out ahead of an opponent after the pitstop. Or run fewer laps on the fresh tyres in the second stint hoping to push harder then.

AI is more likely to pick the optimal solution each time which might hurt it over a race.

 

[meekrab]

Verification is just running the ai output lap and comparing against other times. Same as traveling salesman. That’s why it’s a beast of a problem.

No, verification is confirming that a solution is correct. How do you propose to verify that a given lap is the fastest possible lap?

 

[The_Tullster]

Did you notice that the computer was slightly feathering throttle through the "flat" right handers between the tunnels to minimise required steering angle and maintain optimum speed? That's something that would be very difficult for a human to manage accurately for a few reasons:

• The computer has a much more accurate speed measurement available, so will notice much smaller variations in speed
• The computer can check speed more frequently without intefering with it's control and perception of track position
• The computer is able to control throttle setting much more precisely
• The computer is able to change throttle setting in response to changing situational information much more quickly (shorter reaction time)

Also, it's important to note that the computer didn't run a fast time first lap out. I haven't read the paper but it must have run quite a few laps in the learning process before achieving the superhuman level, though it had the advantage of playing on four PS4's simultaneously to reduce the learning time. Isn't multiboxing a breach of terms of service?

 

[breeminator]

No, verification is confirming that a solution is correct. How do you propose to verify that a given lap is the fastest possible lap?

https://math.stackexchange.com/ques...-verification-for-travelling-salesman-problem

i.e. it's the decision version that is NP-Complete, but this is not relevant to this thread. Finding the fastest lap is NP-Hard.

 

[stpatty]

The most interesting thing about this is how the AI version was playing without the gear indicator on-screen, which I didn't think you could turn off.

Also, pretty much every corner at Tokyo has walls along the side of the road marking the track limits. How would it perform in that regard on a regular race track with kerbs and white lines?

 

[Deadpool]

Eh, some of those things are going to be easier for a computer to handle given their higher sensitivity.

It's going to be able to accurately measure exactly how effective the brakes are, how much the tires slip due to wear, etc. and be able to perfectly hit brake points and turn-in points the same way each lap which means it'll be able to calculate how to adjust for worn tyres and so forth.

Situational awareness down the track is going to be worse though, that's true. It won't be able to look far down to the next corner through traffic and see the spin that happened there and prepare for it. It won't be able to judge that the cars going two-wide for the next corner aren't going to make it. It won't take personalities into account, e.g. "the #23 car always goes for the aggressive inside pass".

Humans are better at that stuff. Humans are also better at making worse-is-better decisions like staying out one or two laps on worn tyres and running suboptimal laps on a gamble to come out ahead of an opponent after the pitstop. Or run fewer laps on the fresh tyres in the second stint hoping to push harder then.

AI is more likely to pick the optimal solution each time which might hurt it over a race.

Eventually they will be easier for a computer to understand. But that will take a LOT of programming so that it will understand tire fall off, graining etc. I think that’s still a ways off, but yes it will be the easier of the hurdles. It also needs to visually calculate the tire degradation/grip of its opponents.

The situational awareness will also fall into creativity. Sometimes drivers on the limit will try new lines and creative ways to make a mass, Ricciardo/Verstappen being prime examples.

It will def get better at avoiding accidents, and having basic situational awareness, akin to say automated cars. But it will be focused on avoiding wrecks, which will limit its ability to go fast until they can handle the more difficult scenarios. It will get there one day, but def a ways off before it’s even remotely competitive over a race distance.

 

[TonyJZX]

Ironically just this last week AI fighter pilots just went like 6 - nil against hardened decades experience fighter pilots.

With the way Elon's autopilot is going we're clearly on the cusp of AI besting just about every 'human controlled' vehicle activity.