Sony's AI Racer Wins By Being Nice
Written by Sue Gee   
Friday, 05 August 2022

Formula 1 Racing is an aggressive sport and competition can even become vicious. Through reinforcement learning Sony's racing AI, GT Sophy, has evolved a driving style that combines speed with "etiquette" and can outrace top human competitors. This breakthrough has now been recognized with an award.

ACMSIGAI

The award is the 2022 ASM SIGAI Industry Award for Excellence in Artificial Intelligence which carries a cash prize of $5,000 and a plaque presented at International Joint Conference on Artificial Intelligence held last month in Vienna, Austria.

This annual award from the ACM Special Interest Group on Artificial Intelligence goes to individuals or teams who have transferred original academic research into AI applications in recent years in ways that demonstrate the power of AI techniques. The projects must illustrate this through a combination of features, including the originality of the research novelty and technical excellence approach; the importance of AI techniques to the approach; and the actual or predicted societal impact of the application.

In view of the final stipulation you might be surprised that the award has been made to team that has developed an AI agent that excels in Gran Turismo, the well known PlayStation 4 racing simulation game. 

Gran Turismo Sophy is the first AI agent to emerge from Sony AI. which was founded in 2020 with the mission to:

"unleash human imagination and creativity with AI".

It initially came to public attention when it was featured on the cover of Nature in February which carried the article Outracing champion Gran Turismo drivers with deep reinforcement learning authored by Sony AI's 27-person team, led by Peter Wurman, Director of Sony AI America. 

GTSophynature

The paper describes how Sony AI trained agents for Gran Turismo that could challenge, and beat, the world’s best e-sports drivers.To quote from the abstract:

We demonstrate the capabilities of our agent, Gran Turismo Sophy, by winning a head-to-head competition against four of the world’s best Gran Turismo drivers. By describing how we trained championship-level racers, we demonstrate the possibilities and challenges of using these techniques to control complex dynamical systems in domains where agents must respect imprecisely defined human norms.

There were two stages in achieving this success. By June 2021 GT Sophy was able to outrace top GT drivers when it was a head to head race on a empty track. In this video Emily Jones gives a commentary of her experience of being beaten by GT Sophy and what she learned as a result: 

The tables were turned once GT Sophy lined up against multiple human drivers in July 2021. In this scenario the program was at times too aggressive, racking up penalties for reckless driving, and at other times too timid, giving way when it didn’t need to. Retraining was required and by October 2021 GT Sophy won the rematch with ease.

GTSOPYCONTROLS

The difference was due to adding a new dimension of driving skills. In addition to Race Car Control, the essential physics for car dynamics, racing lines and precision maneuvers and Racing Tactics, the split-second decision-making for passing and overtaking, training was provided in the Racing Etiquette deemed essential for fair play. To achieve this not only did the team have to find ways to encode the written and unwritten rules of racing into a complex reward function they also had to choose training examples for reinforcement learning that provided the correct balance of aggression and timidity.

Acquiring this skill is also what makes GT Sophy relevant beyond Gran Turismo. According to Wurman:

Etiquette between drivers on a track is a specific example of the kind of dynamic, context-aware behavior that robots will be expected to have when they interact with people, 

GTSophy2 

More Information

Sony AI

Gran Turismo Sophy

Outracing champion Gran Turismo drivers with deep reinforcement learning (Nature)

Related Articles

Why AlphaGo Changes Everything

Can DeepMind's Alpha Code Outperform Human Coders?

Robot cars - provably uncrashable?

The Triumph Of Deep Learning

 

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

 

Banner


Training and Mentorship Promote Job Satisfaction
02/08/2022

Results of the EDB Open Source Talent Survey 2022 indicate that employees value the opportunity to work with and be trained in cutting edge technologies. Security training is also highly prized.



Pythonic Code Automatically
27/07/2022

Python is different in that its community has a strong sense of what makes code "pythonic". It means that beginners and interlopers are easy to spot and easy to put down - "dude your code is NOT Pytho [ ... ]


More News

pythondata

 



 

Comments




or email your comment to: comments@i-programmer.info

Last Updated ( Friday, 05 August 2022 )