After several years leading the Open AI Robotics team, Wojciech Zaremba, one of Open AI's co-founders, has disbanded the team that had made significant advances with unsupervised learning, including solving a Rubik's Cube, and has moved on to other projects where more data is readily available.  

Back in October 2019 when we reported AI Wins At Rubik's Cube With Just One Hand, I Programmer Editor, Mike James asked:

What is the importance of this work?

and gave the answer:

We keep seeing examples of neural networks solving difficult problems, but mostly simulations or game playing. It must have occurred to you that a really interesting application would be to put a neural network into a robot and see how well it walks or runs or whatever. This is what has now been tried at OpenAI.

Our report went on to describe how Wojciech Zaremba's Robotics team at Open AI had used unsupervised learning to train a human-like robotic hand and its feat had been to solve a Rubik's cube. 

Now it transpires that this promising avenue of research is no longer being pursued at that the team undertaking it has switched to other projects.

This news, that was picked up by Venture Beat, emerged when Wojciech Zaremba was interviewed by Lukas Biewald for his Gradient Descent series of machine learning related podcasts from Weights and Biases. The topic of this podcast, aired in June 2021 was "What Could Make AI Conscious" and  about halfway through their conversation comes the exchange:

Lukas: Why did you choose to work on robotics

Wojciech: Actually here is a reveal ... as of recently we changed the focus at OpenAI and I disbanded the robotics team.

Lukas: Oh wow 

Recovering from his shock, Biewald asks why this decision was taken and  Zaremba explains that basically it was down to the fact that Open AI is making more impact in other areas where training data is available.

"So it turns out that we can make a gigantic progress whenever we have access to data. There [are] actually plenty of domains that are very, very rich with data. And ultimately that was holding us back in terms of robotics. This decision was quite hard for me. But I got the realization some time ago that actually, that's for the best from the perspective of the company." 

“The sad thing is, if we were a robotics company, the mission of the company would be different, then I think we would continue. I believe quite strongly in the approach that [the] robotics [team] took and the direction. But from the perspective of what we want to achieve, which is to build AGI [artificial general intelligence], there were some components missing.

So when we created robotics, we thought that we can go very far with self-generated data and reinforcement learning. At the moment, I believe that actually pre-training allows to give model 100X cheaper IQ points and then that might be followed with other techniques."

In response to further questions, Zaremba refers to the ease of training GPT-3, or other language models, on an unsupervised tasks such as next word prediction. This is an area where Open AI has made amazing progress as we reported in  The Unreasonable Effectiveness Of GPT-3 and Zaremba explains that the models required for this builds in the internal representation that allows the model to:

"off-the-bat solve many tasks."

In the case of robotics there is no equivalent data and that to make similar progress he thinks would require both a natural way to collect a lot of data and very powerful video models which would require:

"orders of magnitude more compute".

Zaremba said he had no regrets about Open AI's foray into robotics which resulted in really amazing technology of which he is very proud, but he explained that Open AI had initially started all sorts of projects without clarity about what they wanted to do: 

 And over the time, we got way more clarity and the amount we can increase the focus in different directions.

After this podcast which brought the disbandment of the robotics team to public attention, OpenAI spokesperson summed up the situation with:

"After advancing the state of the art in reinforcement learning through our Rubik's Cube project and other initiatives, last October we decided not to pursue further robotics research and instead refocus the team on other projects. Because of the rapid progress in AI and its capabilities, we've found that other approaches, such as reinforcement learning with human feedback, lead to faster progress in our reinforcement learning research."

More Information

 Open AI Disbands its robotics research team (Venture Beat)

Related Articles

AI Wins At Rubik's Cube With Just One Hand

OpenAI Five Dota 2 Bots Beat Top Human Players

Open AI And Microsoft Exciting Times For AI 

The Unreasonable Effectiveness Of GPT-3

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Comments

Make a Comment or View Existing Comments Using Disqus

or email your comment to: comments@i-programmer.info