DeepCoder Learns To Write Programs
Written by Mike James   
Monday, 07 August 2017

Deep Neural Networks are solving many difficult problems, but programming? Microsoft Research and Cambridge University think that it is entirely possible and are having some success - enough for us to panic? At the moment it looks as if our jobs are safe but ...

MSR

The work started a few years ago but the latest results have just been presented to the International Conference on Learning Representations (ICLR, 2017).

There are many problems with machine learning applied to programming but the biggest is that fact that the target output isn't differentiable. That is. a small change in the text of a program doesn't make a small difference to the output. In fact a small change to the output often makes the program invalid and so we go from something working, and perhaps effective, to something that is nonsense.

A common approach is to try to make the language input/output connection in some way differentiable. This is not the approach used in this work:

"We argue that machine learning can provide significant value towards solving Inductive Program Synthesis (IPS) by re-casting the problem as a big data problem. We show that training a neural network on a large number of generated IPS problems to predict cues from the problem description can help a search-based technique."

However, there is still a need to use a special simplified language - a DSL Domain Specific Language. This is something like SQL or LINQ and is capable of tackling real world problems. The functions it uses are restricted to working with integers or integer arrays. You can see from the example below the language is very "functional".

deepcoder1

Next the grammar of the language was used to generate a dataset of programs that connect inputs to outputs. The neural network was then given the input output sets and trained to predict the presence or absence of individual functions in the language.

This produced a neural network which could take the input-output sets and predict the characteristics of the program that produced them. This was then used to guide a search algorithm to find a program that produces the input-output sets. As well as working with programs of a fixed size, the team also tried training on shorter programs and then using this to find larger programs as a test of its generalizability.

The program worked in that it reduced the time to find tagets by 1 to 3 orders. In addition:

"We have found several problems in real online programming challenges that can be solved with a program in our language, which validates the relevance of the class of problems that we have studied in this work. In sum, this suggests that we have made significant progress towards being able to solve programming competition problems, and the machine learning component plays an important role in making it tractable."

So it might not be the end of your job, but it might be the end of fun with programming competitions of a particular type. 

MSR

More Information

DeepCoder: Learning to Write Programs

Matej Balog, Alexander L. Gaunt, Marc Brockschmidt, Sebastian Nowozin, Daniel Tarlow

ICLR (2017)

Related Articles

Front-End Developers Your Day Is Done - AI Can Do Your Job

More Machine Learning Applied to Google Sheets

Neural Networks for Storytelling

Automatically Generating Regular Expressions with Genetic Programming

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

 

Banner


Five Tips for Managing Hybrid Development Teams
08/03/2024

Managing hybrid development teams can be challenging, but  can also be a rewarding endeavor. Here are some tips to follow to ensure success. 



WasmCon 2023 Sessions Now Online
01/03/2024

The recorded session of the premier conference for technical developers and users interested in exploring the potential of WebAssembly are now online.


More News

raspberry pi books

 

Comments




or email your comment to: comments@i-programmer.info

Last Updated ( Monday, 07 August 2017 )