No Glittering Prizes For Creative Robots
Written by Sue Gee   
Wednesday, 25 May 2016

Can algorithms create "human-quality" short stories, sonnets and dance music sets? This was put to the test last week in a new variation on the Turing Test and the answer was a resounding "No".

The idea for the Turing Tests in Creative Arts came from Michael Casey and Dan Rockmore of the Dartmouth College Neukom Institute for Computational Science. 

Alan Turing's Imitation Game, which we now refer to as a Turing Test, was based on the premise that computers will be judged as intelligent once more than half of humans conversing with an entity that might be a man or a machine identify computer-generated conversation as human.

The question the Neukom Institute set out to explore with its annual Prizes in Computational Arts is whether people can distinguish between human and algorithmic creativity. The goal is not to replace human creativity, but to assess whether current computational technology can produce outputs that cannot be distinguished from it and to inspire innovations in computational methods that generate artistic products, such as literary, musical, and visual art.

For the inaugural year three competitions were organized:

  • DigiLit for short stories 
  • PoetiX for sonnets
  • AlgoRhythms for dance music DJ sets

Entries had to be in the form of computer code that generates novel works in one of these three creative modes. Submissions had to be received a month in advance of the Turing Test event for which a panel of highly distinguished judges convened.

Entries to the DigiLit were expected to be:

algorithms able to produce a "human-level" short story of the kind that might be intended for a short story collection produced in a well-regarded MfA program or a piece for The New Yorker. The Prize seeks to reward algorithms that could, for example, write stories for a creative writing class in which students are asked to submit a new short story each day.

Three completed entries were received so at the judging stage the three human judges received 11 short stories, eight human generated and three computer generated. One of the judges was fooled by a submission generated by an algorithm from Toksu and Ibrahim with the seed “thesaurus”. No 1st prize was awarded but a 2nd prize of $100 went to Judy Malloy for an algorithm that creates variations on a story by randomly arranging pre-written sentences into paragraphs.

The rules for PoetiX called for “traditional” sonnets: fourteen line poems, in iambic pentameter, in either “Shakespearean” or “Petrarchan” form. There were two entries and here is a poem from the competition winner given the seed "wave":

poetixsample

 

Each judge received 10 sonnets, six human generated and four machine generated (two from each of the two machines). No modifications were made by prizes were awarded. The top prize of $3000 went to Marjan Ghazvini Nejad, Xing shi, Yejin Choi, and Kevin Knight (USC/ISI, USA) for:

extraordinary programming effort that is a marriage of multiple AI techniques. 

Runners up Andrea Gagliano, Emily Paul, Kyle Booten, and Marti Hearst. (UC Berkeley, USA) received $1000 for "Pythonic Poet".

 

algoryth

 

The task for AlgoRhythms was autonomous performance of a 15-minute dance music set made using algorithms and eight entries were received. For the live test, which involved more than a hundred people at a dance party, three Human DJs and three Algorithm Finalists were hidden from view while audience danced and voted to 10 minutes of music from each. 

Although none of the algorithms passed the criterion of gaining more than 50% of the votes for Human, two got quite close with 39% and 38%. Two joint first prizes of $2,000 were awarded to Jaume Parera and Pritish Chandna (Music Technology Group, Upf, Barcelona, Spain) for DJ Codo Nudo and to  CJ Carr and Zack Zukowski (Medford, MA, USA) for DadaBots - Jungle Bot. A 2nd prize of $1000 went to Gerard Roma (Surrey, UK) for Dub Life, which 28% of the audience voted as "Human".

If you want to hear the 10 DJ sets and evaluate for them yourself there is an online poll for the AlgoRhythms competition.

So while poets and creative writers are not going to be replaced by robots any time soon, DJs might be beginning to feel at least a little threatened.

diglitsq

 

It will be interesting to see what happens when Neukom adds visual arts into the competition line up - there should at least be rather more entries as this seems to be a more popular form of creativity for computer generation, as evidenced by the results of the inaugural RobotArt.org contest we recently reported on. 

More Information

Turing Tests in Creative Arts

Looking for art in artificial intelligence

AlgoRhythms "DJ2RNGT3ST" On-Line Poll

Related Articles

Turing's Test, the Loebner Prize and Chatterbots

Loebner Prize Judges Could Easily Identify Chatbots

Passing The Turing Test Brings It Into Disrepute

A Better Turing Test - Winograd Schemas 

Replacing the Turing Test

Lovelace 2.0 Test - An Alternative Turing Test

Can Robots Create Art? 

 

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, FacebookGoogle+ or Linkedin.

 

Banner


Suspended For Claiming AI Is Sentient
15/06/2022

Can a large language model be sentient? This is a news item that has been doing the rounds and just about everyone has some comment to make about it.  But who better to comment than a man accused [ ... ]



MongoDB 6 Adds Encrypted Query Support
07/06/2022

MongoDB 6 has been released with improvements aimed at letting developers innovate more rapidly. To achieve this the new release has integrated features into the core database platform including appli [ ... ]


More News

 

pythondata

 



 

Comments




or email your comment to: comments@i-programmer.info

Last Updated ( Wednesday, 25 May 2016 )