An algorithm for face transfer
Written by Lucy Black   
Wednesday, 21 September 2011

With a little work you can take a photo of a face and paste it directly onto another face in a video. The result is that videos can now easily be made to lie.

Sometimes you just need a creative spark to notice that something can more or less be achieved with the technology we have to hand. Face detection/recognition is currently being used for all sorts of applications, but most of them are fairly obvious - security, photo tagging, photo cropping etc.

One really clever idea has been thought up by artist Arturo Castro and he has implemented it by borrowing on other peoples implementations of face detection and so on - after all one of the joys of software is that it is reusable. The idea is surprisingly simple. Using a face detection algorithm he was able to locate the position of a face in the video stream. Using this information he then used a photo to paste another face to the same location.

 

facepasteThe video Mao Tse-Tung never made.

 

The really clever part is that, as face detection locates the various regions of the face - eyes, mouth etc, this can be used on the photograph to distort the 2D face to the configuration of the face in the video. The face tracker software returns a mesh that can be used to mould the pasted face to fit.  What this means is that when the subject moves their mouth or eyebrows, the pasted image is distorted to fit. The result is that it appears that the pasted face is animated.

In Arturo's video below the pasted face is minimally blended with the live video and so there are artefacts that indicate that the video has been modified. In the second video by Kyle McDonald the blending has been improved and there is a much smoother transition between the pasted and original face. The overall effect is smoother but in many ways it seems to be more disturbing.

 

The original Arturo Castro video:

              

 

The smoother Kyle McDonald video:

 

 

While this is all great fun, there are some obvious things that this could be used for beyond just art installations. It clearly wouldn't take very much tweaking to make the result look naturalistic and then the possibility of producing videos impersonating people becomes all too easy. Now videos can lie just as easily as a photo.

More Information

Arturo Castro's website

The software used:

​FaceTracker

ofxFaceTracker addon

Openframeworks​

 

To be informed about new articles on I Programmer, subscribe to the RSS feed, follow us on Twitter or Facebook or sign up for our weekly newsletter.

 

Banner


The Canvas Fingerprint - How?
29/07/2014

There is currently a lot of fuss going on about an additional method of fingerprinting browsers to track users as they move from one website to another. The technology is very simple but the real ques [ ... ]



Facebook Cup 2015 Scheduled For January
15/08/2014

The dates have been announced for the 2015 Facebook Hacker Cup contest  and registration is now open. All the online rounds will take place in January 2015.


More News

Last Updated ( Wednesday, 21 September 2011 )
 
 

   
RSS feed of news items only
I Programmer News
Copyright © 2014 i-programmer.info. All Rights Reserved.
Joomla! is Free Software released under the GNU/GPL License.