An Algorithm For Face Transfer
Written by Lucy Black   
Wednesday, 21 September 2011

With a little work you can take a photo of a face and paste it directly onto another face in a video. The result is that videos can now easily be made to lie.

Sometimes you just need a creative spark to notice that something can more or less be achieved with the technology we have to hand. Face detection/recognition is currently being used for all sorts of applications, but most of them are fairly obvious - security, photo tagging, photo cropping etc.

One really clever idea has been thought up by artist Arturo Castro and he has implemented it by borrowing on other peoples implementations of face detection and so on - after all one of the joys of software is that it is reusable. The idea is surprisingly simple. Using a face detection algorithm he was able to locate the position of a face in the video stream. Using this information he then used a photo to paste another face to the same location.

 

facepasteThe video Mao Tse-Tung never made.

 

The really clever part is that, as face detection locates the various regions of the face - eyes, mouth etc, this can be used on the photograph to distort the 2D face to the configuration of the face in the video. The face tracker software returns a mesh that can be used to mould the pasted face to fit.  What this means is that when the subject moves their mouth or eyebrows, the pasted image is distorted to fit. The result is that it appears that the pasted face is animated.

In Arturo's video below the pasted face is minimally blended with the live video and so there are artefacts that indicate that the video has been modified. In the second video by Kyle McDonald the blending has been improved and there is a much smoother transition between the pasted and original face. The overall effect is smoother but in many ways it seems to be more disturbing. 

The original Arturo Castro video:

Faces from arturo castro on Vimeo.

 

The smoother Kyle McDonald video:

Face Substitution from Kyle McDonald on Vimeo

While this is all great fun, there are some obvious things that this could be used for beyond just art installations. It clearly wouldn't take very much tweaking to make the result look naturalistic and then the possibility of producing videos impersonating people becomes all too easy. Now videos can lie just as easily as a photo.

More Information

Arturo Castro's website

The software used:

​FaceTracker

ofxFaceTracker addon

Openframeworks​

 

To be informed about new articles on I Programmer, subscribe to the RSS feed, follow us on Twitter or Facebook or sign up for our weekly newsletter.

 

Banner


Julia 1.1 Adds Array Alternative
17/10/2024

Julia 1.1 has been released with improvements including a new Memory type that provides a lower-level container as an alternative to Array.



Tesla's Optimus Robot Expected To Cost Less Than $30,000 ... Eventually
18/10/2024

No fewer than 37 Optimus Robots mingled with guests at last week's We, Robot event and the audience were treated to the spectacle of around seven of them performing a dance routine. Elon Musk also mad [ ... ]


More News

 

Last Updated ( Sunday, 14 June 2020 )