Real-time Face Animation
Written by David Conrad   
Saturday, 19 March 2016

Since Photoshop you haven't been able to trust a still photograph. Now it seems you can't trust a video. Real-time face capture and reenactment allows one face to control another so that you can make anyone say anything with any expression you care to pull. 

facere

 

There have been facial reenactment systems before, see Create Your Favourite Actor From Nothing But Photos, but this is the first time that there is the ability to put words into another person's mouth and an expression onto their face.

The system works by taking a video stream featuring the target person and modifying it based on the facial expression of a source actor. So, if the source actor smiles, the person in the target video stream smiles - even if they didn't in the actual video stream.

 

facere2

 

 

The easiest way to fully understand what this means and what potential it might have is to watch the following video. Notice that the expressions of the source actor appear on the target in the reenactment video.

 

 

What is impressive is that it works in real-time with a cheap camera. The way that it works is fairly straightforward - detect the face and the features in the source actor's video and in the target video, then distort the target's facial features to match thos of the source actor. Easy to explain but very difficult to do well. 

Given some of the subjects used in the video as the test subjects - President Putin, Donald Trump, President Obama and so on -  you can start to imagine what mischief the system could be used for.

Want to change a polical speech?

Very easy.  

 

facere3

 

Of course, there are more honestly useful applications for this sort of technology. You could redub films in different languages and have the actors seem to say the new dialog. It might even be better to get the actors to do the job without saying a word in any language and then add the dialog later. You could also correct actors errors in expensive staged stunts.  

It used to be the folk law that the camera never lies.

Now video cameras can join still cameras in the role of excellent liars. 

More Information

Face2Face: Real-time Face Capture and Reenactment of RGB Videos Justus Thies, Michael Zollhöfer, Marc Stamminger, Christian Theobalt, and  Matthias Nießner 

Related Articles

Create Your Favourite Actor From Nothing But Photos 

Watching Paint Dry - GPU Paint Brush

Better 3D Meshes Using The Nash Embedding Theorem

3-Sweep - 3D Models From Photos

Time-Lapse Videos From Internet Photos

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Banner


Android Studio Iguana With Crash Reports
05/03/2024

Google has announced that the latest version of Android Studio, Iguana, is now stable. It has version control system support in App Quality Insights and new built-in support for creating baseline prof [ ... ]



iOS 17.4 Released With Support For App Stores In The EU
06/03/2024

I have written about Apple's approach to complying with regulation, characterizing it as malicious compliance. It also seems that Apple is a master of creating the unintended consequence and letting i [ ... ]


More News

 

raspberry pi books

 

Comments




or email your comment to: comments@i-programmer.info

Last Updated ( Sunday, 03 June 2018 )