Developed by Google researchers VLOGGER AI is a system that can create realistic videos of people talking and moving from a single still image and an audio clip as input. 

As outlined by Enric Corona and his co-authors in the paper, "VLOGGER: Multimodal Diffusion for Embodied Avatar Synthesis", the technology relies on a pair of machine learning models to synthesize realistic video footage.  The first model predicts the body movements, facial expressions, and even blinks based on the audio. The second takes the predicted body controls from the first stage and uses a temporal diffusion model to iteratively refine each frame to generate a smooth, realistic video of the person talking:

Put together with generative AI and foreign language translation, there are obvious uses for VLOGGER in content creation and communication. There are of course concerns about the misuse of this technology, such as creating deepfakes.

More Information

VLOGGER: Multimodal Diffusion for Embodied Avatar Synthesis

Related Articles

Google Adds Gemini To Bard

Generate 3D Flythroughs from Still Photos

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Comments

Make a Comment or View Existing Comments Using Disqus

or email your comment to: comments@i-programmer.info