Researchers have created a video-modifying program that transfers the expressions of a subject onto live video of another subject’s face in real time. The video warrants cliché: it has to be seen to be believed.
Video system lets you control another human’s expressions in real-time
The project is a collaboration between researches from University of Erlangen-Nuremberg, Max-Planck-Institude for Informatics, and Stanford University. Footage of the project was been released in September with a paper titled “Real-time Expression Transfer for Facial Reenactment.” Here’s how the abstract describes what makes their system unique:
The novelty of our approach lies in the transfer and photorealistic re-rendering of facial deformations and detail into the target video in a way that the newly-synthesized expressions are virtually indistinguishable from a real video.
For those who’d like to know the nitty-gritty, the paper is worth a read, and written in a refreshingly accessible fashion. The video speculates about use-cases, like a translator applying their expressions and voice over their subject, or someone in casual clothes projecting onto a source in formalwear. But what I see is an opportunity for Disney to using a blank-faced Robert Downey Jr. wax model as a target model to produce Iron Man movies for the rest of time.
How exactly would Disney do this? I’m glad you asked.
Follow topics and authors from this story to see more like this in your personalized homepage feed and to receive email updates.
