When British director Scott Mann’s latest film, Fall, was set to receive an ‘R’ rating from the MPAA for the number of ‘F’ bombs dropped during its one-hour, 47-minute runtime , he did what any reasonable person would: he used artificial intelligence to digitally alter the actor’s performance to change the swear words into more acceptable terms. A shot of whore genius, if you ask us.
For those curious: about 35 “F” words fell between a PG-13 rating and an R rating. Mann’s dilemma, then, became trying to figure out how to preserve the integrity of his film without redoing or double. It is, of course, more difficult than it looks.
On a traditional film, you would just bite the bullet and do reshoots. It’s expensive, but it’s better than having your audience lose immersion because they just heard a line delivered differently from what they clearly saw the actor say on screen. It’s much easier to read lips in 4K.
Greetings, humanoids
Subscribe to our newsletter now for a weekly roundup of our favorite AI stories delivered to your inbox.
But Fall is not a traditional film. It’s a horror movie about two women who scale a 2,000 foot radio tower. I won’t spoil anything, but it was practically shot. This means they built a giant tower and filmed their actors on top.
For Fall, Mann and the team at Flawless AI had to record the actors saying alternate phrases such as “damn” to replace the film’s vulgar language. Then they ran both the new audio and video through the company’s bespoke neural network, or what they call “True Sync” technology. This allows the team to, essentially, fake the actors’ faces when saying the new lines.
As for the results, with a “certified fresh” rating on Rotten Tomatoes, it looks like no one noticed the AI at work when they watched the drama unfold on the big screen.
The system is not only cost effective compared to reshoots, but, according to Mann, it is also relatively easy to insert into production. He told TNW via email that about 80% of the workflow is automated, and it was designed to handle everything through the mail — it doesn’t require any special equipment or direction during the film’s production. He even says that “vubbing,” or video dubbing, can save up to 50% on retakes costs.
But Flawless AI wasn’t founded just to help filmmakers save money in post-production. In fact, the main focus of the company is accessibility. According to Mann, “accessibility is very important to us because we believe in being able to share stories as they were meant to be and preserve the performances of the actors and the way the directors intended the story to be. told”.
The company was founded after Mann watched one of his previous films, Heist, with Rober Deniro, in a traditional dubbed format. He found the status quo methods of dubbing to be “limited and broken”.
“I saw how the traditional foreign dubbing hurt the film, as the writing, performance and meaning were lost,” Mann told TNW. “It led me to read a Max Planck Institute white paper on neural networks and ultimately led to the creation of Flawless.”
Going forward, Mann and the team at Flawless AI hope their work will allow storytellers to reach audiences around the world in more intimate ways. It’s one thing to know what an actor is saying because you hear a dubbed sound in a language you understand, it’s a totally different experience to see them deliver the line with all the nuance and character that has given it were given in the original performance.
We tip our hats to Mann and the Flawless AI team for using AI for good. Art is for everyone.
#Flawless #AIs #True #Sync #technology #game #changer #movie #dubbing