With only 9 visual effects artists, the team behind “Everything Everywhere All At Once” harnessed artificial intelligence to execute many complex film scenes.
With the robust development of artificial intelligence, AI-supported technologies have infiltrated every nook and cranny of modern smartphone and computer user life.
From chatbots that can answer anything users ask to tools that turn a few commands into works of art, AI has now ventured into the film production industry, becoming an invaluable assistant for visual effects artists.
Few would guess that a blockbuster like Everything Everywhere All At Once had only 9 visual effects staff. (Photo: A24).
Oscar Nomination with Just 9 Visual Effects Artists
When visual effects artist Evan Halleck worked on scenes for Everything Everywhere All At Once, he used the most advanced artificial intelligence tools from Runway.
This is one of the AI companies that created Stable Diffusion – an AI model that generates images from text content. In February, the company introduced a completely new technology that transforms text into video using AI.
Specifically, Halleck shared that the film crew faced numerous challenges, even with the simplest and smallest scenes.
“We used pulleys to push rocks, gravel, and sand forward. But when I cut those elements out of the footage, everything looked blurry and unclear,” Halleck revealed.
Meanwhile, the visual effects team was not only short-staffed (with only 9 members) but also had to work under tight deadlines. Therefore, he sought alternative solutions to expedite the process and reached out to Runway’s CEO, Cris Valenzuela.
From that point, the film’s visual effects team implemented a background removal tool to make it easier to place the main objects of the scenes onto green screens.
“AI cuts subjects out much better than the human eye, and it gives me a clean-cut image that I can use for other tasks,” Halleck stated.
Halleck affirmed that Runway’s AI tools optimized his work. “I can cut characters and arrange them neatly in a photo in minutes instead of spending half a day,” the expert shared.
He believes that the application of AI in the entertainment industry is growing significantly. From the development of deepfake technology to AI being used to create scripts, artificial intelligence is gradually infiltrating film production work.
“It’s faster and cheaper than manual labor. In my view, visual effects is a process that is very time-consuming and labor-intensive. Therefore, it’s great that everything can be automated,” he continued.
Expert Halleck also admitted that he wished he had discovered a tool like Runway sooner. In the scene where character Evelyn, played by Michelle Yeoh, glides through multiple universes in a few seconds, all 300 frames were processed manually. This consumed the crew’s time for several weeks.
“I look back and wish we had AI when we were working on those scenes, instead of spending weeks working in Photoshop,” the visual effects artist shared.
AI is Changing Film Production
CEO Valenzuela witnessed the potential of Runway while executing projects for brands like New Balance, singer Alicia Keys, and even the show The Late Show with Stephen Colbert.
“More and more filmmakers and visual effects artists are adopting AI. They use it not only for speed but also because it automates the tedious and time-consuming aspects of video and film creation,” Valenzuela remarked.
The use of AI, especially in visual effects, does not necessarily mean automating everything. Valenzuela noted that in the case of Everything Everywhere All At Once, he had to work closely with Halleck to deliver the clearest shots.
Valenzuela believes AI will help many small film crews produce top blockbuster hits. (Photo: Zohar Lazar).
“AI is revolutionary because it saves time and costs, allowing a focus on creative ideas. It reduces work that used to take days to just a few minutes,” he commented.
Although AI has only been widely applied recently, this technology has been around for some time. The creators of the controversial animated series South Park, Matt Stone and Trey Parker, produced an entire episode using deepfake video of former U.S. President Donald Trump.
“They are paving the way. There will be many amazing applications of artificial intelligence, and that can scare some people, but any new technology does that,” Runway’s CEO shared.
“I call it Hollywood 2.0, where everyone will be able to make films and blockbusters that only a select few could create before,” he added.
