ADELAIDE, SOUTH AUSTRALIA—Rising Sun Pictures (RSP) has added two post-doctoral researchers from the Australian Institute for Machine Learning (AIML) to lead a new initiative centered on the application of Artificial Intelligence (AI) to its visual effects pipeline.
Dr. John Bastian and Dr. Ben Ward will develop pipeline tools based on machine learning and other AI applications as a means of streamlining visual effects (VFX) production and creating visuals that look more photo-realistic.
Artificial Intelligence is widely regarded as having the potential to revolutionize VFX production and RSP is committed to working at the forefront of this new technology. “AI holds great promise for visual effects applications, especially in terms of accelerating labor intensive tasks and augmenting human creativity,” says RSP’s Managing Director Tony Clark. “Our early work with AI has produced spectacular results and we are eager to push development further.”
Working through AIML, which is associated with the University of Adelaide, Bastian and Ward recently collaborated with RSP on visual effects sequences for Marvel Studios’ current box office hit Shang-Chi and the Ten Rings. They developed a novel technique for facial replacement that was used in high-intensity martial arts combat sequences. Machine learning tools were employed to train neural network models of principal actors and apply them to the faces of stunt doubles. The process proved significantly faster than traditional means of facial replacement and produced more lifelike results.
At AIML, Bastian and Ward have been post-doctoral researchers involved in researching and developing machine learning and computer vision technology for three years. They have contributed to projects related to medicine, defense, agriculture, manufacturing, environmental sustainability and general science. Both men earned their PhDs in computer science from the University of Adelaide.
In their new roles, Bastian and Ward will work with RSP’s creative team in applying AI tools to specific production challenges. They are currently involved in three film and series projects involving facial replacement applications like those used in Shang-Chi. They will also seek to develop general workflow tools for such routine visual effects in both 2D and 3D, AI can be a means of relieving artists of tedious, labor intensive tasks, speeding up the delivery of visual effects and controlling costs.
“Our focus is on building AI-tools that can be integrated into RSP’s existing workflow to overcome the limitations of current technology,” Ward says. “It’s about helping artists improve results, achieve outputs faster and push beyond what was previously possible.”
Bastian says that AI should be viewed as a means of empowering artists. “AI can help artists accomplish incredible artistic effects without the tedium of executing every frame in a long sequence,” he explains. “Facial replacement was a good place to start because faces are largely rigid, but we’ll soon be able to do the same with the human body. We’ll be able to make an actor taller, shorter, skinnier, or more muscular. With AI the possibilities are limitless.”