“We've created many humans, digi-doubles and superheroes in the past, but Kiyan was on a different level. Creating Kiyan as the 30-year-old man he would have been today was a sensitive and delicate process, and we took our responsibility seriously,” says Woolley. “Making sure we did Kiyan and his family justice by realizing him as he would have been today was the biggest challenge of this project and, quite possibly, of all my projects to date.”
As the end sequence for the promo film wasn’t concepted until after the planning for Kiyan’s likeness had been completed, the creative team was left with no time to go down the traditional and more laborious digi-double route. Alongside this, there were no source images of Kiyan at 30 to train the deep fake on, either.
To pull off the promo’s end sequence, Framestore’s creative technologist, Johannes Saam, partnered with Sydney-based Matt Hermans, founder of the creative company Electric Lens Co., to work on the promo video assets and began the challenging task of bringing life and movement to 30-year-old Kiyan. Using a mixture of cutting-edge development and some secret sauce, the team was able to incorporate Scalf’s stills into the process to create synthetic data, which they were able to train the deep fake on.
“Deep Fakes are often seen as gimmicks, to service a trend, but when we heard about the project and the potential to use deep fake for good, we knew we had to find a solution, employing new neural rendering-based techniques,” says Saam.
Based on a first pass of the geometry and painted reference frames, a virtual dataset was assembled with an automated pipeline rendering 6,000 pictures as a base layer, which was then iteratively refined.
“The first pass added facial animations to the training material. After that, the pipeline was fed a detailed lookdev pass with facial hair and additional texture detail to keep the machine in line with our 2D likeness,” explains Saam. “Based on this dataset, we used a pre-trained neural network and the Kiyan-specific images to focus the deep fake network on translating the human facial performance into a digital representation of Kiyan.”
This network replaced the entire head frame-by-frame and was then composited over the photography, as well as removal of sponsor logos from the shirt using Foundry’s Nuke’s brand-new CopyCat node.
All proceeds raised will go directly to Kiyan Prince Foundation, a charity run by Kiyan’s father, Dr. Mark Prince OBE, who’s been tirelessly campaigning to prevent knife crime since his son’s tragic death..