Cubic Motion, creator of true-to-life digital humans, has launched Persona, a new facial animation product that enables actors to perform as high-fidelity digital characters in real time.Persona continually captures and translates an actor's performance onto a computer-generated counterpart, enabling immediate animation in real-time engines such as Epic’s Unreal 4. Persona paves the way for a new era in digital performance and virtual production – an era where characters, actors and audiences all co-exist and interact.
“Our advanced computer vision techniques allow Persona to capture an actor's performance with stunning accuracy and detail. Resulting animated characters are able to respond and engage with real-world events immediately,” says Steve Caulkin, CTO at Cubic Motion. “Whether it’s for immediate broadcast, live installations, virtual production or offline animation, Persona will provide you with faithful replication of the actor’s performance.”
The use cases for Persona are virtually endless, from bringing game and movie characters onto the stage at live events, allowing digital characters and avatars to engage with fans in real-time online conversations, to providing the perfect previsualization solution for any virtual production.
Cubic Motion’s computer vision technology has already proven successful at a host of industry events, including GDC and FMX, and was the winner of SIGGRAPH 2016’s Real-Time Live showcase.
Components
Persona’s helmet-mounted camera system (HMC) and on-body animation system facilitate a completely untethered performance. The proprietary design is capable of delivering consistent, accurate capture over prolonged periods of time.
Using Cubic Motion’s signature front-facing and profile camera configuration – as well as adjustable, inbuilt infrared lighting – the HMC can accurately record even the most subtle facial expressions and poses. The onboard timecode electronics guarantee controlled motion capture, in-sync with all other components of production on a shoot.
At the heart of each Persona setup is an on-body computing appliance called the Performance Pipeline, which captures and solves facial performance onto a CG character. The Performance Pipeline is a lightweight, battery-powered piece of hardware that can be tethered or wireless.
To ensure faithful representation, the Performance Pipeline is configured to suit each actor and the character they portray. When combined with Cubic Motion’s Identity System, an engaging and intimate portrayal of the character is delivered. The pipeline also provides full capture capability, meaning that all video and animation data can be reused after live events or polished before the final broadcast.
Cubic Motion’s Identity System uses machine learning algorithms to match a purpose-built tracking model for the actor with a bespoke solver for the character rig – all in a form that is ready to be consumed by Persona. By analyzing both the actor’s facial features and capabilities of the character rig, Persona can interpret the unique qualities of this pairing and more accurately transfer a performance into the digital world.
Persona provides a GUI application called the Performance Manager, which allows users to remotely monitor each actor and configure their performance. After installing the application, actors are automatically detected and tracked. An array of tools are provided by the Performance Manager, simplifying everything from the alignment of the HMC to control over the animation stream. Users can even preview character animation before it is rendered.