“It was important for the Keeper creatures to take on an insect-like aesthetic, and we spent several days in performance-capture workshops with Lana and Andy [Wachowski] directing gymnasts in mocap suits to try a variety of poses and movements to really flesh that out,” says Gary Marshall, lead motion-capture TD as well as real-time operator for on-set compositing.
To this end, Framestore’s real-time retargeting and rendering of those performances from Vicon’s Blade into Autodesk’s MotionBuilder gave the directors instant feedback on the character models. “We could experiment with inverting limbs and heads, for example, all in real time, to really create some nightmarish looks,” says Marshall. “These sessions proved extremely useful for driving the final look of these creatures.”
Framing and eye lines in scenes with numerous CG characters and sprawling environments proved difficult. To ease the complications, the animators worked with the Ncam camera-tracking system on set, which provide real-time compositing services of live action and CG elements. As Marshall points out, the challenge was the animation elements that had to be dynamically triggered to respond to certain events on set, in the context of the shot.
“I wrote a suite of plug-ins in Autodesk MotionBuilder to cue, trigger, and warp animation events – be it creature motion, spaceships taking off, or actors on motion paths. All this could be controlled from a MIDI input device, with events mapped to buttons and time-warping mapped to a jog wheel,” explains Marshall. “Overall VFX supervisor Dan Glass could then radio to us when he wanted the events to fire as the shot was rolling. This worked out very well for us.”
Framestore Supervising Mocap TD Matt Rank captured facial animation on set with four 720p cameras mounted to a helmet rig (a Vicon Cara system) whenever the hero Sargorn character, Greeghan, was on set. Although final facial motion was handcrafted, this data proved to be valuable reference for seeing exactly what the actor had done at the time, especially in light of the creature’s unique facial anatomy.
“When capturing the Sargorn creatures, we had mocap actors on stilts (heavy-sprung lower leg attachments), which forced us to really think about how the leg should solve. It was more a dog-leg kind of anatomy, so we had to get creative with our marker sets and make sure the resulting motion was faithful to how Lana and Andy had envisaged it,” says Marshall. “Fortunately, with Vicon's Blade software, we weren't locked into a rigid template for marker solving, so we could experiment with how we captured that motion.”
Using real-time mocap techniques, the group could set up “virtual mirrors” so a mocap performer could see himself as his creature avatar, to get a real feel for how the character should move. This was done by projecting a render from a virtual camera in the real-time engine on a wall, allowing the actor to see himself reflected back.
“This show felt like it became a real effort to progress the blurring of lines between production and post — with real-time compositing, on-set postvis, and motion capture all enabling the directors to tell the story they are trying to tell with more realism and more efficiency than we have seen in the past,” says Marshall.
Linda Romanello is the managing editor of Post Maagazine, CGW’s sister publication.
See a detailed story about the boardroom action scene and building the spaceship in the March/April 2015 issue of CGW.