The CG robot in Chappie learns how to think and feel
With Chappie, Director Neill Blomkamp continues his exploration into a dystopian near future set in South Africa. In this future, Scouts, which are robots with some intelligence, act as the police force. The story takes off when two misfit gangsters – Yolandi and Ninja (South African rappers Yo-Landi Visser and Ninja, respectively) – kidnap Deon (Dev Patel), the Scouts’ creator, to have him turn off the robots. But Deon has a bigger idea: He instills a damaged and discarded Scout with abilities beyond thinking. That Scout becomes Chappie, a robot that can think and create.
Albeit a full-human-sized killing machine, Chappie is born again as a child. The child needs training, so Daddy, Daddy, and Mommy, if you will, take on the parental roles. Daddy Deon, the inventor, is the good parent. Ninja, the gangster, the bad. And Yolandi is mommy, somewhere in the middle. Should Chappie be a policeman or a criminal?
Virtuous or not?
Chappie, the Scouts, and a human-directed rather than AI-directed bipedal mech called Moose are all CG characters created at Image Engine. Chris Harvey, the overall visual effects supervisor and the VFX supervisor at Image Engine, joined the production about six months before filming began, and spent about two years working on the feature.
The main challenge was creating Chappie, which would star in 1,000 of the 1,300 visual effects shots in the film. Sharlto Copley, who was Wikus van de Merwe, a bureaucrat who becomes a prawn in Blomkamp’s District 9 and was the agent Kruger who ended up wearing an exoskeleton in Blomkamp’s Elysium, played Chappie during filming and provided his voice.
Two things about the project are unique: First, Image Engine built a 3D model of a generic Scout from concept art provided by Weta Workshop, Weta Workshop printed the model for use on set, and then Image Engine adjusted the model to compensate for changes required for the 3D print. Second, although some reports suggest the opposite, Image Engine did not motion-capture Copley for Chappie’s performance.
Building the Robot
Work on Chappie’s body began at Image Engine during the months before production as the crew turned concept art from Weta Workshop into a 3D model of a generic Scout. That digital model would travel to Weta Workshop for printing and assembly into a full-scale, poseable prop that would be occasionally used on set.
Although the Scouts – and Chappie before his rebirth – would have robotic motion, Chappie would need to have Copley’s full range of motion, and yet be plausible as a robot made of metal.
“We knew that Sharlto [Copley] would perform Chappie,” Harvey says. “It was important that Chappie could move well. So, we spent a huge amount of time on joint articulation. And, every piece had to be manufactureable.”
For reference, the crew looked at prostheses, gearboxes, engines, motorcycles, hard drives, and real robots.
“It was important to Neill [Blomkamp] that everything we did was based on something,” says Barry Poon, asset manager at Image Engine. “The concept art from Weta Workshop didn’t go into fine detail, so we had to work out the internal parts in the 3D model.”
Five modelers worked in Autodesk’s Maya for four months to build the basic, generic Scout model that Weta Workshop would send to 3D printers, that Image Engine would augment to create the damaged Scout which becomes Chappie, and that Image Engine would duplicate to create the 100 police robots in the film.
The final model sent to Weta Workshop was made with 2,740 objects.
“Realistically, he’s more than that,” Poon says (see “The Many Parts of Chappie (And Moose),” page 24.) “Our 3D model of the palm of his hand is made of 20 pieces, because of all the mechanics we needed to move the fingers around, but Weta could print fewer. The external case for our hard drive on the [digital] robots would have to come apart, but they could print it as one piece.”
To print the model, Weta Workshop needed to thicken some of the parts, which they sent back to Image Engine to match.
“Anytime the Scouts were inactive, they wanted to film practical assets, so we wanted to match them as closely as possible,” Poon says.
Damage and Other States
The crew at Image Engine, meanwhile, turned the generic sculpt of the Scout into the CG robots that would appear in the film.
The 5,000 links in the chains gangsta Chappie wears move with rigid-body dynamics.
“You never see the CG version in a pristine state,” Poon says. “The robots all needed to look like they were in service for a year or two. When we gave the model to Weta, we included our thoughts on what materials to use per object. They did prints and paints on the practical robot. Then we took photo reference and created our materials to match, but pushed them a little more, trying to follow the practical lead as much as possible. We start with physically accurate materials and then art-direct as needed.”
Because the CG characters’ physical appearances change through the film, the artists created 16 versions of Chappie and eight of Moose. There is also a third robot in the movie, a spoiler.
“We see it in only three shots, but it’s really important,” Poon says. “It’s pristine. It was really important to Neill that the audience knows who it is.”
The third robot doesn’t change during the three shots, but Chappie goes through numerous changes in his 1,000. He takes an RPG to the chest, he has graffiti painted on him, he’s damaged in other ways, and in some shots he wears chains.
“We had one asset for Chappie with all the different damage states built in, so it was a lot of work to manage,” Poon says. “He had multiple legs, arms, and panels, and we had to track what was damaged, where, and when. We charted it out, but then animators would make a change. At one point, Neill has Chappie splash through a puddle. So, it was a lot of work for us to manage. We knew it would be difficult, though.”
The studio used Shotgun’s software for tracking, and within that tool, they tagged appropriate assets per shot so that lighters, for example, would know which version of Chappie and Moose to use. To help Shotgun and Maya communicate so that Maya would load the correct model and textures, the studio wrote proprietary software.
“Some damage states are driven by geometry and textures at the same time,” Poon says. “Some by texture only. If there is blood on Chappie or Moose, that would be a damage state that would pull the correct textures for the shot.”
The Scouts were identical; however, they, too, had wear and tear.
“We mainly changed their numbers and textures,” Poon says.
In 450 of the shots, Chappie, which has become part of a gang, wears gold chains made of 5,000 individual links, each depending on a full rigid-body simulation to bounce and jiggle as Chappie moves.
“The chains interact with one another, and people interact with the chains,” Harvey says. “And that was only a tiny aspect of Chappie. It was a massive undertaking.”
The effects team pre-set the simulations in Side Effects’ Houdini so that a first pass from animation could run automatically. After shot review, the effects artists could tweak the simulation as necessary.
Chappie had 16 stages and 12 sub-stages of damage and destruction.
“It was a slick process,” Poon says. “The chains had default settings, so the animators would run a pre-roll on the animation to go from static to the first frame, and then the chain simulation would run. The cool part was that the effects artists didn’t have to get involved until the first animation pass was done. It was a big investment up front to set up the system, but it was successful.”
The effects team also simulated the hoses hanging off Moose, but Chappie’s hoses moved via the rigging system.
Robots on Set
During filming, Copley performed Chappie. A stunt actor played Chappie when he was still a Scout and performed Copley’s stunts. And other stunt actors played the stiffer, more robotic Scouts.
“Sharlto [Copley] played Chappie loose,” Harvey says.
For the more tank-like robot Moose, a member of the Image Engine crew walked around with a tennis ball on a pole.
“We intended to have him in the shots only for rehearsals,” Harvey says. “But, the actors wanted him in the shots. So we had to paint our own guy out of the shots. We tried to be as smart as we could. We didn’t start roto paint until we had the first pass of animation. That way we could paint away only the part that needed to be removed.”
Although Copley and the stunt actors were on set, the crew decided not to motion-capture them.
“We did a lot of tests and decided we could better spend the money elsewhere,” Harvey says. “If we had 10 or 30 characters in a scene, it would have been prohibitive to animate them by hand, but we had one main character. The amount of time needed to clean up motion-capture data isn’t that much different than animating by hand, so there was no efficiency gain in terms of money. We could do the animation by hand as easily. And, the visual effects crew on set could be more streamlined and less intrusive. We didn’t have to calibrate an array of cameras; we didn’t slow anyone down. Neill shoots very, very fast.”
Copley wore a gray suit on set, but that was for lighting reference, not motion capture. Rather than motion capture, animators did rotomation.
“We had him in the back plate, and animators animated on top by hand to mimic his performance,” Harvey says, “and embellish it. They also added Sharlto’s performance to the stunt actors.”
The filmmakers shot the entire movie on location; there were no digital environments or digital extensions. However, to integrate Chappie into those environments, the crew created digital versions of all the locations. In all, the crew created between 65 and 70 environments with unique light rigs.
“We didn’t Lidar-scan the locations,” Harvey says. “We took hundreds of thousands of photos, used [Agisoft] PhotoScan to create the digital versions with photogrammetry, built models based on that, and then mapped the low-res geometry with HDRI photos. We didn’t take only one area, we took a series along an entire path to map out the entire volume. Then, we’d reproject onto this low-res digital environment, to reflect light back onto the digital character. That way, you could see the reflection move as he walked past a table, for instance. It gave a real dimensionality to the lighting.”
For compositing, Image Engine uses The Foundry’s Nuke; for rendering, DNA Research’s 3Delight. For lighting, the studio uses its proprietary software called Gaffer.
Articulation
On set, the filmmakers would pose and position the practical robots, so those 3D-printed props needed to articulate correctly. But for the digital robots, the animators needed a more complicated articulation.
“The shoulder and hip joints were the most difficult,” Poon says. “The robots needed to put their hands over their heads and across their chests.”
Although a CG robot’s joints could, theoretically, move with more degrees of freedom than a human’s, the crew restricted the movement to what a human could do. But, that didn’t make the joints easier to design, model, and rig.
“The robots needed to put their hands over their heads and across their chests.”
“Chappie has an incredibly complicated articulation,” Harvey says. “He isn’t a toy robot. He had to be a strong police robot, so every joint had to be a single axis for strength. For side-to-side movement, we have a separate joint.”
There are no ball joints in the robots. The shoulder joint comprises three joints; the knee, two. Bending metal wasn’t an option.
At Image Engine, a new IK solver managed the complex articulation, but the rigging process continued throughout the animation process.
Three riggers worked on Chappie, one at a time through the project, with help from R&D to develop new ways to move the metal parts.
“The animators saw pretty much the same system as always,” Harvey says, “but the system underneath is technically complicated.”
The big question was Chappie’s facial expressions. Animators accomplished facial expressions within a limited set of possibilities: an eyebrow bar, chin bar, ears, lights, and icons for eyes. “On any visual film, the goal is to make the character believable,” Harvey says. “We had made Chappie plausible, but had to go beyond to have people believe him as if he were a person. We had to create emotional arcs.”
Emotion
After shooting on the film ended in January, the Image Engine crew did postvis on the 1,000 shots with Chappie.
Animators used body language to create Chappie’s emotional performance.
“Neill set an aggressive goal for discovery on the character,” Harvey says. “We experimented with how the movement might work. And right after that, we picked three mini-sequences in the film to try to put a soul into the metal robot.”
The three sequences were when Chappie first wakes up, when he is beaten by street dudes, and a comedic scene.
“We knew if we could get him to work across those arcs, we would be successful,” Harvey says. “We worked on his antenna ears, his brow bars, his chin bar. We had lights that could blink if he was thinking or processing. We took the movement of Sharlto’s eyes in quiet, subtle moments, and applied it to all of Chappie’s head and had it twitch like an eye might. We really pushed the ear and eyebrow animation until it was almost cartoonish, to help the emotions read.”
On set, Harvey had encouraged the actors playing the robots to interact with the environment.
“It helps sell the tangibility,” Harvey says. “Any time we could get that interaction, we tried to use it.
It breaks that sense of disbelief. Chappie touches things, he interacts.”
By the end of April, the team had completed the three mini-sequences.
“Neill came to Image Engine with a couple of his people, and we screened these little clips,” Harvey says. “We had set the bar high, and you never know. But, Neill was very excited. We were all laughing. It was a really cool moment.”
“When I read the script, I had one idea of who Chappie was,” Harvey adds. “When I watched Sharlto, I had a whole new vision. He did such an amazing job. And then when we did the keystone tests, all the pieces came together – the design, Sharlto, animation. It was exciting to see all the pieces work, to see the emotions coming through.”
Even though Chappie didn’t have the $200 million budget of many visual effects films, the crew at Image Engine managed to create 1,000 shots with a believable CG character in a starring role, and even more remarkable, the character is a metal robot with an emotional arc. Imagine what Director Blomkamp will think of next.