He’s little, but this boy with a thousand faces has a mighty presence in a big, wide world. Eight thousand faces, and more than a million expressions, in fact, when animators arrange and re-arrange Norman’s tiny mouth and eyebrow pieces like a jigsaw puzzle to create happy faces and sad faces, and lip-sync his dialog. As for the big, wide world, we see the star of Laika’s stop- motion feature ParaNorman at home and at school, in chase sequences through a well-populated town, in a graveyard and a forest, and a variety of other locations.
Visual effects artists painted out the seams caused when animators applied face parts to change Norman’s expression.
“Historically, stop motion has felt like it was shot on a tabletop,” says Laika President, CEO, and Animator Travis Knight, who received Annie and VES nominations for animating the star of Coraline. For ParaNorman, Knight was producer and lead animator. “We wanted ParaNorman to feel more expansive. We did that through set design and animation performances.”
Computer graphics tools and techniques helped make this possible in two ways. First, designers worked with Autodesk’s Maya, Pixologic’s ZBrush, and Adobe’s Photoshop to shape and color facial expressions before sending digital models to 3D Systems Corporation’s full-color ZPrinter 650 for outputting the face parts.
Second, the visual effects crew used a combination of Maya, Side Effects Software’s Houdini, and Pixar’s RenderMan to extend backgrounds, swirl witch-infested tumultuous skies, populate the town with CG puppets, and, in general, help make it possible to create an action/adventure film in stop motion. Compositors put it all together with The Foundry’s Nuke.
Pushing the Envelope
The Focus Features comedy tells the story of an 11-year-old boy named Norman, who is normal in every way but one: He can see and talk to the ghosts of dead people. And, that makes Norman an outcast. Norman lives in the town of Blithe Hollow, which was the site, 300 years earlier, of a famous witch-hunt. He has one friend, the chubby Neil, also an outcast, and both boys suffer bullying. Even at home, Norman is misunderstood. He makes his father angry when he talks to his grandmother’s ghost. His older sister is annoyed, his mother forgiving. Norman doesn’t care.
He bonds with Neil by talking to the ghost of Neil’s dog, which Norman can see but Neil can’t. He pins zombie badges on his backpack, and decorates his room with zombie posters. When zombies invade Blithe Hollow and ghosts rise from graves in response to a centuries-old curse, Norman becomes a hero. “The epic finale of the movie is an amazing blend of old tech and new tech, 2D, stop motion, CG, compositing,” says Sam Fell, who directed the movie with Chris Butler. “That was a high point in terms of pushing the envelope. And, we have a dynamic chase sequence that tops every action seen in stop frame. Action was a big ambition. I thought at some point we’d be reined in, but we never were.”
“We had a very ambitious style of acting, with extreme close-ups and reaction shots,” adds writer/director Butler. “We wanted to do proper filmic acting, not just pose to pose. Part of the reason we were able to achieve that is we asked a lot of our animators on set. But, the technical innovations of face replacements gave us a degree of acting in the faces we haven’t seen before. Pushing the envelope is almost Laika’s brand. Push and push and push, and if can’t be done, that’s a reason to try.”
Face Off
Laika first used rapid-prototyping machines to print small face parts for Coraline. Animators popped mouth and brow shapes onto Coraline’s face, rather than create her facial expressions by sculpting pliable silicon models with armatures inside.
Rapid-prototyping printers are similar to ink-jet printers, but rather than applying ink to paper, multiple print heads spray a UVsensitive resin in layer after layer onto a powder-based supporting material to build the model.
“They look like sugar cookies from the oven when they emerge,” says Brian McLean, creative supervisor of replacement animation and engineering, and director of rapid prototyping. The crew removes the face parts from the supporting material, and cleans and sands them. Then, rinses and repeats. Thousands of times.
For Coraline, the groundbreaking technique worked so well that the studio took it to another level for ParaNorman, creating more parts for more characters, and using the machine for new purposes.
Coraline, for example, had approximately 200,000 potential facial expressions created with combinations of mouth and eyebrow parts. Norman has 1.5 million. For Coraline, artists painted color onto the parts. This time, the crew used full-color printers that could build color into the model. They put the printers to work creating props as well as face parts. And they even printed visual effects. For example, to simulate motion blur, they printed Norman’s face with his nose in triplicate.
Making Headway
Each character starts as a pencil design on paper, followed by interpretations drawn in Photo shop. Once the artists and directors narrow the choices, modelers create clay maquettes in key poses for each character, and the armature department begins working on the tiny skeletons with ball and socket joints that will fit inside the bodies.
fie heads and faces for ParaNorman’s characters took two shapes. One, used for the zombies and some background characters, had silicon Thesh with a mechanism inside, a traditional method of creating facial expressions, albeit with new, softer silicon over foam latex for added Thexibility and longevity.
fie second technique, used for the main characters, employed the use of rapid-prototyping machines to print face parts. Much as they would if they were creating a CG character for a film, the artists began the process by scanning the head of the clay maquette. “We captured tool marks in the clay, facets, everything that made the clay sculpt handmade,” McLean says. CG modelers created the rough topology from the scan in Maya, moved the result into ZBrush to exaggerate the details, and sent the file to the printer for a test run.
“The printer softens edges, grooves, and valleys,” McLean says. “We needed to see what to push in terms of color and texture. We would never want to show the directors the ZBrush model or a rendering of that file because it’s so exaggerated.”
Once the printed head received an artistic buy-off, the dense ZBrush mesh moved into PixelMachine SRL’s TopoGun software, where modelers redrew it with a lower-resolution topology.
Digital skies and set extensions helped set builders add details and open Norman’s world beyond the tabletop.
“We produce each face thousands and thousands of times,” McLean says. “If the topology is too dense or the file size too great, we can’t produce very many parts at one time. 3D printing is a new art form; there’s no established workThow. Our rigging supervisor, Michael Laubach, had to figure out how to produce high-level detail with low file sizes.” At this point, a character’s CG head is still a complete shell; there are no removable parts yet. fie shell moves in two directions—to two different groups working with Maya on the inside and outside of the characters’ heads. For the outside, a CG modeler creates face masks. For the inside, a CG modeler designs and engineers the elaborate mechanical system. For Norman, it had 78 parts that animators could set in motion to move his eyes, eyelids, and ears. Some parts are printed plastic. Others are tiny metal bolts and screws.
“People often think we’re printing whole faces and eyes,” McLean says. “But we build heads to be transformers.” Norman’s face is a mask held in place by magnets, as are all the characters’ faces that use this system. When an animator removes the face mask, Norman’s eyeballs are still inside the shell of his head. Animators move his eyes using the internal mechanical system. They close his eyelids and move a blade of his hair with an X-Acto blade. Once moved, the parts stay in place until the animator moves them again.
On the outside, eight thousand faces composed of snap-together eyebrows, foreheads, mouths, and cheeks give Norman his facial expressions. His buddy Neil has 3,000. “We build the faces in kits,” McLean explains. “One series of faces in a kit might allow Norman to say any line of dialog with a smile. With another kit, he could say any line of dialog with a frown. We create each of his emotions with phonemes to build a foundation for the whole film.”
Making Faces
To build the kits, Laubach rigs the CG characters in Maya. fie rigs give animators the ability to do poses and create each character’s range of facial expressions, as they would with any CG character. fie difference is that Laubach must be cognizant of printing limitations.
“We print physical objects, so there are real-world limitations,” McLean says. A lip stretched too thin might break when printed, for example.
CG animators shape the facial expressions by moving control points predetermined by the rig, working from dialog samples and 2D drawings that provide a blueprint for each character’s range of expressions and phonemes. “They’ll take a line of dialog from Norman with a good range, for example, and use that to build his kits,” McLean says. “They digitally sculpt his faces. fie difficult part is making sure each shape is unique and will be used in the film. They have to be as economical and considerate of printing as possible.”
Before a stop-motion animator puts a Norman puppet into a set, he or she builds expressions for the shot using the CG face kit. “fie animator will sit with a facial animation specialist,” McLean says, “someone who knows every face. They listen to the dialog together and create a playblast with the digital faces.”
Animators typically move the bodies on “ones” and faces on “twos.” fiat is, they position the body in a unique pose 24 times per second, and change face parts to create facial expressions 12 times per second.
“Animating on twos gives us more pop,” Knight says. “If we want more subtlety, we might animate the faces on ones.”
When the animator and facial animation specialist have assembled all the digital face parts into the series of expressions they want—perhaps of Norman screaming as he runs from zombies, or frowning, laughing, or smiling— they show the playblast to the lead animator or director. “If he approves the playblast, we create a shopping list of face parts,” McLean says. The shopping list goes to a face librarian, who picks all the physical face parts for the shot. These go into a box that the animator takes to the set, along with an X-sheet that lists expressions and face parts frame by frame.
“Because this might be the first time these faces appear in sequence, we run tests to be sure the color and registration are accurate,” McLean says.
At left, the physical tornado and layers of tulle (top) created and manipulated in the art department provided reference for the CG stormy skies in the image above. In that shot, when the witch moves her face, volumetric CG clouds react.
Printing Makeup
Color was the major difference between the process used at Laika for ParaNorman and Coraline. “For Coraline, we used the plastic printer,” McLean says, “the same one we use now to print eye parts. So we hand-painted her face. That’s why she has five freckles on each cheek. We needed to keep the paint really simple.”
When 3D Systems’ new color printers became available, they considered new possibilities. “Color printing had been around for a while, but the new machine had rich blacks and white whites, which had been the Achilles’ heel,” McLean says.
The studio decided to test the printers with a little model of a zombie head. “When we got it back, it looked fantastic,” McLean says. “The yellows were vibrant. The greens, consistent. We purchased the machine.”
Thus, rather than hand-tinting each face part, the artists could use Photoshop to paint one digital part and print multiple copies in color. They would soon learn there were other advantages as well.
It wasn’t easygoing in the beginning, though. When they first tried printing faces, the skin tones looked terrible. What the painters saw on screen was not what the machine printed. Except for the zombies.
“We asked if our machine was broken,” McLean says. “They told us that the machine prints green and yellow really well, but other colors can be a problem.” Character painter Tory Bryant took the problem in hand. “She went through the Pantone book, the thousands of colors, and printed each as a physical chip,” McLean says. “That became her palette.”
As they tested the machines to calibrate the color, the crew’s excitement about this new process grew. “We realized the printer puts color one-sixteenth of an inch deep into the model,” McLean says. “That was unlike anything we’d seen before. In addition to getting wonderful colors, this technique gave us subsurface scattering for free. The model didn’t look like a thin shell; it looked and felt like skin. We realized we could put color where we wanted, so we put a little red inside Norman’s ears, and they looked like real ears. And that broke us free to do a lot of other things.” Most important, they could change the characters’ designs.
McLean explains that the characters in stop-motion films that use faces animated with replacement parts have lollipop heads— big heads on little bodies—for a reason. “It’s a wonderful design, but it’s also a side effect,” McLean says. “The bodies are made of silicon, which is translucent and has color all the way through, so you have subsurface scattering. But, you can’t put a hand-colored, hard, printed face next to the translucent body, so you need a long neck.”
But now, with the color printer, they could put the translucent, color-printed face next to a soft silicon body. “We created Alvin the bully with a thick neck,” McLean says. “We could never have done that before. That’s one reason why the designs in ParaNorman are so unique.”
A big inventory solved any problems with continuity. Although each printer outputs slightly different shades of red, the colors fade through the life cycle of the printer heads, and humidity and temperature affect the color. “When you have 10 or 15 copies of a smile, you can find faces that match,” McLean says. “Tim Yates, our face librarian, can look at hundreds of face parts and pick the ones that work well together for color.”
Once an animator finishes a shot and the shot has made its way through editorial, the faces move back into a CG pipeline where artists on the visual effects crew paint out the seams between all the face parts. They also work on the entire shot, removing rigs and, sometimes, reconstructing sets.
“We do all the traditional, invisible effects,” says visual effects supervisor Brian Van’t Hul. “Sometimes an animator will have to cut away a portion of a set to reach the puppet. We shoot a clean plate and tell them to go for it. These are paint and roto tasks no one will no- tice, hopefully, but they are extremely important and allow animators to stay in a creativeperformance mode.”
Big, Wide World
But, the most dramatic role the visual effects crew played was in opening up the world for the filmmakers. “Ninety to 95 percent of the film has stop-motion puppets and miniature props and sets,” Van’t Hul says. “Things people made. But, occasionally, we remind the audience of the bigger world. Stop motion can feel claustrophobic. Travis [Knight] gave us the task of expanding the world and opening the environment.”
Visual effects artists typically paint skies for stop-motion films. For ParaNorman, they created a volumetric storm. They also extended roads and streets in the town and populated it with digital extras. “We have a lot more CG elements in this film than in Coraline,” Van’t Hul says. “There was a conscious decision to open up the production value and make the world feel bigger.”
They did so in much the same way a visual effects crew would enhance a live-action film. Despite the painstakingly slow, frameby- frame performances, stop-motion is live action, and the visual effects crew needed to match the materials and scale of that stylized world. “We have to balance photorealism with what the directors want in style,” Van’t Hul says. “But the tools make it easier.”
As they might for a live-action film, the crew shot HDRI images of the sets to better match digital environments and characters with the practical sets and puppets. For HDRIs, they used exactly the same type of camera, a Canon 5D Mark II, that the animators and cinematographer used.
During a chase sequence in which a blue van rockets through the town, for example, the crew extended the set with digital backgrounds and composited those backgrounds into the shot. Then, they tracked the practical van in the shot, match-moved a CG version, lit and rendered the shot with the CG van. Lastly, they removed the reflections of the digital background from the CG van and applied them to images of the practical van.
The most dramatic environment the crew created, though, was for the climax of the film in which an enormous face of a witch appears in turbulent storm clouds.
“It would have been easy to have told the visual effects department that we needed a super natural storm,” Butler says. “But our reference was an imperfect, handmade storm that [production designer] Nelson Lowry had come up with.” To match the design, the artists created storm clouds out of tulle, material typically used for ballerina costumes, animated the material clouds, and lit them. Then, the CG artists needed to match the practical element.
In the sequence, volumetric CG clouds fill the sky. “We used shapes and shaders and cloth simulations,” Van’t Hul says. The effects artists started building the sky using a series of curves in Houdini to construct the cloudscape, put hundreds of pieces of twisting cloth material based on the weave in tulle on top, and added the volumetrics. The cloth simulations gave the clouds a tactile feel, the volumetric effects added scale. A giant, greenish witch’s face appears in the clouds, and as it moves through the sky, it impacts the volumes.
Eventually, the witch resolves into a puppet that animators perform, but with CG enhancements. “They wanted her dress to be flowing and electric,” Van’t Hul says. “So we tracked and match-moved the puppet. We put a CG dress on her and gave her Tesla-coil-like hair. We didn’t base the Tesla electricity on real electricity. We used a drawing that looked like someone had blown through a straw at ink spilled on paper. We wanted it to look like we had paid an animator to do an intricate, timeconsuming thing for six months.”
The crew rendered the clouds in Side Effects’ Mantra. For the characters, they used Pixar’s RenderMan.
Digital Puppets
In addition to the digital environments, another way in which the visual effects crew expanded the puppet’s world was by mixing digital characters with real puppets in the school and in town.
“We created digital ghosts, background adults for a mob sequence, and kids in the school,” Van’t Hul says.
Eleven real puppets sat in the foreground of the school auditorium; the rest of the characters in the background were CG. In the mob, puppets intermingled with CG characters.
“The CG characters had to match the puppets, even their skin texture,” Van’t Hul says. “We had multiple versions of skin. Background characters had rubber heads, which has a slightly different look and feel than the face-replacement heads. Our characters had to match each type. Our shader writers would hold a puppet and see how the light played off it.”
The effects artists also put a CG owl in a school play, moths in a teddy bear’s mouth, and butterflies in the sky. “We really tried to use the computer to enhance the film,” Van’t Hul says.
For the directors, the combination of CG and traditional effects is what gives the film its own identity. “We didn’t want to make just another stop-frame film,” Fell says. “There’s no point going round the same track.”
Butler adds, “Years ago, CG was in danger of killing stop motion. Now the approach is to embrace the age-old technique and drag it into the next century through innovation. The scope, the set extensions, the digital background characters, the face animation we can get with CG don’t detract from stop motion. They add to it.”
By pushing the state of the art, the artists at Laika kept the tangible—real light, real textures, real photography. But with the help of computer graphics, they were able to take stop motion off the table and animate puppets with smoother, subtler performances.
“I think we’ve got the best of both worlds,” Fell says.
Barbara Robertson is an award-winning writer and a contributing editor for Computer Graphics World. She can a be reached at BarbaraRR@comcast.net.