Digital effects studio Animal Logic choreographs a new pipeline to create its first animated CG feature
By Barbara Robertson
Oh, those irresistible penguins! They can’t help but charm as they shuffle, toboggan, swim, fall, tumble, huddle together, and waddle across the ice, the adults all dressed up in their jazzy tuxedos and the children as fuzzy as a stuffed toy. But there’s one thing penguins can’t do. Penguins can’t dance.
In the penguin world, this isn’t a problem, but for the Sydney, Australia-based animation and visual effects studio Animal Logic, making digital penguins dance became a four-year obsession. And dancing was but one tangle the studio had to rumble through to create Warner Bros.’ musical-adventure-comedy Happy Feet.
The CG feature is the first animation directed by Australian filmmaker George Miller, who directed The Witches of Eastwick and wrote and directed three Mad Max films and Babe: Pig in the City. Happy Feet blends a song-and-dance coming-of-age story with an epic adventure. The star of the film, a fuzzy young penguin named Mumble (Elijah Wood), can’t sing, which is a problem because penguins choose their soul mates with a "heart song." What Mumble can do, though, is dance, and that he does, to hip-hop, salsa, jazz, pop, rock—to any music he hears.
Mumble’s mother (Nicole Kidman) is sympathetic, but not his father (Hugh Jackman) or the village elders, who blame poor little Mumble for disturbing the natural order of Emperor Penguin land and causing fish to disappear. When they banish Mumble, he waddles away and begins an epic journey through Antarctica to prove them wrong.
Animal Logic’s epic journey began four and a half years ago. At that time, around 60 employees crafted commercials and visual effects for such features as The Matrix, The Lord of the Rings: The Fellowship of the Ring, Moulin Rouge!, and the two Babe films. At peak, the studio expanded to approximately 500 people for Happy Feet and its ongoing digital effects work.
Digital supervisor Brett Feeney led the Animal Logic teams that designed the pipeline, the processes, and the tools for Happy Feet. "Obviously, the dancing crowds of penguins were a challenge we had to meet," he says. "And the surfacing of the penguins, wow. We never had to do that many hairy, feathered creatures before. But of all the challenges, being able to get the environments looking real took the longest time."
Penguin Anatomy 101
Because Miller wanted Happy Feet to be photoreal even though the penguins would burst into dance, Animal Logic had to tiptoe around penguin anatomy. "The director wanted to portray the characters in a natural state," says Damien Gray, supervising character technical director. "We huddled them together like you’d see in any documentary. We had to include a high level of realism and still cater to their dance behavior as well."
Although audiences seeing Happy Feet might think it reminds them of the 2005 documentary March of the Penguins, the character team began its work years earlier. For inspiration, they referenced the BBC television series Life in the Freezer. They also looked at various types of penguins to find those most amenable to dancing. "Penguins aren’t built in nature to dance at all," says Gray. "Their skeleton is quite squatted. They have a small thigh-to-shinbone ratio. They have a massive sternum plate. They’re designed to form a torpedo shape underwater. On land, they walk on their toes and have no real shoulders."
All that had to change, but without losing the birds’ "penguin-ness." Mumble and his friends tap their heels on the ice and have fake shoulders but no sternum plate. As for their legs, even though the film features Emperor, Adélie, and other penguin varieties, all are based on the anatomy of a King penguin. "The King penguin is more athletic," says Gray. "It gave us more freedom in terms of the leg length and shinbone and thighbone ratios. We wanted to avoid having the penguins look like people with their pants around their ankles."
For creating the dancing penguins and their environment, Animal Logic used Autodesk’s Maya and Softimage’s XSI for modeling, XSI for rigging and animation, Maya for effects and lighting, Pixar’s PR RenderMan for rendering, and the studio’s own MayaMan to move RIB files from Maya into the RenderMan pipeline. In addition, the motion-capture crew used software and a system from Giant Studios.
Penguin Moves
To help animators perform a chorus of dancing penguins as well as Mumble’s fancy steps, Animal Logic relied on motion capture and, for the dancing crowds, the studio’s own system, called Horde. "We used Massive for some crowd shots, but for the dancing, we thought it was easier to build our own system than to divide the dance steps into tiny bits and put them into brains," says Feeney. Instead, they captured as many as 15 dancers at a time and 45 individual takes of the dance steps, which they multiplied onto half a million penguins. "We could mess with the timing enough so it looked more natural than a simple offset," he says.
At left, Mumble (Elijah Wood) steps out using mocap data topped with keyframe
animation. At right, dark shadows dramatize a shot with Lovelace (Robin Williams).The studio also captured dancers for the dramatic scenes and some action shots. To learn basic penguin behavior and postures, the dancers attended ornithologist Dr. Gary Miller’s "penguin school." Once onstage, however, Miller directed the action. "The beauty of the Giant system was that George [Miller] could see what the penguins looked like while the humans were performing," says Feeney. "The system was ahead of everything else at the time." Thus, when a dancer bent forward, for example, the director could determine whether the movement matched the penguin’s natural behavior closely enough.
It took 12 months to motion-capture all the designated scenes in the film. During that time, Animal Logic used Softimage XSI to edit the data, the bits and pieces that Miller selected. "We’d take a bit out of this take and that take, and blend them together in the same way you would if you were editing sound or video on an Avid [system]," says Feeney. The edited takes traveled to the layout department for camera moves, then to editorial and on to motion edit for cleanup, before being handed to the animators. "Even though we captured an astounding amount of data, the motion-edit group only cleaned up the bits that were necessary," he says.
Jazzy Rigs
Animators keyframed all the facial animation and some of the performances, and enhanced 85 percent of the motion-captured data. In addition, they animated props, vehicles, and icebergs. "We rigged 425 individual assets," says Gray. "Some characters had 10 versions of their rig; Mumble easily had 50 different versions. We also rigged anything the characters interacted with or touched."
Each character started with a lightweight motion-capture rig driven by the mocap performance. A second rig for animation included controls for facial animation and for adjusting the motion-captured performance. In addition, many of the characters had swimming and other action rigs. A deformation rig, which included a muscle system that preserved the penguin’s round shape, created the final mesh. Other rigs helped speed the animation process.
"We cobbled together components of various rigs into ‘Frankenrigs,’ " says character supervisor, Aidan Sarsfield. "For facial animation, we used a high-resolution face and low-res body. For crowd animation, we had the opposite, a Frankenfaceless, which was a body rig without any face." In addition, three variations of the motion-capture rig accommodated various file formats. Even so, Sarsfield claims that motion transferred through all the various rigs ended up aligned, within pixels, in the final render.
To give the penguins facial expressions, the riggers had to dance a special jig. Penguin beaks don’t lend themselves to lip sync, and their eyes are on the sides of their heads. Thus, rather than flapping the beaks like clamshells, the animators tuned the corners of the birds’ mouths using rotations and translations. "To make an ‘oooo’ sound, the corners come forward in a circular formation on the beak," says Sarsfield. "For an ‘eeee,’ the corners travel back."
For the eyes, the riggers worked with eye and brow shapes to overcome the perception that the eyes were looking in different directions. "Mumble probably has the most forward-facing eyes because it was important for the audience to engage with him," says Gray. "We did a lot of work sculpting the brows to achieve the classic Elijah Wood empathetic look. But the biggest things were eye darts. We labored to create the high-frequency flicker that eyes tend to do."
One of the trickiest parts of the rig, though, was the knee. "We had pivots for the knees, shins, ankles, and hips," says Gray. "But when it came to deformation, we often cheated the bulge of the knee. We dropped it a bit lower than where the pivot point was." The rig catered to both the motion derived from the joint and the offset for the bulge. Because the skin wasn’t bound to the knee, the knee could move further than the surface, and because the renderer didn’t render the bones, the result was a subtle skin deformation. That meant a penguin could do a high kick without distorting the mesh so much it wasn’t believable, or an extreme knee crunch that caused the bones to poke out of the surface.
Feathers and Fuzz
Although Mumble hadn’t yet molted into adulthood for most of the film, and the crew could fluff his surface with RenderMan RiCurves, a majority of the penguins are feathered birds. "We used the same system to do feathers and fluff, although the feathers had geometry," explains Feeney.
That proprietary, rule-based system worked within Maya. "It was quicker to have 10 or 12 artists set up the feathers so they couldn’t collide with each other than to write a dynamics system and run simulations," Feeney adds. "With the penguins, if the feathers intersected a bit, it didn’t matter."
Basically, the feathers followed this rule: The tip of a feather that was anchored to the body had to be a particular distance off the body. So, when all the feathers followed the rule, none interpenetrated. For the angry, seagull-like skua birds, however, the team added collision avoidance. "They had bigger, more visible feathers," Feeney says.
In addition, for the "Amigos," a group of the Adélie penguins that Mumble meets on his journey, animators keyframed the head feathers. Additional controls allowed the animators to flare chest and tail feathers on all the birds.
Baked Birds
To make life easier for the animators, the surfacing crew, and the lighters, Animal Logic created a special file format, a shape animation cache file format they named "bobject."
"Chris Bone [head of R&D] and I came up with the idea because he had done a project where he had baked out OBJ files per frame," says Feeney. "It was a good way to go from one package to another, but it was unwieldy with many files. So, we made smaller, binary object [bobject] files. We’d select mesh parts and bake them out as bobjects." Both Maya and XSI could read and write the shape per frame format. So, for example, when animators finished a performance, it was loaded onto the muscle rig and translated into a bobject. The surfacing and lighting crew accessed only the frames they needed from the bobject cache.
With scenes packed with hundreds of feathered CG creatures, such streamlined processes became critical. "At the end of the day, we had characters with three million individual feathers," says Sarsfield. "So, for any given character, we’d get the performance and then bake out the ‘hair’ for that performance. We’d also strip out the animation controls, fur animation controls, and guide hairs, anything that would weigh it down." Lighting TDs could work with as many as 40 of these stripped-down, render-only characters.
Ice Pack
The feathers presented a prickly problem, but surfacing the world was a much larger challenge than surfacing the penguins. "Once we established the look of the penguins, we dedicated ourselves to the environments," says Feeney.
At first, they thought they might scan glaciers to create landscapes. "We took a Lidar scanner to the South Island of New Zealand," he says. "It worked in some cases but was useless in others. Photogrammetry was also useless because there’s not much contrast range in ice. We even thought of dying bits of ice with food coloring." They ended up relying on the power of modelers who could shape the world as directed and on matte painters who could extend it.
"When we started, we had the idea that we’d make nine or 10 environments and we’d have the movie done," says Feeney. "Instead, we had 12 different locations, and each could have up to 30 different stages." Matte paintings projected onto 3D geometry with Maya camera projection surrounded the modeled and surfaced areas.
A custom layering system for shaders, called Freezer, that Animal Logic built inside Maya helped the crew create the surfaces. "Most of the artists knew Photoshop back to front, so we created a system that let them layer their displacements and color textures as if they were painting a map in Photoshop," notes Feeney.
A skua bird grabs Mumble’s fi sh over water created by deformers pushing
the surface up. Particle-based splashes and drips were separate elements.To make the ice translucent, the technical team began looking for subsurface scattering cheats. They ended up biting the bullet, as Feeney describes it, and working through the techniques in Henrik Jensen’s technical papers. "We did come up with a reasonably unique system to do subsurface scattering with high detail on vast environments," says Feeney. Once again, it was back to the oven for a solution: They ran subsurface scattering on a few kilometers of terrain and baked it out. "If we had 100 machines, we’d divide the landscape by the number of machines and give each a piece," Feeney says. When finished, a custom stitching program put it all back together as a giant texture map.
"We also baked in some ambient occlusion," Feeney says. "It made it a bit easier, although we couldn’t do that when something moved—when there was a landslide, for example." The lighting department then added key lights and so forth to the baked surface with the subsurface scattering and ambient occlusion.
Lighting supervisor Ben Gunsberger estimates that the lighters used between 20 and 30 base lighting setups. Every scene and shot was lit individually and often included a story arc. For example, a scene in which Mumble comes back to the community starts off upbeat and sunny. But when the situation turns sour, clouds move in and the sky darkens. Clouds, it turns out, were useful in many ways.
The first fully rendered icy environments with fully surfaced characters slid out of the rendering pipeline in March 2004—a scene in which a remedial teacher is training Mumble how to sing. "We were all quietly unhappy with it," Feeney recalls. Cinematographer Rory McGuinness, who had filmed in the Antarctic, helped them see what was wrong. Feeney says, "He looked at it and said, ‘It’s the blacks. You have nothing dark in the scenes other than the penguins.’ "
Mumble, Lovelace, and the Amigos meet the elephant seals under a cloudy sky created
from a blend of painted and color-graded photographs.Thus, the crew began to look for ways to add such elements as clouds. "We couldn’t play with shape and contrast on a flat ground, so we used moving cloud shadows to provide contrast and interest in the shots," says Gunsberger. To create the clouds, the studio used photographs taken in Tasmania and Fiji of 360-degree panoramas blended, painted, and color-graded.
Modelers working in Maya shaped the icy landscape, surface artists textured it with a custom
shader-layering system, and matte painters extended it using projections on 3D geometry."We had a small team that prepared the sky elements using a method similar to that used by matte painters, except they were working with moving elements," says Gunsberger. When Gunsberger began working on the film in 2003, he had a core team of five artists and TDs. By the end of the film, his crew topped 70. Lighters at Animal Logic composited the shots they lit; the studio output each light and every component separately.
"The penguins’ faces and heads are small in proportion to their bodies," says Gunsberger. "We worked hard to get expressions, particularly in some of the black-faced characters. If we had too much diffuse light, the black faces looked dull and gray. By controlling each part of the character with separate mattes—the faces, eyes, mouths, beaks, tongues, and so forth—the lighters could preserve and enhance the animators’ performances.
Back in the Huddle
Toward the end of production, the studio decided to outsource four action sequences to Giant Killer Robots and Rhythm & Hues. "George [Miller] picked the scenes with lots of motion blur and particles spraying ice and water so that if they weren’t reproduced exactly, it wouldn’t show," says Feeney. "But the scenes look great." Rhythm & Hues handled a scene in which Orca whales chase penguins, and another in which Mumble chases a boat. Giant Killer Robots handled a bobsled run and a scene in which Mumble sees signs of "aliens."
The film was a huge undertaking for what had been a small digital effects studio, but like the penguins, Animal Logic steadfastly handled the journey, step by step. "We just took it day by day," says Gray. "Toward the end, we could turn things around in a couple hours that would have taken days at the beginning; we could make large changes and propagate them through the entire pipeline efficiently."
Feeney is especially proud of the work the team did on the pipeline. "This was a new toy for George [Miller]," he says. "And many people had never worked on a CG feature before. We knew there would be changes. Yoga classes couldn’t make you as flexible as our pipe at the moment."
One of those new to CG features was Sarsfield, who had worked on The Matrix films and Moulin Rouge!. "If I’d known what I was getting into three years ago, I would have been very scared," he says. "Now that we’ve done it, I’d do it again in a heartbeat."
Barbara Robertson is an award-winning writer and a contributing editor for Computer Graphics World. She can be reached at BarbaraRR@comcast.net.