When Warner Bros. released the first
Happy Feet movie, people wondered what they were thinking down under. An animated feature in which many of the character performances started with motion-capture data? Blasphemy. But,
Happy Feet’s joyous story caught the imagination of audiences worldwide, and the film went on to win the Oscar for Best Animated Feature Film in 2007.
Following that win, director George Miller founded his own studio, Dr. D, in Sydney, Australia, and began preparing for a sequel. In 2009, he hired Rob Coleman to build an animation team and direct the animation for Happy Feet Two, which picks up where the first film left off. Mumble, the Emperor Penguin who could dance but not sing, is now married to Gloria; they have a son, Erik. Erik can’t dance, but when he meets the “Mighty Sven,” a puffin that Erik mistakes for a penguin, Erik becomes determined to fly. Returning penguin voice actors include Elijah Wood as Mumble and Robin Williams as Ramon and Lovelace.
Prior to joining the
Happy Feet Two crew, Rob Coleman was an animation director and supervisor at Industrial Light & Magic, where he received two Oscar nominations for best visual effects (for
Star Wars: Episode II – Attack of the Clones and
Episode I – The Phantom Menace) and two BAFTA nominations (for
Episode I and
Men in Black). We spoke to Coleman soon after work on
Happy Feet Two wrapped.
At top: A team of 75 animators from 14 countries worked at Dr. D studios in Sydney, Australia,
to perfect and amplify performances captured from dancers for the penguins, and to add facial
expressions. At bottom: A separate team of animators used motion cycles and a rules-based
system to animate crowds of penguins and schools of fish.
How did you begin this project?
I sat with George [Miller] and looked at what he liked and didn’t like in the first film, and I spent the first year, from April 2009 to April 2010, building an animation team.
How many animators did you have on your team?
I had 75 animators at peak from 14 countries, with 32 from Australia. I was worried when I first came down here because I knew CG animation wasn’t huge. There are companies doing CG, but there aren’t a lot of character animators. But, just before I started hiring, Animal Logic was finishing Guardians and didn’t have another big show yet, so I was able to pick up a lot of senior and mid-level animators and a couple of leads who probably otherwise would have gone to Canada or the UK. Then, I committed to hiring only Australian junior animators.
How did you organize the team?
I had a number of leads, which is similar to the way I worked at ILM, and divided the work into sequences. At peak, we had nine teams, but most of the time we had six or seven. Each lead had around seven animators. Everyone did penguins, but two of the teams became really good at animating krill, so I cast more krill sequences to them. And, we didn’t teach every team how to animate elephant seals, which were all keyframed.
The krill?
Will the Krill and Bill the Krill, voiced by Brad Pitt and Matt Damon. They’re the reason I wanted to make this movie. There’s a parallel story about the tiny little krill, and their story is so good and so funny. And, to animate things as little as krill sounded amazing. They look like little brine shrimp. They are almost at the bottom of the food chain; they’re insignificant. But they have a big impact on the biosystem of the world. Fish feed on them, whales feed on them. We have thousands and thousands of krill.
What is their story?
Will the Krill decides he doesn’t want to be in the krill swarm, so he and his best friend, Bill the Krill, break away from the swarm and end up as two little individuals in the ocean. They have contact with our hero penguins. Although neither species knows about the other, we see them both.
Did you use motion capture for the penguins, as on the first film?
We used some motion capture, predominately for the dancing and dramatic sense when the penguins walk around. [Director] George Miller comes from a live-action background, and he’s comfortable directing actors on stage, so he could do take after take quickly. He could get a performance he wanted in an hour or a day that would have taken us a month. But he also enjoys the animation process because he can plus the performances and add facial animation.
Motion capture allowed the two worlds to come together, and because the characters are humanoid and walking around, I’m fine with it. When the characters come to animation, they have their weight built in already. So, we get the combination of movement from talented performers directed by the director and performances stylized by our talented animators. Also, when George wanted thousands of characters on screen dancing intricate choreography, keyframing would have been impractical.
Did you motion-capture any of the other characters?
We also used motion capture for Sven, the puffin, when he’s on the ground, but we keyframed him when he’s flying.
But, not the krill, of course.
There was an attempt. They did a bunch of experiments. The krill have 10 legs and arms, so they had a conga line with five dancers trying to do the legs. They captured Savion Glover [dancer, choreographer] tapping for the krill, as well, which was extremely beneficial for me and the animators. The mo editors [motion editors] could take what he did, apply that to a low-resolution krill model, and we could study the feet and get movements that would have been difficult if we were keyframing 20 legs and arms. We also had early experiments with a puppeteer moving a krill body on the mocap floor, but we couldn’t get the right scale of motion when we put the krill into the water.
Were there any other unusual motion-capture experiments?
There were always experiments. For the elephant seal, we had four people performing together, but trying to wrangle that was too much effort. I could get a talented animator to do something really beautiful in not too much time.
What was the motion-capture process?
We used a Giant Studios’ system. We had a bunch of talented people here who had worked on the original, then worked on
Avatar, and came back to do
Happy Feet Two. George [Miller] would cut the audio first, and they would broadcast that onto the floor so everyone could hear it. The performers pantomimed to the dialog. They might not hit the accent of a word exactly, so it was up to the mo editors to make it feel like the voices were coming out of the bodies.
Each day we would recalibrate the dancers. We’d measure their legs and arms precisely so we could translate them to the character maps for each species of penguin. We could capture up to 10 at a time, and could see the penguins walking around in real time as the dancers performed. Their feet weren’t locked to an ultra-resolution set, but we could see where they were.
Did you need to change the data much to have the dancers move like penguins?
The dancers all went to ‘penguin school,’ and learned how to dance like Emperor or Adelie Penguins, but it took a fair amount of labor to get [the characters] to move and act like penguins. Penguins are like little flour sacks, like little fluffy pillows. When you have a human walking like a penguin, it’s one thing if they pantomime it for you by keeping their legs together and waddling. It’s another to have it look real. Our motion editors had to manipulate the data to make it work the way George Miller wanted.
Once George had directed the motion capture, he would make selects. The motion editors worked in [Autodesk’s] MotionBuilder and [Giant Studio’s] Nuance. They would pick the matching human performance and re-map the data onto the penguin bodies, and then put the penguins on the ultra-resolution set, the undulating ice field, and spread their toes. I would review the work in progress and make critiques. When I was happy, they converted the files into [Autodesk] Maya files and sent those rigs to the keyframe animation team.
Director George Miller recorded Brad Pitt and Matt Damon acting out the dialog together for
three days to give Will the Krill and Bill the Krill their voices. Although the team experimented
with motion capture for the tiny creatures, animators created all the performances with
keyframe animation.
Did you develop any particular tools or rigs for the keyframe animators?
We had a similar skeleton for each species of penguin, and we had offset rigs. The offset rig was a parental rig on top of a child rig. The child rig received the keyframe data from MotionBuilder. With the parental rig, the animators could add rotation and translation to the big volumes—the hips, head, shoulders, and chest. They were all IK. Often, once George Miller saw [the motion-captured animation], he wanted to go broader than what he saw on the floor. So the animators might put a translation on the chest, or change the eye direction by swinging the head, and so forth. The animators could supersede the data and movement with our offset rig.
When did they do keyframe animation?
We keyframed the animation when the characters swam or when they did dangerous actions. We also keyframed the whales, leopard seals, and fish. The fish are basically food in this film.
And, there’s no facial capture. Every penguin ended up being a hybrid. George was very happy with what Animal Logic had done. But now that he had some experience with animation, he wanted to spend more time on the movement of the eyes, the dilation of the eyes—the eye dart-ness as he calls it. He was very particular about beak sync, lip sync, and phonemes—about the movement of the tongue and lips—and that was cool with me. And, we spent a lot of time on the non-verbal, reaction shots. It was challenging to get the penguins to look good from multiple angles and still connect with the audience and characters on the screen. You have to see their emotion. They have humanoid faces, but their eyes aren’t binocular. They’re set 30 degrees back on an angle.
Did you use the models from the first film?
We based the characters on where they left off on the first film, but we were using Maya, and Animal Logic had used Softimage XSI. So, most of the characters were redone and rebuilt; we upgraded the models. And, we started over and redid all the rigs. That wasn’t a big factor for me. If we hired someone who knew XSI, we could teach them the new keystrokes in a week. What we cared about were their animation and acting skills.
Did the animators have video of the actors as reference, as well as the motion data?
During the voice recordings in Los Angeles and Sydney, I had a team of videographers shooting the main actors. Even though they’re performing to microphones, once they get into the characters, they start performing with their faces. There are nice things you can do if you’re there to watch or capture them on video; you can use their expressions to drive the animation later. Elijah [Wood] did some things with his eyes that became part of Mumble’s performance. Brad [Pitt] might do something with the tilt of his head, the furl of his brow, that is inspiration for the animators down the road. We weren’t motion-capturing. We were videoing. But, we would see patterns. I cut together what I called ‘spirit reels’ from the videos and had QuickTimes for the animators to reference.
Did any of the actors record the dialog together, or did they work separately?
Brad and Matt came in for three days, so we had them in the same room, acting to each other. And we had many of the other voice actors in the same space at the same time interacting; upward of eight performances in a big space all recorded at the same time. You get better performances. You get talk-over, which you also have in live action, so why not have it in animation? George [Miller] had the actors do the initial performances until they were happy together. He recorded that. Then, they could do the lines themselves as they had done in the ensemble piece. That way he had the clean lines, and if someone stepped on a word, he could replace it.
How did you animate the schools of fish?
We’d animate the main characters, the hero characters, and provide swim cycles for the fish—and the penguins—to the crowd team. We had about 25 artists plus a director and supervisor on the crowd team. They’d attach our keyframe animations to a system that had run-time rules, and the fish would scatter like in nature when they came near the penguins. It was amazing.
When the characters are swimming, did you animate to the movement of the water, or did the simulation team move the water based on the keyframe animation?
We handled water in two ways. Basically, if the shot was about the character, the water team would match the animation. We would talk about the water with George, and he’d tell us whether he wanted the water to be calm or move a lot. We’d do some keyframe animation, and he might tell us to tumble the characters more or soften their movements as if they were in a swell. Then the effects artists would put the water around them. If the shot was about the effect, like ice tumbling into the water, we would match their simulation.
We used [Exotic Matter’s] Naiad for all the splashes and for the interaction of the characters with the water. The effects team then stitched the Naiad splash elements into a high-resolution surface simulated in [Side Effects Software’s] Houdini. They were able to create a realism on the surface of the
water that I think is breathtaking. They also did volumetric light shards coming down through the water. There’s a beautiful shot with our two hero krill clutching the bottom of a piece of ice, with smaller pieces of ice tumbling in the turbulence of the water. We look up through the water and see the caustics.
For the krill, because they are about the size of a thumbnail, they put silt and dust in the water to help with the scale. It’s amazing when you see it in stereo.
Do the little krill survive?
They do. Through a series of events, a massive rogue iceberg crashes into the entrance of Emperor Penguin land and traps the penguins, which is something that actually happened in the real world. In our movie, all the communities come together to overcome the troubles of the world, and even the krill have an impact.
Did the icy environments change much?
We had two main environments, Adelie land and Emperor land, and they change during the movie. We start with compacted snow on ice, and then we have fluffier, powdery snow. When the animators go into the scenes, they had a packed-ice layer or a packed-snow layer for the penguins’ feet. Then another team added footprints in the snow, so we’d see little foot trails. The rendering of the snow, with sparkling highlights, is so amazing. It makes it feel like you’re there. Snow is a big part of the story; we had about 50 artists working on the effects team creating character effects, water, volumetrics, and destruction. There are beautiful shots of compacted ice and snow breaking apart.
Did you have favorite characters?
Well, the krill are certainly high on my list. The elephant seals were a pleasure to animate. The main one, Beach Master, was a fantastic character to get into, and he has a sidekick named Wayne. I liked them a lot. And, animating to Robin Williams is always great. He was Ramon the Adelie penguin, and the Rockhopper. Ramon is very over the top theatrically, and he gets a love interest in this film, so that added a whole other layer to his performance. We see Mumble worried about his son, and Sven wrapped up in being an inspirational speaker. We have big catastrophes and massive dance numbers. The first film hit a very high mark. We tried to step above it.
Barbara Robertson is an award-winning writer and a contributing editor for Computer Graphics World. She can be reached at BarbaraRR@comcast.net.