Pose by Pose

Dracula Like most of the characters in Hotel Transylvania, the Imageworks crew had modeled and rigged Dracula before director Genndy Tartakovsky entered the picture. Dracula was so important, though, that he was one of the few characters that changed to accommodate Tartakovsky’s style. “He is older than the most recent model had been,” says Dan Kramer, visual effects supervisor. “And a little scarier. The new Dracula is more cartoony, simple, and likeable. And, he has some resemblance to Adam Sandler.”The changes to his model and rig helped animators polish his performance. “We could really shape-change and go crazy with him, and that was fine,” says James Crossley, senior animation supervisor.

If the director of Hotel Transylvania weren’t so inspiring, if the animation style he embraced hadn’t been so unique, if the shots hadn’t been so hilarious, the poses so dramatic, things might have turned out entirely differently at Sony Pictures Imageworks. There, a crew of approximately 300 artists worked on 1,250 shots for the animated feature.

Directed by multiple Emmy-award winner Genndy Tartakovsky, Hotel Transylvania is a monster movie with an inventive, comedic twist: Count Dracula owns and operates a luxury resort hotel where monsters can escape the demands of the human world. The idyllic surroundings give the vampire count a perfect safe haven for Mavis, his teenage daughter—until a human teenage backpacker named Jonathan accidentally crashes Mavis’s 118th birthday party.

The film stars the voice talents of Adam Sandler as Dracula, Selena Gomez as his daughter Mavis, Steve Buscemi as the werewolf Wayne, Kevin James as Frankenstein, CeeLo Green as Murray the Mummy, David Spade as Griffin the Invisible Man, Jon Lovitz as Quasimodo, Andy Samberg as Jonathan, and many others.The idea of a hotel for monsters had been in development at Sony Pictures Animation for six years before Tartakovsky moved into “Hotel T.” Previous directors had approved character designs. Artists at Sony Pictures Imageworks had created the CG characters. The technical team at Image-works had set up a pipeline based on Arnold, a physically accurate renderer. There was just one problem.

pushed beyond the previous preparations.

“I was on the show for a year, maybe a year and a half, before it went on hold for a rewrite,” says James Crossley, senior animation supervisor at Imageworks, who led a team of 90 animators. “The characters were built and ready to go. But, we had set up the characters for a naturalistic style, and Genndy wanted a different style of animation. We had to achieve what he wanted with what we had. We didn’t have time to start over.”

Tartakovsky, known for the Cartoon Network shows Dexter’s Laboratory and Samurai Jack, had not worked with CG characters; he had not directed a feature film. He and the Imageworks crew had about a year to create this film. they had to hit the ground running.

Off Model

All told, the film has 40 main characters, plus variations of each to populate the resort hotel—a castle. “The castle is so immense, we needed to fill the lobby with a lot of variety,” says visual effects supervisor Dan Kramer, who joined the project about the same time as Tartakovsky. “We couldn’t have 100 skeletons and zombies.”

By the time Tartakovsky arrived, the crew had already built and rigged those characters for the most part. “they were 80 percent done,” Kramer says. “Maybe more. We tend to build to about what we think the animators will need to do in the shots, but there’s always a little bit of room for animators to work on characters during shot production.”

The director asked for changes to Dracula’s face and a complete redesign for Jonathan. “Jonathan was a different character before,” Kramer says. “A simple character. Nothing special. Genndy likes to make his characters look more special, and in Jonathan’s case, he wanted him to look goofy.”

Otherwise, the crew used the existing assets for the most part. For those characters, Tartakovsky wanted the animators to push the rigs further than animators had ever pushed them before. “For his style, it made sense to hit extreme poses momentarily,” Kramer says.

Some poses were so extreme, in fact, they took the characters way “off model.”

“Genndy hated the term ‘on model,’ ” Crossley says. “He wasn’t worried about whether Dracula’s eyes were different sizes in the same shots. It wasn’t important to make sure we didn’t pull a mouth corner too high. That was fine. His viewpoint was, ‘Yeah, we’re changing the design. this is the drawing I want.’ ”

Thus, during shot production, the crew often found themselves trying to update characters and make them more pliable. “this was Genndy’s first CG feature,” Crossley says. “He did hand-drawings. He’d say ‘I don’t know if we can get near this, but this is what I’m hoping for.’ Mostly, we never said no.”

Tartakovsky gave animators notes via a Wacom tablet and a proprietary, frame-by-frame player with custom plug-ins, by drawing over keyframes exactly the pose he wanted the animator to hit.

For example, the first character Crossley had tested, helped build, and set up for animators was Frankenstein. He had created a performance using the character for a couple of shots. “The performance was subtle and natural,” Crossley says. “And then Genndy came on the show and had an animator do a shot. Genndy did a drawing over the pose when Dracula is screaming. He drew the jaw literally three times more open and broader than we had ever planned to do. If we had tried to do that, we would have broken the character. His jaw would have crashed into his chest.”

Mavis The animators were more reserved in performing Dracula’s daughter than the other characters. “You can’t get as silly with heroines,” says James Crossley, senior animation supervisor. “With a guy, you can get goofier than with a girl. But, we kept trying.”Like Dracula, gravity doesn’t affect Mavis, so she can walk up walls. She can also transform into a bat. Animators would create the character transition from bat to vampire or vice versa, and the effects artists would spawn vaporous fluid particles from the characters to create a magical effect. For these transformation effects, the artists used Imageworks’ proprietary fluid system. To render the volumes, Arnold. “Before, we would composite the effect and the environment,” says Dan Kramer, visual effects supervisor. “Now, we can get global illumination within the volume. We can define the vapor to have light, and that becomes an area light for the environment.”

Jury Rig

Stretching the rig beyond what the character TDs designed it to do and fixing problems with the geometry caused by an extreme pose became a standard modus operandi for the animators.

“There’s always room for the animators to work on a character during shot production,” Kramer says. “We set up the characters to survive rig upgrades. the only issue is that it was painstaking work for the animators. To hit the poses, they sculpted a character frame by frame, rather than letting the rig do its job. But no one wanted to hold back; we all fell in love with the animation style. So, we just went for it. And, if a problem came up often enough, if it was a consistent note, something the animators tweaked every time, we’d put it into the rig.”

Animators working in Autodesk Maya would start with sliders preset by the character TDs. Then, to manipulate a character’s pose to match Tartakovsky’s drawings, they would begin sculpting.

“We’d go into the control points,” Crossley says. “We used lattices, clusters, blendshapes, and a Maya deformer called SpWrinkleFree. If you push pieces of geometry together and your vertices overlap, making the model jagged and messy, SpWrinkleFree will average the points and relax them.”

Although one animator ended up reshaping a witch’s hand, the animators more typically used the tools to reshape faces, especially the areas around mouths and eyes.

“This show turned animators into modelers,” Crossley says. “I think they probably spent about 30 percent of their time modeling. We’d model a pushed shape for one frame. There were even times when we were making characters as we went. We had to fill the environment with a lot of characters. If we had a zombie and needed another, skinnier one, we’d reshape the face.”

Graphic Moves

Moving the characters from pose to pose gave the film the hand-drawn feel that Crossley wanted and that suited Tartakovsky’s style. “A lot of people in CG do a layered approach in animation,” Crossley says. “They move the body, smooth the motion, and then add legs and arms. That’s fine for a lot of situations, but this show was all about character and design. I asked the animators to give me poses for the five main drawings, make sure they were nice and strong, and we’d break it down from there.”

It was a new style and an interesting learning experience for the classically trained ani-mators on the crew, and those used to a more naturalistic, subtle type of animation. “It was easy to hire animators for this film because Genndy was a big draw, the character designs were great, and everyone knew it would be fun because it’s about monsters,” Crossley says. “But this style is more complicated than most. They couldn’t rely on making things physically believable. They had to know how to design and pose. People have a tendency to over-animate and over-complicate. A lot of times on this film we’d simplify the animation to make nice, clear reads.”

Mummy Before Tartakovsky arrived, modelers had wrapped a scary-looking Mummy in bandages. Because it was important that the bandages didn’t stretch as the Mummy moved, the riggers created a complex system that slid the bandages one on top of another. “If we pushed it too far, though, it would crash,” says James Crossley, senior animation supervisor. “He was an energetic character. And then Genndy [Tartakovsky] comes onto the project and tripled what we did before. We would shape-change him a lot, and use deformers to fix anything that was breaking.”To help nail a performance that met Tartakovsky’s style, Crossley asked the animators to spend a week doing Mummy tests. “I asked them to be really graphic and treat the poses as graphic shapes,” Crossley says. “One of the animators tucked all the Mummy’s limbs into a ball, had him go into a roll, and then had him pop out and land on his feet again. Inevitably, Genndy had us put that in the film. The animator had the Mummy move so fast we couldn’t see what broke during the transition.”

For animators who learned to create a believable walk for a character by paying attention to weight shift, to the way the character’s hips move, it was a new way of working.

“[Weight shift] wasn’t important on this show,” Crossley says. “We’d start with minimal movement. Take Dracula, for example, walking confidently with a proud, stiff back. We’d do just the Z channel of him moving forward. We had his legs moving, but we resisted doing anything that gave away weight, the up and down. He almost slides across the screen.

When animators would point out that this was not how to create a believable walk, Crossley would answer, “That’s how you do a caricatured walk.”
 
“We traded physics for design,” Crossley points out. “We might lean a character on a 45-degree angle or move to a fast stop. Cloth simulations would fly all over the place. We’d push to get the energy and caricature the director wanted, pass everything off to cloth and hair, and hope they could deal with what we gave them. They had a great challenge on this show.”

Between the Frames

Hair, which grows from base geometry, wasn’t as much of a problem as cloth simulation. The cloth simulation team had to consider sub-frames.

“In the blurred areas between the frames, if a character took a different path and the cloth didn’t take that into account, the characters could come out of their clothing,” Kramer says. “So the cloth simulation team had to go in and make sure the simulation worked on the sub-frame, on, for example, maybe four samples per frame.”

Fortunately, it wasn’t always necessary to have cloth simulation be quite that accurate and subtle. “When we animate a character, we bind the clothes to follow along, and sometimes we’d use that version of the cloth since it tracked perfectly,” Kramer says. “If the character moved incredibly fast, it wasn’t noticeable. We’d turn the simulation off for the sub-frames, then switch it back on after.”

The snappy animation style presented an interesting rendering challenge, too. Often, the animators moved characters so fast they disappeared. “We use Arnold, a physically-based renderer that tries to mimic the real world,” Kramer says. “When characters moved fast, we had issues with motion blur. A character moving 100 miles per hour completely disappears; a hand becomes transparent.”

But, Tartakovsky wanted to see the character, the hand, in the motionblurred frames, no matter how fast anything moved. We had fast animation on Cloudy with a Chance of Meatballs,” Kramer says. “This film pushed us further. We had to massage motion blur throughout the film.”

One solution was to reduce the motion blur, which produced strobing images. Another was to render two versions—one with full motion blur to produce a streak, and one with low or no motion blur—and composite the two together, which produces a solid leading edge and blurry trail. The third option was multi-segment blurring.

“Normally, we start at one frame and blur to the next,” Kramer explains. “If there’s motion between, it gets lost unless you sample the object between the frames, on the sub-frames.”

Kramer uses a car wheel spinning fast as an example. If we see the top of the wheel on frame one and the bottom on frame two, the renderer doesn’t know the wheel traveled in a semi-circle, so it renders streaks between. “You need to sample the wheel at the sub-frames, between the frames, to find the path, and accurately do the blur,” Kramer explains.

As in the example, the animated characters often moved so quickly that the only way to keep them visible was with multi-segment blurring. But, there was a cost. It took longer to render the scenes, and it resulted in more data.

“When we translated the characters out, rather than baking on frames, we baked on sub-frames,” Kramer says. “We might have six versions of a character. We could control how many depending on how fast the character moved and how much fidelity we needed in the blur.”

Jonathan Jonathan was the one character that artists at Imageworks changed completely for the director. The teenage backpacker, who started as a simple, normal character, became goofier. A more flexible rig gave animators the ability to create extreme performances for Jonathan more easily than for many of the other characters.

Non-photorealistic Raytracing

This wasn’t the first animated feature for which the crew had used Arnold. That honor goes to Sony Pictures Animation’s Oscar-nominated Monster House, released in 2006. But, the graphic style Tartakovsky wanted for Hotel Transylvania pushed the lighting and rendering crew in new directions. Imageworks artists use The Foundry’s Nuke for compositing and Katana, developed in-house, for lighting. “There are lots of cases when the lighting is not photorealistic,” Kramer says. “We suppressed detail. Suppressed light.”

One sequence that was especially striking takes place during a party. Dracula stands in full color amidst hundreds of characters all rendered as dark silhouettes, whether in the background or foreground.

“To do that with Arnold, we used AOVs [auxiliary output variables],” Kramer says. “When Arnold lights a character, it has information about all the lighting components and combines them into a final image. We use AOVs on live-action shows as well, but on this film, we used them to sculpt the frames in non-photorealistic ways.”

Thus, in addition to asking for a final, combined image, the crew had Arnold write out separate, multiple images. Compositors working in Nuke could then put selected and perhaps tweaked images back together.

“We’ll get one frame plus maybe 50 others,” Kramer says. “Each is a different component. It might be the lighting component, or a matte we defined, the hair on his head, his clothes, his face. That gives us tons of extra images we can control in the composite. Nuke loads in the main frame, and sort of automatically loads up the AOVs as options to be used in the comp. We customized it a bit. Rather than 50 nodes, we had one that’s aware of all the files that go with it.”

This tweaking took place on a large scale, as for the party, and on smaller scales, too, to achieve the graphic style Tartakovsky wanted. “One thing we often did was for Dracula,” Kramer says. “He had a shiny cape, but Genndy [Tartakovsky] wanted the cape to feel like a black mask. Dark, with no wrinkles, no light. So, we would go through our AOVs, find the specular light, and subtract it from the final image.”

Learning Experience

Non-photorealistic rendering also helped bring Tartakovsky’s style to the backgrounds as well as the characters. “We were constantly mitigating detail to focus on silhouettes and the portion of the frame Genndy wanted to focus on,” Kramer says. “The castle [re-sort hotel] was really detailed. So we ended up throwing backgrounds out of focus and dumbing down the textures. If the action was in mid-frame and the foreground was distracting, we’d darken it way down to get rid of the detail.”

As with the characters, modelers and texture artists had already built Dracula’s resort hotel and the immense lobby before Tartakovsky arrived.

“It was basically done,” Kramer says. “[The crew] had built a lot of areas for the previous version of the script. The reception desk was perfect. But, the castle is immense. We had other areas that hadn’t been touched much at all. Genndy tweaked the design as much as he could to fit his own style and taste.”

And that happened, sometimes, even after the shots were in production. Layout artists might have already designed camera moves and were ready to send the shots into animation and lighting when the shots went back for changes in texturing and modeling.

“It was a challenge to push look-development, model building, and environments further into production than on previous shows,” Kramer says. “We were building the bridges as we were going across the canyon. We kept pace, but we were much closer to lighting than we’re normally comfortable with. We were doing look-dev on characters and environments a month and a half before we wrapped.”

Quasimodo “As soon as Genndy [Tartakovsky] came onto the show, he wanted to change Quasimodo’s design,” says James Crossley, senior anima-tion supervisor. “He was four heads high, but Genndy wanted him to be two heads high. He didn’t want to change his costume, but he wanted his nose bigger, his eyes closer, and he wanted him to have more of an egg shape. So that was a drastic change, but we couldn’t change the model. The animators would hit a pose, scale the values, and shape-change toward the new design.”In one shot, Quasimodo hangs from a chain. “Genndy drew him like an egg with little limbs,” Crossley says. “We tucked his neck in. If you squint, his body has an eggish shape.”Although in retrospect it might have made sense to remodel this character, the decision wasn’t only a matter of efficiency. “He was a bit of a moving target,” Crossley says. “He wasn’t always one way.” With his design changing in the beginning of production from one shot to the next, it would have been difficult to settle on a model.

There was a good side to this, however. The result was stunning, and the crew learned from the process.

“We often knew exactly where the cameras were,” Kramer says. “So rather than building whole environments and over-building, we could build more efficiently, which was imperative when we were so close to lighting. I’m going to think about this system for the next film—decide how much we build ahead of time, how much to hold off on. We might need to crew the film differently, but I think some sort of hybrid might work. Sometimes we were too close to the metal. But other times it was more efficient.”

When you look at the character animation and clean graphic design of this film, you might think, “Oh, simple.” It wasn’t. Creating the look and feel of a hand-drawn, 2D animated film with 3D tools, especially with characters and environments designed for another style, was difficult. It was a creative process, as well.

“I was looking for something different,” Crossley says. “This film was a creative, fun project. It had a unique energy. And, I could see on the faces of the animators, ‘Wow. This show is different.’ This project changed so much, but in such a positive way, and in ways I didn’t expect. I’m really proud and glad to have been a part of it.”

Barbara Robertson is an award-winning writer and contributing editor for Computer Graphics World. She can a be reached at BarbaraRR@comcast.net.