Medieval Magic

By Barbara Robertson

Once upon a time, in a land not so far away, 275 people were given a daunting task: They had to transform zeros and ones into a rich landscape filled with delightful and sometimes rude creatures who would tell a tale so fascinating and funny that millions of citizens, perhaps even teenagers, would leave their homes to go see it. The 275 people huffed and they puffed. Slowly an enchanting world began to take shape. And then, as if by magic, three years later the zeros and ones were completely transformed-into a hilarious movie called Shrek.

Scheduled to open May 18 in US theaters, Shrek, based on the children's book by William Steig, is the second animated feature from PDI/DreamWorks, which previously created Antz, and the fifth feature-length film created entirely with 3D computer graphics. Directed by Andrew Adamson and Vicky Jenson, the animated comedy adventure stars, as voice talent, Mike Myers in the title role of Shrek, an ornery ogre; Eddie Murphy as Donkey, Shrek's wisecracking, fast-talking, sidekick; Cameron Diaz as Fiona, a beautiful princess with a deep, dark secret; and John Lithgow as Lord Farquaad, an anal-retentive, power-hungry villain. What more does a shattered fairy tale need? Well, certainly a fire-breathing dragon, knights in shining armor, gnomes, and fairy godmothers, and this movie has all that and more.

Using painted and procedural textures, grass shaders, and often as many as 100 lights, sequence supervisors created settings such as this for Princess Fiona, Shrek, and Donkey (Images copyright 2001 PDI/DreamWorks.)

With each new 3D animated feature, Toy Story, Antz, A Bug's Life, and Toy Story 2, the backgrounds and characters have become more varied, complex and detailed; the make-believe worlds have, in effect, become more three-dimensional in every way. Shrek, like all these films before it, has raised the bar again.

PDI tackled nearly everything that's difficult to do with computer graphics. Characters of all sizes and shapes ranging in style from cartoony to photoreal appear together in settings ranging from stark medieval castles to goopy organic swamps. The movie has hair, fur, cloth, fire, and fluid simulations. And there are crowds. In an early sequence, 1000 fantasy characters invade a swamp. Later, a cast of 1000 Duloc citizens cheer from the stadium stands during a mud-wrestling tournament. And toward the end, 1500 characters attend a wedding. The team also created 36 unique environments, 28,186 trees with 3 billion leaves, and 68 character models, including the four primary characters, several secondary characters such as the dragon and the knights, and 31 fairy-tale creatures. In this movie, the Big Bad Wolf, Pinocchio, and the Three Little Pigs work alongside such characters as the Pied Piper and the Gingerbread Man.

Animators could use more than 750 controls to create Shrek's performance. Some controlled one joint or muscle, others controlled groups of several.

The theme of Shrek is about not taking stereotypes and fantasies for granted, and the movie cleverly uses fairy-tale conventions and characters to make the point. "Whenever possible, we try to take a fairy-tale stereotype, set it up as a fairy-tale stereotype, and twist it in a comic way," says Adamson. "I've always thought that fairy tales don't have great messages: If you don't go to bed, the ogre is going to get you. If you meet your knight in shining armor, who is always a perfect, handsome prince, everything will be all right. So it's been fun to use pretty much every fairy tale ever told as huge comic fodder."

The story begins with Shrek happily living alone in a swamp to avoid being the monster others expect of him. One day, hundreds of insolent fairy-tale creatures are dumped there-the evil Lord Farquaad has banished the annoying characters from Duloc. Donkey has gotten mixed in with this crowd, and when Shrek inadvertently saves his life, he becomes Shrek's sidekick. Shrek's determination to get his peaceful swamp back takes the pair to Duloc, where Shrek finds himself fighting Farquaad's knights. He wins, and Lord Farquaad offers a deal: Shrek can have his swamp back sans fairy-tale creatures if he rescues a beautiful princess, who is locked in a castle guarded by a dragon, and delivers her to Farquaad. And thus, Shrek's journey begins.

Special skin shaders and hair programs developed by the effects department at PDI/DreamWorks help Fiona and Robin Hood look like digital humans.

"Oh. You were expecting Prince Charming?" says Shrek to Princess Fiona when, after rescuing her, he takes off his helmet to reveal his true identity.

For character creation and animation, PDI relies almost exclusively on its proprietary software tools, which have evolved during the 20 years the studio has been working with 3D animation. For Shrek, various teams developed new tools for that proprietary system in nearly every area-the underlying animation system, character set-up, cloth simulation, methods for creating crowds, and the rendering of skin, hair, and fur.

The animation system used for primary characters in Shrek is based on muscle movement. Fiona, shown here, has 90 muscles.

Modelers started building the digital characters by using data scanned from a sculpted maquette and imported into Alias| Wavefront's PowerAnimator and Maya or Avid's Softimage|3D, but this was only the beginning. Shrek's 3D model went through 50 revisions, and Fiona's face took a year of experimentation before the team got the look they wanted. Shrek's model lost weight to make him younger looking and easier to animate; Fiona's face became softer and more realistic.

"It was hard to find the right level for Fiona," says Adamson, "because she has to be beautiful and something people can respond to on a visceral level, yet she also has to fit the stylized world." Once her maquette was scanned, Fiona's stylized eyebrows produced harsh shadows over her eyes, a cute upturned lip looked puffy when animated, and large catty eyes gave her a spooky look. "The sculpture looked good, but once we got it into the computer, it began to look freaky," says Jeff Hayes, modeling supervisor. "There's something that happens in the translation from real life into the computer that gets tricky, especially for a humanoid." Other models were less problematic, but even secondary characters were complex: It took 60,406 polygons to create the body, head, and clothing for generic characters in medium shots.

In one of many spoofs of live-action effects movies, Fiona shows the Merry Men some Kung Fu. The effects team created trees procedurally; Maya's cloth simulator helped them move Fiona's skirt.

When finished, models were imported into PDI's proprietary software and were moved within that system, the so-called "pipeline," for the duration of the process. First stop: the character setup department, where a team of 15 technical directors (TDs) fitted the models with bones, joints, muscles, deformations, and animation controls. In each character, the joints moved bones, the bones moved muscles, and the muscles moved the skin. Installing the muscles and joints and determining how the muscles deform was a complicated process, but this gave the digital actors fundamental characteristics-the shapes Fiona's mouth could make when she smiled, for example.

Character TDs created methods for assembling crowds and systems that automatically wrinkled tight-fitting clothes as characters moved.

Animators moved joints and muscles using animation controls; high-level controls organized by character TDs allowed them to manipulate several animation controls at once. When an animator turned Shrek's head, for example, wrinkles would automatically form on the back of his fat neck. Similarly, the animators could use high-level controls to cause characters to crack grins, make fists, or move their chests to simulate breathing. Dick Walsh, lead character TD, describes the system: "When you decide you want to smile you don't think, 'I'm going to make my cheeks puff, I'm going to expose my teeth,' you just smile and stuff happens. That's the way our system works. We define the muscles a smile will activate, and from that point on everything happens; the muscles pull the flesh around. We stabilize certain areas of the face that in a human typically don't move, though, which helps give a sense of reality."

Both primary and secondary characters used the same facial setup, which had between 100 and 200 animation controls; primary characters also had 585 animation controls and around 300 joints on their bodies. "We keep our setups consistent to make the motion animators' lives easier," says Lucia Modesto, character TD co-supervisor.

New animation controls make the characters' lips stick together a bit before opening. Farquaad's forehead wrinkles automatically.

The secondary characters-the dwarfs, gnomes, fairy godmothers, guards, and so forth-were created by the character setup TDs from generic models (male, female, child) and body types (fat, skinny, slightly fatter, slightly skinnier). In addition, by tweaking the facial animation controls, they changed face shapes. With these modifications, plus changes in skin color, hairstyle, and clothing, they had enough variety even for a crowd. The variations were handed to the layout department, which organized the scenes and created the camera moves. "If layout needed a bunch of guys for a shot, they could pick head zero, body 2, tunic whatever, put those numbers on a curve, and if they didn't like the result, they could just change it," Modesto explains. Because the characters had identical setups, typical animations, such as clapping, could be copied from one generic character to another.

Character animation for Shrek was divided among 25 motion animators, each of whom worked on entire sequences rather than on specific characters. "I think it would be fine to work on one character for a while, but it would be too much for two and a half years," says Raman Hui, animation supervisor. Thus, the animators often needed to create performances for several types of characters within one scene. "When we got more than 20 characters in a scene, we divided them into two shots," says Hui.

Getting characters to react with the environment was an important part of making this storybook world three-dimensional and believable.

Although the setups for the primary characters were similar, there were necessary differences between the human types of characters and the animals. Moreover, the styles of animation varied from the energetic, cartoony Donkey to the more realistic Fiona. The tricky part for animators working with Lord Farquaad, a caricature whose style fit between that of Donkey and Fiona, was in making actor John Lithgow's big voice sound plausible in the short character. They solved the problem by exaggerating the character's chest movement each time they heard breathing in the voice track.

Breathing was all hand animated using high-level controls; however, a few character motions were automated. For example, "We don't have to animate Donkey's ears unless he's using them to show expression," Hui says. And, when Shrek twists his body, his shirt wrinkles appropriately, thanks to systems set up by the TDs that automatically moved relatively tight-fitting clothing based on the motion of the character wearing them.

The torchbearers were created by TDs, who used a facial animation system to alter their faces, then added various hats, clothes, and hairstyles.

Simulations for clothes that drape, such as Fiona's long skirt and Farquaad's cape, were created by the effects department, a catchall group that stepped in to develop special rendering shaders, global procedural systems, and techniques for animating non-character elements that needed to deform.

With more than 700 shots requiring dynamic clothing, many with 20 or 30 characters, the team needed to devise an efficient process for simulating the movement of clothing on the animated characters. Rather than develop a proprietary cloth simulation system, the effects team developed a method in which models were generated for each frame of a shot during a batch rendering, stored on a model server, and then fed into Maya for cloth simulation. "Once we have clothing flowing nicely around the character's legs, we export the geometry to our model server, and at that point, the PDI proprietary software takes over again," says An drew Harris, cloth effects artist. To help speed the simulations, the team looked for ways to simplify the geometry. They discovered, for example, that skirt geometry and a balloon shape representing upper legs was sufficient for characters wearing dresses. "We looked for the general case, for what all the characters shared," says Bill Seneshen, effects lead.

The dragon's fire below, torch fire above, and roiling fire at right were created with geometry and particles. A fluid simulation was used for the hot lava under the bridge at right.

In addition: "On this movie, we generated models only when motion would change, and we'd store those models on a server. This saved a lot of computing time," explains Ken Bielenberg, visual effects supervisor.

The effects team also created skin shaders and the hair and fur systems. "The skin was tough," says Jonathan Gibbs, effects lead. "I did a lot of research and found very little in computer graphics, so I ended up across the street at Stanford reading dermatology papers." The surface shader he created simulates light bouncing inside several layers of skin.

For hair, the effects team designed a system that gave animators control yet was largely automatic. With this system, large clumps of hair created by the modeling department were put onto a character's head for the animators to pose. Inside the clumps, a hair shader could grow thousands of individual hairs, which followed the deformation of the clump. Lighting controls determined translucency and specularity.

Only an ogre could love taking a shower in mud. The mud is a fluid simulation; Shrek's wet skin is created with a shader.

For fur, the team used a geometry shader. In PDI's system, shaders, which are units of code, can bring up or deform geometry during rendering in the same way that surface shaders might create materials and bump maps. The fur shader, Bielenberg explains, uses triangular polygons with normals tweaked so they appear to be round. "For each piece of fur, you can control how many polygons to generate and the curvature," he says. "The curvature and the direction the hair is grown can be animated so we can get blowing fur."

This shader was used for Donkey, for stubble on Shrek and Lord Farquaad's face, for eyebrows and eyelashes, and to grow grass. "Grass is just long, green hair," Bielenberg says.

To create a scene with grass, the layout department would give ground geometry to the surfacing department, where maps would be painted to define the grassy areas. Grass softened hard edges between objects on the ground, served as groundcover in large vistas, and helped give the Shrek landscapes an organic look.

"The first CG movies were in contained worlds but because Shrek is a quest and a journey, we wanted to create the sense of a whole world," says Adamson. Shrek and Donkey start in the swamp, go through rural areas to Duloc-which Adamson describes as a kind of fascist theme park city-travel through fields and forests to the dragon's fiery keep, and then return the way they came.

The fluid simulator used in Antz now produces particles for mixed fluids such as the beer and mud in this WWF-style wrestling scene.

For these environments, the modeling department created some 1500 models and props, which the layout department organized into scenes. The effects department provided the complexity, creating fire, fluids, and foliage.

Fire was particularly important. "Since this film takes place in medieval times, fire is the light source, cooking source, and heating source, so we have every type of fire you can imagine," says Mark Edwards of the effects department. "We have torches, candles, fireplace fires, campfires, things catching on fire, and the dragon's fire."

Even so, the team used CG rather than live action elements to simulate the various fires. "Traditionally with effects, even in 2D animation, fire is a live action element that's composited into a scene," says Bielenberg. "We chose to use CG to have more control in motion and style. Often, though, CG fire looks too soft. We gave it a more hard-edged look that I think is extremely successful."

To do this, they used geometry at the base of many fires, particularly those in torches. This geometry is a cylinder that could be translated and deformed to match the motion of a torch being carried. Special animated shaders created a fiery turbulence inside the geometry, and particles sent the flames into the air.

In one dramatic scene, when Shrek tries to rescue Princess Fiona from the dragon, the dragon sends roiling, billowing balls of fire after them as they run across a wooden bridge. For this fire, the team used hand-animated, simulated spheres with volumetric noise shaders inside and with particle simulation helping create the final effect.

For the hot, oozing lava beneath the bridge, however, the team used PDI's award-winning fluid simulation program, FLU, created by Nick Foster, animation software developer. FLU was also put to work on the antithesis of fire-mud, an important element in a film with a swamp.

For Shrek, Foster expanded the simulator, which was first used in Antz for the flooding scenes, to allow disparate fluids to be mixed in one simulation-beer with mud for a WWF-style wrestling sequence, and water with mud for scenes in Shrek's swamp. In addition, hooks into the simulator gave the effects team controls to help characters interact with the fluids. For Shrek's mud shower, effects artist Juan Buhler devised a method for detecting when particles generated by the simulator collide with Shrek's skin so that he could change the skin's surface properties to make it look wet as the mud slides down Shrek's back.

Although the plants above vary slightly, all were duplicated and deformed during rendering using geometry shaders. A fluid simulator for wind, and virtual hair dryers on the characters' feet moved the grass, which was created with a fur shader.

One of the difficulties in working with a fluid simulator is that it produces particles, not surfaces, and most rendering techniques rely on surfaces. Rather than generating implicit surfaces, or so-called blobbies, as is often done in these situations, the effects team used a technique called scattered data interpolation to direct and tessellate the particle information. "I think you lose too much detail with implicit surfaces," explains Bielenberg.

Adding detail to the landscapes also became a challenge for the effects team. "This is a traveling road movie that predominately takes place out of doors, so we had to create trees and leaves and grass and dirt and dust and all of the great outdoors detail," says Bielenberg.

Effects lead Scott B. Peterson and a "digital greenhouse" team, with help from Gibbs, created a system that used geometry shaders to procedurally add detail during the rendering. In this system, geometry was duplicated in the renderer, and the renderer also deformed that geometry along an X or Y axis. Thus, without storing models on disk or using a lot of RAM, the team gave each flower in a field a different pose.

Similarly, trees were stored as a series of curves that were tessellated by the renderer, and procedural techniques grew tree leaves during rendering using a series of numbers that de scribed the distribution. "It's like the ultimate form of data compression," says Gibbs.

"We wanted to create a magical storybook world, so a large part of that is the grass and trees," says Adamson. "But if there isn't a little wind blowing in the grass and the trees aren't waving gently in the breeze in the distance, the scene doesn't look three-dimensional."

After experimenting with procedural noise and even with putting springs in the trees, the effects team realized they could use FLU to create eddies of swirling motion that simulated wind. By adding oscillation to the vectors produced by the simulation, they kept the grass and leaves from looking like they were moving under water.

"I think one of our achievements was to create a full world that is as rich an environment as if we'd gone out with a camera and shot in many locations across Europe," says Adamson.

So, having created that rich world filled with delightful characters out of simple zeros and ones, what reward was granted to those 275 people? They get the opportunity to do it all over again for a different story with a different look. In fact, PDI/ DreamWorks has already begun working on its third all-CG feature film, Tusker. Stay tuned.

Barbara Robertson is West Coast Senior Editor for Computer Graphics World.