When visual effects supervisor Jim Rygiel heard that The Fellowship of the Ring, the first film in The Lord of the Rings trilogy, had won the 2001 Academy Award for Best Visual Effects, he didn't spend much time celebrating.
That's because he and the artists at Weta, the Wellington, New Zealand-based production, post-production, and visual-effects facility that created the award-winning effects, were already busy working on the second film. "We knew the effects in film two had to be bigger and better than those in film one, and we had only a year to create them," he says. "So the pressure was on to raise the bar."
 |
Real actors (in foreground) overlook a digital army of Uruk-hai in formation in the valley below.
Image courtesy Pierre Vinet/New Line Productions © 2002. |
And raise the bar they did. The Two Towers, the big-screen version of the second story in J.R.R. Tolkien's epic trilogy, The Lord of the Rings, boasts roughly 920 digital effects shots—some 50 percent more than the first film. Furthermore, whereas digital characters had only supporting roles in The Fellowship of the Ring, several digital 3D characters have starring roles in the sequel. Even Weta's Massive crowd simulation software was made more, well, massive for the second movie.
A New Line Cinema project directed by Weta co-founder Peter Jackson, The Lord of the Rings film trilogy comprises The Fellowship of the Ring, which was released last year; The Two Towers, which hit theaters in December 2002; and The Return of the King, scheduled for a December 2003 release. Picking up where The Fellowship of the Ring left off, The Two Towers details the arduous journey that Frodo Baggins takes with fellow hobbits Sam, Merry, and Pippin in an attempt to destroy the Ring of Power.
Like its predecessor, The Two Towers features elaborate 3D environments, colossal armies of CG soldiers, fantastical 3D creatures, and photorealistic digital doubles of several of the film's principal actors. But while the types of effects and many of the digital tools were the same for both films, Rygiel says the quality and quantity of the effects, and the way in which digital technology was used to achieve them, are greater and more complex in the second movie.
For example, besides creating photorealistic 3D environments into which they composited digital and live actors, the artists also frequently built 3D environments to supplement or enhance live-action plates. "If Peter [the director] wanted to change the way a plate was shot, we would create a digital version of the plate in Maya, using simple 3D geometry. Then, we would project the plate as a texture onto that geometry," Rygiel explains. "By having a real plate projected onto a 3D representation of it, we could change camera moves after the footage was shot, or create entirely new camera moves." This gave the director more creative freedom without going to the expense of reshooting the plates.
According to Rygiel, although the artists used this projection technique in a couple of hundred shots in each film, they used the technique more extensively in the second. Many of those shots in the second film take place at Helm's Deep, a fortress nestled in a mountain cavern.
The Helm's Deep set appears in many shots. In some, the set was computer-generated; in others, parts of a full-size physical set were used. And at times, footage was shot of miniature Helm's Deep sets built in two different sizes: 1/4 scale and 1/85 scale. The artists used Apple's Shake to composite digital and live actors shot against bluescreen into the miniature and CG sets.
"We used the projection technique on a lot of the Helm's Deep shots so that Peter could, for instance, swoop down lower than a crane could, or zoom in more quickly than the live camera had," Rygiel says. "This resulted in more interesting shots, and saved a lot of money. It would have cost hundreds of thousands of dollars to get all the actors back and rebuild the sets to shoot those plates again."
In addition to the projection technique, the artists also used Weta's proprietary Massive crowd simulation and Grunt rendering software extensively in The Two Towers. One of the biggest sequences in which Massive and Grunt were used is the final battle of Helm's Deep.
In these scenes, a conflict is raging between tens of thousands of men and Uruk-hai, a breed of Orc soldiers created by the dark lord Sauron. The Uruk-hai are tall, large, strong, and covered in black armor. The armies of men and Uruk-hai are composed primarily of digital actors modeled and textured in Alias|Wavefront's Maya, animated in Massive, and rendered in Grunt. Real actors with make-up and prosthetics were used only for tight shots.
According to Rygiel, Weta significantly enhanced Massive for The Two Towers. To improve the movement and look of crowd characters, they developed Grunt, the shading and rendering program designed for use with Massive. "Grunt allowed us to get a lot closer to the soldiers and have them hold up larger in frame," says Rygiel. "Also, it provides better dynamics for things like cloth and hair so that they react realistically to wind flows."
The artists also enhanced the way that motion cycles are interpreted in Massive, as well as the intelligence in the software that allows crowd characters to make decisions as to how to move. "When these characters are in groups they need to 'see' and 'hear' what's around them," Rygiel says. "Massive has both artificial sight and hearing to aid in that process."
While The Two Towers' environment and crowd simulation is impressive, the character work stands out even more. "We did a rough calculation, and the rendering time of all the digital characters was about 460 years if rendered on a home PC," comments Rygiel. "That's about 4 million CPU hours." On the Weta machines, which included more than 1000 SGI and IBM Linux workstations, rendering took about 10 months.
Weta modeled and animated 15 different 3D characters that play fairly dominant roles, plus scores of warriors for the battle scenes. In addition to the armies of Uruk-hai, the artists also created armies of Orcs. The size of small men, Orcs are fierce, hideous-looking warriors with hunched backs, bowed legs, charred skin, and fangs for teeth. When they're not charging on foot, the Orcs go into battle on the backs of Wargs, which resemble large wolves and can bring down a horse with a single bite. Like the Uruk-hai, the digital crowds of Orcs and Wargs were modeled and textured in Maya, then animated and rendered in Massive and Grunt. To create the hair on the Wargs, the artists used Maya Fur.
Film two also includes Balrog, which made its debut in film one. Composed of fire and smoke, Balrog is a 3D model built and animated in Maya. The artists generated the smoke and fire through Maya particle effects, and rendered them through Pixar's RenderMan.
In addition to Balrog, The Two Towers also features a few cave trolls based on the 18-foot-tall cave troll character that was one of Weta's most ambitious NURBS models in the first film. According to Rygiel, the new cave trolls are essentially copies of the initial model. To create that character, the artists built an animatable version of the troll derived from 3D data acquired by laser-scanning a physical model built by Weta's Workshop division. They also created a high-resolution, detailed version by using Alias|Wavefront's PowerAnimator to build NURBS patches on top of the scanned point cloud data.
Then, using a proprietary Maya plug-in, they imported the NURBS patches into Maya, where they cleaned up the resulting data. They added detail to the trolls using Artisan, Maya's brush-based sculpting tool, and then, as a final step, extracted the NURBS patches as a displacement map and applied the map to the animated model at render time.
The artists also wrote a muscle system in Maya that has dynamics and runs atop the troll's skeleton. They used the muscle system to drive the animation of the skeleton, which in turn drove the NURBS model.
 |
| Gollum, who was modeled and animated in Maya, was challenging to create because he needed to have human-like characteristics, and because he had to share many scenes with human actors. |
One digital character that plays a leading role is Treebeard, a walking, talking tree. The other is Gollum, a creature who has become frightening and somewhat evil after encountering the Ring. Merry and Pippin team up with Treebeard, while Frodo and Sam partner with Gollum in their quest to return the ring to Mt. Doom.
In some scenes Merry and Pippin interact with Treebeard. So, rather than build and animate a 3D model of the character, which would have been difficult to composite seamlessly with the live-action actors playing Merry and Pippin, the artists used a latex puppet created by Weta's Workshop division for Treebeard's body and shot the footage practically. Later, in post, they added a digital head and face, modeled and animated in Maya. To track the digital head and face to the puppet body, they used Science-D-Visions' 3D-Equalizer. All lip sync animation was done in Maya.
Treebeard also appears in a sequence in which he is directing an army of fellow leaf-covered Ents as they march to Isengard to attack Sauron and the Orcs. "It's a pretty cool scene. You've got this forest of walking trees, and each one has human facial features, so they look like humanoid trees," Rygiel says. Some of the Ents were modeled and animated in Maya, grouped in Massive, and rendered in Grunt and/or RenderMan. Others were modeled, animated, and rendered in Maya.
While Treebeard is composed of a combination of practical and digital effects, Gollum is a completely digital character that, according to Rygiel, was quite challenging to create. "With the other creatures we had some liberty to go in different directions because they weren't supposed to be as humanlike," Rygiel says. "But we knew that Gollum had to be perfect.
"It's easy to get digital characters 80 percent of the way there, but achieving that final 20 percent takes a long time," Rygiel continues. "You're literally working on individual eyelashes. Your level of detail gets down to the pores on the skin, the moisture in the eyes. We had to achieve that level because in some cases Gollum appears in full head shots, where his head and face fill the frame."
To create Gollum, who appears in about 270 shots (many alongside and interacting with live actors), the artists began by modeling the character in Maya. To ensure a high level of humanness in the character, the artists added some facial features belonging to Andy Serkis, the English character actor who supplied Gollum's voice. "Once we put some of Andy's facial features into Gollum, it started selling him as a real rather than a CG character," Rygiel recalls.
For most of the shots, the artists hired Giant Studios, a motion-capture facility based in Atlanta but with a satellite office at Weta, to capture Serkis's body movements using its proprietary motion capture system. Then the artists used the mocap data to drive the animation of the 3D Maya model. For shots in which Gollum had to interact with a live actor, they shot the footage with Serkis as Gollum's stand-in, then used Maya to paint Serkis out of the frames and Shake to composite Gollum into them. "In those cases we used Maya to roto-animate the character to Andy, so wherever Andy moved, we moved Gollum frame by frame," Rygiel explains.
Meanwhile, all facial animation for Gollum was accomplished in Maya. The few strands of hair on his head were created in Maya Fur, and his loincloth was made in Maya Cloth. Because Gollum is so scantily clad, a lot of his muscular physique is revealed, which also needed to look photo-real. For this task the artists used the muscle system they wrote in Maya so that when Gollum moves, his muscles flex realistically.
According to Rygiel, The Two Towers represents Weta's best work since the studio formed in 1993. "We did some spectacular work in film two. I'm extremely proud of all the artists who poured their hearts and souls into this project," he notes. At press time the artists had just begun working on The Return of the King, the final installment in The Lord of the Rings trilogy. And Rygiel adds that already it looks like the third film will surpass the others in terms of the quantity and quality of digital effects. "From what we've seen so far," he says, "with film three we'll be raising the bar yet again." ..
Audrey Doyle is a freelance writer based in Boston.
toolbox
Alias|Wavefront www.aliaswavefront.com
Apple www.apple.com
Giant Studios www.giantstudios.com
IBM www.ibm.com
Pixar Animation Studios www.pixar.com
Science-D-Visions www.sci-d-vis.com
SGI www.sgi.com
Weta Digital www.wetadigital.com