Keys to the Kingdom

Visual effects were still “special effects” when Lucasfilm produced Paramount Pictures’ first Indiana Jones film in 1981. Filmmaking was an optical process, and the most radical application of computers in Raiders of the Lost Ark was to control the motion of cameras that shot miniatures. Industrial Light & Magic (ILM) created the effects for the film. It was the young studio’s third movie and its first non-Star Wars project.

Directed by Steven Spielberg and based on a story by George Lucas, the rollicking action-adventure film scored four Oscar nominations, including those for Best Picture, Cinematography, and Director; won four Oscars, including Best Effects, Visual Effects; and raced to the top of the box office: The $18 million production grossed $384 million worldwide.

Fast-forward to 2008 and Paramount Pictures’ Indiana Jones and the Kingdom of the Crystal Skull, based on a story by Lucas, directed by Spielberg, starring, as before, Harrison Ford as the intrepid archaeologist, and with effects created at ILM. Set in 1957, 19 years after Raiders and at the height of the Cold War, the film opens with a gag for Indy fans: In previous films, the Paramount Pictures logo dissolved into a mountain. This time, the logo dissolves into a prairie dog mound.

The prairie dog, a CG character created at ILM, pops up, looks around, and narrowly escapes a bunch of teenagers speeding past in a convertible who drag race with a convoy of army jeeps. The jeeps turn onto a restricted road leading into a nuclear test site. Inside, we find Indiana Jones (Ford) kidnapped by Russians—in particular, one Irina Spalko (Cate Blanchett), who insists that he help her find a powerful crystal skull.

Pre-release publicity for Crystal Skull made much of Spiel­berg’s intent to film this fourth sequel using techniques from the previous three, all created in the ’80s. “The lighting in this film was very similar to the first films,” says Pablo Helman, visual effects supervisor. “Steven Spielberg even shot with some of the same lenses. So, our visual effects had to capture the same style he was shooting. And, the effects had to be invisible.”

So, did the studio turn back the clock and use old visual effects techniques? “We thought about it,” says Helman. “For about a second.”

All told, ILM’s crew of approximately 350 created 540 shots and touched 48 minutes of the film. In addition to digital prairie dogs, ILM’s CG monkeys and ants scamper through the scenes, but much of the studio’s work, as in the first films, was environmental. For example, in Crystal Skull, ILM extended a set with an elaborate digital environment to expand a spectacular shot in a warehouse filled with packing crates, a flashback to a similar warehouse shot in Raiders’ dramatic finale.

“The shot gives you the geography of this immense place,” Helman says. “Steven [Spielberg] had a specific way of framing and making compositions out of really long shots that connect characters and geography. Throughout the show, our extensions are there, but we always start with a location or set because it doesn’t confuse the reality of the film.”
      
It confused the critics, though. In a good way, for ILM. One reviewer, for example, complained about CGI waterfalls, but the waterfalls were real.

Other critics laud the traditional physical effects during a swashbuckling fight sequence through a jungle between Mutt Williams (Shia LaBeouf) riding in a DUKW (amphibious duck boat) and Irina riding in a jeep. At one point during the fight, while LaBeouf straddles the two speeding vehicles, jungle plants whip beneath and between his legs.

Writes Joe Neumaier of the New York Daily News, “In an era of CGI thrills, Blanchett and LaBeouf’s sword duel atop moving jeeps and Indy’s fistfight with a handful of thugs are like watching an ancient art reborn.”

The “ancient” art of digital matte painting taken to an extreme, that is. To create the sequence, ILM intercut bluescreen shots of the actors in digital backgrounds with live-action footage. The result even fooled Richard Corliss of Time magazine and Roger Ebert of the Chicago Sun Times.

Corliss: “A lot of the elements in Crystal Skull may feel like mandatory reprises of the old tropes, but the high-speed, two-vehicle fight between Indy’s team and Irina’s goons is up there with the Raiders jeep sequence, more complex and sophisticated in its engineering of physical action…. If there’s a scene that film students will be poring over, decades from now, this is the one.”

Ebert: “We get such sights as two dueling Jeep-like vehicles racing down parallel roads. Not many of the audience members will be as logical as I am, and wonder who went to the trouble of building parallel roads in a rain forest. Most of the major characters eventually find themselves at the wheels of both vehicles; they leap or are thrown from one to another, and the vehicles occasionally leap right over one another.”
It’s a sequence worth examining in detail.

Jungle Juice
For the live-action footage, the production unit cut a lazy S-shaped road that crisscrossed an existing dirt road through the jungle on a private estate in Hawaii, and then cut a straight road through the S, creating, in effect, a dollar sign. Two camera crews on the original road—one in front and one behind the actors’ vehicle—plus two camera crews on the S-shaped road, filmed the action to provide Spielberg with shots from every direction.

“Although they did shoot some stuff with LaBeouf in a ‘hangman’s rig’ so he could straddle the two vehicles, there was no way to do most of the stunts in the jungle—the road was bumpy and it was too dangerous,” says Marshall Krasser, associate visual effects supervisor. Thus, of the six-minute sequence, about half the shots had CG effects. Of those, some were bluescreen shots with synthetic jungles; others were location shots with digital jungle added to the roads.

To help make it seem as if Mutt Williams and Indiana Jones crashed through the jungle while escaping from the Russians in such live-action shots as this, ILM filled the road with digital plants in postproduction. For about half the shots in the jungle chase sequence, though, the environment is digital because the actors rode in vehicles on bluescreen stages.

“The challenge I had on location was to stitch moving images through the jungle,” Snell says. “I’d use four or five passes of the jungle to create running footage for virtual backgrounds.” Because the vehicles carrying the camera crew moved at different speeds, Snell retimed all the footage before organizing the passes.

To assemble the passes, which he did using ILM’s Zeno software, Snell created a 3D environment by photo-modeling selected elements—the roadway, for example. Then, he used those elements as anchor points to matchmove the running footage. By locking each pass to the same 3D geometry, he created one environment. “In essence, we could have used a fish-eye lens to shoot 180 degrees through the jungle,” he says. “But, we wanted to blow up and use any section as a background, so we needed higher resolution standard lenses.”

Once Snell had all the cameras in the running footage matchmoved to the geometry, he projected the footage onto screens in 3D space in front of a virtual “hero” camera. The hero camera matched the movement of the camera used to shoot the bluescreen elements. “As the hero camera pans across the projected screens, it picks up the jungle images, so it’s as if the camera is moving through the jungle,”  he explains.

Krasser describes the footage as looking into a geometric tube. “We had the right perspective going back,” he says. “We weren’t looking at flat areas. And, we could move the camera left and right, up and down.”

To film the action on the bluescreen stage in Downey, California, the actors rode in vehicles on gimbals that moved left, right, up, and down. In postproduction, ILM separated the vehicles to color-correct each, install rotating CG tires, give the vehicles shake from the bumpy roads, and add visual complexity. “The bluescreens had consistent lighting, but our vehicles needed to move in and out of sun and shadow,” Krasser says. “And, we might slow one down and speed up another to make it seem like the drivers  struggled to keep even.”

Then, to make the environment more interesting and interactive, digimatte artists added digital jungle plants—the thistle plants, for example, that whip between LaBeouf’s legs as he straddles the speeding vehicles—using the matchmoved 3D environment as a platform.

“The digimatte department had a library of plants modeled from reference plants shot in the jungle that they could drag and drop into the scenes,” Krasser says. Each of the plants in the jungle, from grass to vines, was “sim-able”—ILM’s simulation department applied the real-world dynamics based on proxy models of the vehicles.

“When a jeep hits a plant with force, it breaks apart and flies back at the camera,” Krasser says. “That allowed us to have a dynamic interaction between the bluescreen footage and the digital environment. Normally, bluescreen shots are simple—a shot out a car window. But, this was a complex, intense sequence. We’re flying down the jungle in a digimatte background. The artists really pushed synthetic environments to a new level.”

To match the footage Spielberg had shot, ILM added volumetric lights to the bluescreen footage, mist, dust, and insects swarming in sunspots. They sent plant parts flying back and hitting the camera. They created lens flares and light flashes. They added camera shake and ran the camera into trees.

“We followed the master,” Krasser says. “We tried to simulate what he’d shot.” They even added pools of light to background plates that Spielberg shot on overcast days. “A lot of plates have heavy work on them,” Krasser says. “I think it’ll surprise people.”

A Big Blast
Like the jungle sequence, ILM carefully inserted CG elements throughout the film—all but one monkey is CG, for example, as are all the ants. “We had a wide variety of difficult effects work,” says Craig Hammack, CG supervisor. “Crowds, fluids, particles, fire, and water. The difficult part was that Indy has such a history of great physical effects, we didn’t want the CG effects to topple down the house of cards. The Indy everyone knows and loves is dirty, gritty, and you’re in the action with him.”

One dramatic shot, for example, has Indy tumbling onto the side of a mountain in time to see a mushroom cloud form after an atomic blast. The beautifully framed fireball with the iconic shape rising into the sky fills half the frame. Indy is in the foreground.



For this final shot with simulated water crashing down and mixing with real water (below), ILM started with live-action plates from the bluescreen stage (top), adding the waterwheels and other elements (middle). The colors represent fields that affect the water simulation.

Creepy Crawlies
The car chase through the jungle halts when both cars land on top of giant dirt mounds. Thousands of red ants swarm out of the anthills, flow over the vehicles, and attach to people.

“We’ve always done crowds within [Autodesk’s] Maya framework using our plug-ins,” says Hammack. “We don’t have AI baked in, like Massive or the crowd tools in [Softimage] XSI; we have to control every little movement. But, we find it beneficial in most cases to have a fine level of rules.”

For Indy, ILM created a rule-based crowd system in Zeno driven by curves and “Open Steer” flocking code. Open Steer is an open-source C++ library of tools initially developed by Craig Reynolds in the R&D group at Sony Computer Entertainment that grew from his Boids flocking software, which he introduced in 1986.

“We could have the ants crawl up people as they were running by using proxies for the actors and, in some cases, tight match animation,” Hammack says. “Having the system in Zeno made it more convenient for lighting. As the software matures, we’ll have it working in combination with fluid and particle simulations.”

Crystal Clear
The most dramatic effects, however, take place in a circular room in the heart of a temple, a practical set that ILM matched with a digital model. Inside the ancient room, 13 crystal skeletons sit in alcoves on thrones made of bricks. During the long sequence, the five tiers along the inside of the wall swirl layer by layer. The floor spins, the thrones turn to rubble, and the room breaks apart. The room is approximately 50 feet across.

“We had to see and separate out every brick,” Hammack says. “We couldn’t skimp on geometric detail, so we built the room brick by brick.” Moreover, the ancient bricks didn’t lend themselves to simple CG models.

“They didn’t have crisp, clean lines,” Hammack says. “They had chewed edges, and we had to break them in an organic way. Everything had to crumble across a dozen shots.”

At top, to animate the crowd of digital ants terrorizing Irina Spalko (Cate Blanchett), ILM used a rule-based crowd system in the studio’s Zeno software. Above, this CG prairie dog created by ILM stars in one of the first shots in the film; his digital cousins make cameo appearances later.

So that the modelers could see the result, yet still build the room with the bricks, the fracture program produced the rubble and then seamed the bricks back together to perfectly match the original shape. Because the tool retained the UV mapping from the original bricks, when the bricks fractured, the textures stayed on the pieces.

To break apart the bricks, the crew defined each of the rubble bits within the apparently solid bricks as a rigid body. When the creature development artists applied forces to the bricks through the Physbam simulation software, the bricks shattered, and each piece of rubble affected the movement of the other pieces.

Then, from the creature dev artists, the shots moved to the technical directors, who added such effects as sand and dust by emitting particles from the rubble. The TDs also lit the room using particles to represent the heavier geometry.

“This model was a giant amount of data,” Hammack says. “We couldn’t load the entire room, so we worked with it in wedges. Unfortunately, the shot design called for different sections to rotate at different speeds. So, parts of one wedge rotated into parts of another. It was a complicated setup. In many cases, we relied on conversations between the creature dev artists and the TDs. They might say, ‘We’re using wedge 891213, so don’t worry about the rest of it,’ but in a lot of cases we couldn’t do that because the rotations were so different; we had to use all the wedges all the time.”

While the temple is breaking apart, the crystal skeletons merge into one and become fleshy. Irina begs the fleshy being for information, and it complies. “The whole movie is about seeking information about a higher entity,” Helman says. “The moral is, ‘Don’t do that because your eyes are going to burn.’” They burn, thanks to ILM.

Snell created the animatic for that sequence using live-action plates of Blanchett looking around the room, and low-res geometry from the modeling team. The animatic became the layout camera for the shot.

When Indy escapes the temple destruction, he navigates a kind of waterwheel, and somehow ends up looking down onto the temple in the center of a valley. The valley encircling the temple fills with swirling matter, everything starts breaking apart, and the temple collapses in the middle. “It’s a very long shot with a static camera, so we get a good look at everything that’s happening,” Hammack says. “All the pieces rotate, levitate, come to a standstill, drop, fall to the ground, and crush themselves.”

Once again, the team used Physbam’s rigid sim engine to control pieces modeled using the fracture tool. Cylinder fields helped control the rate of the swirl. “We had to choose carefully where we’d let the collisions happen,” Hammack says. “We explored many ways to get that done. At the end, it was a joint effort. We’d run a rigid sim, send it to an animator to correct for general timing and gross motion, and then send it back to creature development to run secondary collisions.”

At the end of the sequence, the valley fills with water. “We got lucky,” Hammack says. “Pablo [Helman] chose to do a miniature shoot for the water spilling into the valley, so we didn’t need to simulate water covering the floor of the valley, just the residual filling. Otherwise, we’d probably still be working.” 

Doom Town During a chase sequence on the atomic test site, Indy finds himself in a perfect little town. Then, he discovers that mannequins, not people, populate the town. Kerner Optical built a miniature town in one-eighth scale, creating 70 mannequins, 30 cars, and 30 fully furnished houses with 30,000 roof shingles for the film’s atomic blast to destroy. ILM built digital extensions of on-set facades built on a bluescreen stage, so the live-action footage would match the miniatures. All told, Kerner used 1100 feet of prima-cord—an explosive rope—to blow up the town; ILM enhanced the estruction with digital effects.

“When you have a director like Steven Spielberg, who’s so enthusiastic and loves what he’s doing, it’s infectious,” says Krasser. “And, it’s Indiana Jones. We wanted to give Steven Spielberg and George Lucas effects that looked like they could have been shot.”

Creating invisible effects is a typical goal for ILM and other visual effects studios, of course. But to create effects that look like those a crew might have created in the ’80s, yet are sophisticated enough for a 21st century audience? That took new technology and skilled artists who could dig into 30 years of filmmaking knowledge at the effects studio that helped whip out the first Indiana Jones film.

Mac (Ray Winstone): “This ain’t gonna be easy.”

Indiana Jones: “Not as easy as it used to be.” 

Barbara Robertson is an award-winning writer and a contributing editor for Computer Graphics World. She can be reached at BarbaraRR@comcast.net.