Evolution

The summary blurb on the Rotten Tomatoes website says everything a director and visual effects studio could wish for: "With intelligence and emotional resonance to match its stunning special effects, Dawn of the Planet of the Apes expands on its predecessor with an exciting and ambitious burst of sci-fi achievement."

A few days before release, the approval rating for the film on the website, which aggregates reviews, averaged an astounding 98 percent.

"I'm more proud of this film than I have been of any film in a while," says Erik Winquist, one of three visual effects supervisors at Weta Digital working under overall VFX Supervisor Dan Lemmon and with the guidance of Joe Letteri, senior VFX supervisor and Weta Digital director.

"It's rare when you get to work on something as strong," Winquist continues. "The story, the performances, the visuals. Everything was firing on all cylinders."

The film brings back Andy Serkis in the role of Caesar, the chimpanzee in Rise of the Planet of the Apes who was dosed by the human character Will Rodman (Actor James Franco) with an intelligence-enhancing drug. In that film, Caesar releases a gaseous version of the viral drug in a primate shelter. After a confrontation on the Golden Gate Bridge, Caesar, the apes from the shelter, and others from the zoo, escape north into Muir Woods. The credits for that film set up this one. The virus, which enhanced the apes' intelligence, kills people - and it's spreading around the world.

Dawn of the Planet of the Apes film opens with a news report about the pandemic caused by the virus. It's 10 years later. Caesar is now the leader of a large community of apes living in Muir Woods. He has a wife, a teenage son, and a newborn. He chases through the forest to hunt food. Unbeknownst to Caesar, a colony of surviving humans lives across the Bridge. When they enter the Marin County woods to re-activate a hydroelectric dam, the conflict begins.

All the apes in both films are CG characters. For Dawn of the Planet of the Apes, a crew of 900 people at Weta Digital worked on 1,100 visual effects shots, some for a few weeks, others for a year and a half to two years.

"We were the only company that created the apes and the environments," Lemmon says.

The Setting

Although the action plays out in the San Francisco area, Director Matt Reeves shot the stereo film largely in British Columbia and New Orleans, with some plate photography in San Francisco.

"We shut down several streets in New Orleans around the intersections of Rampart, Common, and Tulane," Lemmon says. "That stood in for Market and California streets in San Francisco."

Caesar, at left and above on horseback, leads his fellow apes through a set built in the streets of New Orleans, enhanced by set designers to look like decrepit San Francisco, and augmented by artists at Weta Digital.

Set designers covered buildings with vines and other signs of degradation. "The set was amazing," Winquist says. "They built a 30-foot facade in a vacant lot and dressed buildings in present-day New Orleans to look as if they had not been maintained by man. Trees grown wild. Weeds in the pavement cracks. It's hard to imagine things would go that far off the tracks, but they looked at a city in the Ukraine that was taken over by nature after Chernobyl."

Weta Digital artists extended the film sets and added such iconic San Francisco landmarks as the Zoetrope and Transamerica buildings and the Golden Gate Bridge to the plates. Then, they augmented the images to create the type of degradation that would occur if nature had destroyed the city. One sequence set on California Street, though, was entirely CG.

"We had this wealth of dressed sets in New Orleans and in Vancouver, and could take cues from how degraded and overgrown they were to extrapolate them all the way up California Street in San Francisco," Winquist says. "Matt [Reeves] and his director of photography, Michael Seresin, gave us lush, beautiful plate photography to work from."

To create the digital shots in San Francisco, the Weta Digital crew spent several days documenting the location using Lidar scans and gigapixel panoramic photography. Then, the artists built on that using 3D assets and 2.5D matte paintings.

A power plant in New Orleans stood in for San Francisco's Fort Point location, which in the film became a FEMA staging area for weapon stockpiles. Weta Digital artists removed smoke stacks from the plates and added the Golden Gate Bridge behind.

Mocap, Weather or Not

Within those beautiful locations were cast members of the ape community who joined Andy Serkis (Caesar) on set. Actor Terry Notary again performed Rocket, a chimpanzee who is Caesar's second-in-command, and Karin Konoval returned as Caesar's third-in-command, an orangutan named Maurice. Koba, a fierce bonobo (pygmy chimpanzee) took on a larger role than in the last film with the help of actor Toby Kebbell. In addition, actors playing other hero apes were on set, too.

The orangutan Maurice (pictured in the middle of the front row) has 900,000 hairs compared to a horse's 3.2 million, which made fur computation trickier for the horses. Animators worked from motion capture for all apes in small crowds such as this.

"The key to this whole process, the way we've done these films, is that we've worked with really talented human actors," Winquist says. "They are the characters. Andy is Caesar. Toby is Koba. Karin is Maurice. Their performances. Their decisions on set. These human actors working with the director are the backbone of the digital characters."

As before, Weta Digital captured the actors' performances on location during filming using motion-capture cameras, helmet-mounted facial rigs, and spots of LEDs on the actors' bodies.

"This was the first time we filmed entirely on location," says Konoval. "The weather created a visceral environment - we were there. But we were in some incredible stressful environments. It was wet and muddy."

Anticipating the challenges of shooting in funky weather on location with several hero apes, the Weta Digital crew took lessons they'd learned from Rise farther down the road.

"We knew rain played into the story," Winquist says. "Matt [Reeves] was very interested in playing the apes living as one with nature. And, we knew they would film in the springtime in rainy Vancouver and hot, humid New Orleans."

A new motion-capture system gave the Weta Digital crew the ability to gather performance-capture data from the actors in extreme locations and weather.

"We took the same tools we use on stage and made them mobile," Lemmon says. "We worked with Standard Deviation to have wireless motion-capture systems that we could freely place around the set without being tethered to switches and boxes; they are more weatherproof and robust. Some sat 10 feet from walls of fire and explosions, and still delivered good capture data. They were in rain, sleet, and snow."

That gave the crew a huge advantage over the setup they had used for Rise, which entailed putting the gear on a frame.

"We could strap cameras to trees," Winquist says, "put them on stands off-camera…wherever we wanted without worrying about routing cables to a production cart like we had to do on the last film. That allowed us to avoid holding up production, and it gave us loads of great data because we could sneak into tight corners."

Even so, the crew typically used a familiar configuration.

"We'd try to set up the cameras similar to what we'd do on a stage, to get as much coverage as we could and avoid occlusion," Lemmon says. "We'd use eight to 40 cameras, depending on the scene and how much time we had to set up. As we moved from wide to tight shots, we'd often add more cameras for more detail and coverage. For wide shots, the data could be sparse because we knew we'd fill in background characters with studio performance-capture sessions. The great thing about having an accommodating production crew is that we became friends with the greens department. They would use bits of moss or hanging vines to cover the cameras."

Facial animators used data captured from actors on location who wore a helmet with an attached camera and from reference footage to create subtle, ever-changing facial expressions. 

Whenever someone on the crew added or moved a camera, the team would re-calibrate the system.

"We needed to have at least a wand with a set of markers that we could swing through the volume for each new setup," Lemmon says. "In some cases, if we were just adding a couple cameras, we could adjust the calibration. And, in the same way that a cinematographer would pre-light a scene, we would jump ahead and try to pre-rig as much as we could."

In addition to the wireless cameras, Weta Digital worked with Standard Deviation for other equipment.

"We use a lot of their gear," Lemmon says. "The motion-capture cameras, the equipment the actors wear on their helmets to record facial movement, and the equipment that drives their active motion-capture markers. The markers are LEDs encased in rubber and silicon that run on an infrared spectrum. They flash short bursts just as the motion-capture camera opens its shutter. The data goes to the mobile command center along with the witness camera footage, all wireless as well."

The witness cameras are prosumer-grade Sony cameras that record movements and facial detail. If it became impossible to use the full-performance capture kit, those cameras made it possible to do "faux" cap.

"The actors wore suits made of Velcro-friendly material," Lemmon says. "When we couldn't use the active markers, we stick on high-contrast dots and track those markers optically later."

Actor Andy Serkis, on the horse in the motion-capture suit and facial-capture helmet, talks with actor Jason Clarke (Malcolm).

Konoval describes the process from an actor's point of view: "It was a daily wrestle getting into those gray motion-capture suits," she says. "They had lots of Velcro and LED sensors with wires that attached to a battery pack, like a turtle shell on the back. Each day of filming, they'd fit a mask on our faces and paint 52 dots into holes on specific muscles. I don't even remember the number of scans they did of our facial structure that went on outside the filming to meticulously determine the muscles in our faces. But the Weta experts were the most grounded, decent, and fun people to work with." 

The camera on the facial helmet gave the crew 60 frames-per-second, high-definition data, a solid base for the animators.

"The facial animators looked at the reference data as well," Winquist says. "It isn't just about making sure Caesar blinks at the same time as Andy. It's about making sure the animators interpret the performance faithfully. Not the geometry, the emotional performance. We look at the video frame by frame. In the blink of an eye, we can see an emotional state change from elation to fear to concern. We have to find the facial muscles to fire to make that emotional change happen on the ape, even if it deviates from the actor's performance."

And of course, when the apes needed to accomplish actions impossible even for skilled stunt actors, animators created those performances using keyframe animation.

More than Digital Makeup

Animators used a familiar system of controls and workflow to translate the motion-capture data into the apes' final performances. Under the hood, however, was a more complicated network of shapes than before.

"For the first film, we had looked primarily at chimpanzee reference," says Dan Barrett, animation supervisor. "Not exclusively. We included elements of Andy's face in Caesar, for example. But in this film, we had more hero ape work and more dialog. So, we had to go back to the face puppets and work out how to deal with the dialog. We ended up with human variations of certain mouth shapes."

An obvious example is the funnel shape used to say the "sh" phoneme. A chimpanzee's lips can extend into a very long pucker forward, something that doesn't work well with dialog.

The horses were sometimes live action, sometimes CG.

"The funnel was one with a human variation to have a fuller lip and more tightening," Barrett says. "We also looked at eyes and eyebrows. Chimps tend to have almost a mono-brow with a little shelf. Humans have more independent eyebrows and a creasing in the forehead between. So, we created brows closer to human eyebrows. We wanted to get Andy's furrowed brows, and the distinctive creases when Toby frowns."

The translation from data to animation curves was automated to some extent. "The solver does a good job, but it can make a fairly complicated network of curves, so often we need to simplify," Barrett says. "And sometimes the motion editors needed to do full keyframe. It runs the gamut. Jack [Eteuati] Tema, our lead facial motion editor, prepared a lot of the dialog shots. He pretty much started from scratch for the mouth in particular. It was easier because then he could structure the curves the way he wanted from the jaw up to the lip shapes. The solver gives us a fine blocking pass, especially for the body, but there is still a lot of skill required."

On set, the actors mimicked ape movement, whether bipedal or on all fours, and all the actors wore arm extensions. "That was probably the trickiest thing," Barrett says, "having the confidence to take your body weight on those arm extensions in a squat. But the actors on the whole did incredibly well. Except for the high-action stuff, what you see is what we got on the day."

Konoval had a particularly interesting challenge. The 120-pound woman had to perform as a 300-pound male orangutan.

"When I was first cast in Rise, it was a challenge finding the physical weight of him with each step," she says. "So we put five-pound weights on my arms, which made the work more difficult. By the time we went to Dawn, we no longer needed the weights. I had found Maurice's weight. The challenge for this film was not in dealing with the motion-capture technology. It lies within finding the integrity of the character. The thoughts, the voice. Everything. It's the most rewarding and challenging role I've ever taken."

Notary, who plays Rocket, helped actors new to the films understand chimp motion. "He's one of the unsung heroes," Barrett says. "He made our lives so much easier."

And, Jason Chu, a former dancer and gymnast, helped the Weta crew capture performances that were too difficult for the actors.

"On the first film, we assumed we couldn't get that power," Barrett says. "But Jason was so impressive climbing trees and swinging, we added his climbing motion to our library."

Scenes with the apes riding horses bareback took a variety of approaches. "We had only one shot in Rise with Caesar on horseback," Lemmon points out. "But, this film is filled with that imagery. Anywhere we could, we tried to use real horses, but as most successful strategies go, it was a mixture of real and computer-generated horses."

When they could get the performance they wanted from the animals, the crew shot bareback horses. More often, they shot riders using racing saddles covered with something that looked like horsehair and with stirrups hinged up.

"We made it as easy as we could for the people who had to paint that out later," Lemmon says.

The digital horses were evolutions from those built by the studio for Abraham Lincoln: Vampire Hunter.

"We created the horses in partnership with Massey University's equine research department in Wellington," Winquist says. "We did MRI scans for the musculature and lots of motion studies on treadmills. For this film, we took that work to the next level with our latest muscle-based technology for creatures and our tissue solves. From a fur computation standpoint, horses were trickier than the apes. Maurice has 900,000 hairs. A horse has 3.2 million. And we ran fur simulations on their manes and tails. We worked with the shader department to make sure we had BRDFs (bi-directional reflectance distribution functions) that approximated body fur so we wouldn't have to render fur on horses far from camera."

As for the animation, Barrett's team had to modify the motion of riders captured on horseback in saddles with stirrups to create believable apes riding bareback.

"The saddles and stirrups change the way the bodies move, so we had to iron that out," he says. "And as soon as they stood up, it was all over for us. We had to move the weight to the rear and have the apes grip with their feet and ankles."

In addition to the hero apes, the animators had thousands of individual apes to control.

"One of the TDs told me there were 20,000 individual apes," Barrett says. "Those were instances and we used Massive for big crowds. The motion-capture team did a huge amount of crowd work. But when we had 100 or 200 apes, the directors wanted more specific motions - less Massive brains, more human brains. So we captured animators for those."

Clumps of Fur

All the apes had fur that needed to be grown, groomed, and rendered, but the hero apes who performed in the rain presented particular problems.

"We did wet fur for King Kong," Lemmon says. "But, we're at a whole different level now. Making the fur look wet, matted, and mangy was important because so much happens in the rain. We had rebuilt our hair and fur system for The Hobbit, but for this film, the grooming tools had to talk to simulation tools. They had to work together to drive the pooling of water and the clumping of fur. We hadn't put a water simulation through fur and onto the skin before."

A good deal of the feature film takes place in wet weather, so computer graphics scientists at Weta Digital rebuilt the hair and fur system to work well with water-simulation tools.

To help link hair dressing to simulation, researchers grew Wig, a new dynamic rod-based system, from the studio's Barbershop.

"When fur gets wet and starts to clump, it goes through different phases," Lemmon explains. "The water beads on top in some cases, and in others, it wicks down and gets sucked into the clump. We had to be able to model wet fur and set it in different types of clumps. Then, we could pre-saturate areas with water based on that clumping and drive additional water though it in that state. Drips enter the simulation, causing pools to get larger. They break free and fall on skin and other bits of fur. We tried to be efficient by simulating only the part of the ape visible to camera, but it was still gigabytes of data being pushed around."

As this happens, shaders send light into the fur and cause dual scattering. "We had to be very smart about shading the wet fur," Winquist says. "It was easy to make it look like too much hair product."

Peak Performances

The film's climax, the third act, sends Caesar and Koba to the top of the Steuart Tower at One Market Street in San Francisco, a skyscraper under construction in the film but in real life is a 27-story building that houses Autodesk and many other companies.

"That fight didn't get conceived until after production had wrapped," Lemmon says. The production unit brought in a fight coordinator who worked with the stunt performers and actors to choreograph action that moves through the high-rise construction site.

"We shot a big motion-capture pick-up with Andy [Serkis] and Toby [Kebbell] and the stunt guys fighting through different beats of the third act," Lemmon says. "Then we created a new scene. When there were gaps in the performances, we filled in with new beats, capturing ourselves in some cases for more mechanical or background character action. For hero moments preconceived after the edit, we did another pick-up in March. It was a fluid, changing scene that came to shape over a couple months."

The environment for the new sequence was entirely digital.

"Keith Miller looked after a team that created the digital skyscraper environment from scratch," Winquist says. "It was a big departure from the environments on the ground that had plates, so we brought in Kim Sinclair [VFX art director] to flesh out the background and make sure it had all the details and rough stuff you would find in an under-construction skyscraper. As a VFX supervisor on Tintin, Miller had looked after architectural backgrounds."

Animators translated mocap from actors directed on location to preserve emotional moments such as this.

To make the world more interesting, the director asked for wind, swinging lights high up in the girders, and a particular style of lighting.

"We gave the dramatic attack sequence a rich-orange, sodium vapor environment, like present-day downtown San Francisco at 10:00 at night," Winquist says.

It's a scene filled with action and emotion. Caesar fights despite being weak from wounds suffered in a previous attack, several apes are injured in this battle, and the war with humans has begun.

"Technically, the biggest advancement was the robustness of the performance-capture system and integrating that system into film production so that it was no longer considered an added-on piece of visual effects," Lemmon says. "Getting the performance of the digital characters on the day [of the shoot] was one of the main aspects of the feature-film production. Beyond that, the biggest thing for us was that there were many digital characters that had to emote and engage with the audience. The subtlety is higher than anything we've done before."

At the end, the audience is left pondering whether there is more ape in the humans, or more human in the apes.

"My favorite scene is with the [human] teenager Cody," Konoval says. "Matt [Reeves] let us improvise and explore, and it was so joyous. After Rise, I had met Towan, the orangutan I based Maurice on, in the Woodland Park Zoo in Seattle. He's a painter. I'm a painter, too. So I went back two months later and began to paint for the orangutans. I'm with them like Cody was with me in this scene. But, I am on the other side, communicating with a human being."

And then, it was up to the animators to preserve the subtleties of that shared moment between ape and human.

"The strength of the process is that all the director needs to worry about from the beginning is what he sees on the day," Barrett says. "Matt [Reeves] didn't want to make decisions twice; what he saw on the day was what he wanted in the end. If Toby plays Koba mad, that's what Koba will look like. For me as an animator, it was a joy to translate these amazing performances from humans to apes, to have such strong and cohesive performances."

So, who is responsible for the characters in the film? The actors? The animators?

"There's no button labeled 'digital makeup' that we can push," Barrett says. "We do have to do certain translations to keep the essence of the performance when we see things in the human that an ape couldn't do. It's a collaboration. There's a lot of artistry involved in bringing characters like Caesar, Koba, and Maurice to the screen."

Artistry and hard, rewarding work.