Read in-depth features about how cutting-edge CG was created for various movies making Oscar buzz.
Imageworks inserts CG secret-agent guinea pocs into live-action footage and developes unique techniques for creating stereo 3D imagery.
Agent Darwin, leader of the G-Force team, uses his specially-rigged guinea pig hands to help fight an evil menace.
G-Force, Walt Disney Picture’s live-action summer kid flick, packs talking animals, superheroes, robots, and spies into a rollicking action-comedy in which every one of the 1861 shots is a visual effects shot. It’s producer Jerry Bruckheimer’s first stereo 3D film, and the directorial debut for Hoyt Yeatman, who had received an Oscar (The Abyss) for best visual effects, an Academy Technical Achievement Award, and two Oscar nominations. And, it’s arguably the first VFX-heavy, live-action film shot with standard cameras to be converted into stereo 3D.
Sony Pictures Imageworks created all the CG creatures and masterminded 1287 VFX shots. Asylum VFX added on-screen graphics and graphic playbacks to 133 shots. In-Three converted the rest of the film, the shots without CG effects, into stereo 3D.
The story centers on a group of animals trained by the government as covert agents. Their mission? Stop a gazillionaire tyrant who attempts to take over the world by using household appliances.
The core members of the squad are guinea pigs: indomitable team leader Darwin (Sam Rockwell), extreme weapons expert and toy racecar driver Blaster (Tracy Morgan), and sexy martial-arts practitioner Juarez (Penelope Cruz). Mooch, a fly that wears surveillance equipment, is the reconnaissance expert, and a mole named Speckles (Nicolas Cage) is the team computer geek. In addition, the G-Force team encounters Hurley, a slacker pet-shop guinea pig voiced by director Jon Favreau, and Bucky, a territorial hamster voiced by Steve Buscemi, as well as some CG animals and creatures that don’t talk. The hero critters perform their secret-agent duties in a live-action world populated by actors Zach Galifianakis, Bill Nighy, and others, and fight to save the world from transforming household appliances and an 80-foot robot.
The all-CG G-Force team—Blaster, Darwin, and Juarez—considers its next move. Imageworks animators could quickly switch the guinea pigs from a bipedal stance to a four-on-the floor scamper.
Flat World
Imageworks’ Scott Stokdyk, who won an Oscar for Spider-Man’s visual effects, supervised a crew of 326 artists who worked on the 2D version of the film, which the studio calls the “flat” film. Rob Engle supervised a second crew of 150 visual effects artists at Imageworks who converted each shot created by the “flat-film” team into stereo 3D (see Viewpoint, pg. 6). This is Engle’s sixth stereo 3D film at Imageworks. “We needed to take traditionally shot flat photography, add dimension, and then integrate our CG visual effects characters in a way that would seamlessly blend them together, as if they had been photographed at the same time with a real stereo 3D camera,” Engle says.
First, the flat film.
Within Imageworks’ 60 minutes of animation, 40 minutes entailed swinging the CG creatures into live-action plates; the other 20 minutes were all-CG shots. In addition to the guinea pigs and Speckles the mole, the crew created an army of cockroaches, mice, a snake, robot appliances that come to life and terrorize people, and an 80-foot-tall robot that stars in many of the all-CG shots. Made from pieces of machinery assembled apparently randomly, the asymmetrical machine has a somewhat organic feel.
“It’s unique,” says animation director Tony Saliba of the robot. “It’s not only a character, it’s an environment. There were challenges all over the place. The environment is constantly moving, and the guinea pigs have to make their way into the heart of it.”
The stars of the show, however, are the guinea pigs and the mole. “I think that what sells the movie is that you believe the real guinea pigs could do the stunts,” Stokdyk says. “It’s like Stuart Little combined with Spider-Man. The guinea pigs swing on cables, but they don’t do superhero kinds of things. That gives it more impact, humor, and fun. Even though the film’s target is kids, we wanted it to have a sophisticated action-film look and feel. We wanted to use as much realism as we could to make kids feel like they could have their own Transformers or Terminator.”
The pint-sized G-Force team must stop an 80-foot CG robot made from an assortment of constantly-moving mechanical parts.
But, guinea pigs are never real in the film. Imageworks painted out any of the furry critters used on set, and replaced them with CG counterparts that could stretch beyond the real animals’ physical limitations. “We wanted the guinea pigs to feel real, visceral, and tangible, but to tell our story, we needed to have them do some things the real animals couldn’t do,” Stokdyk says.
Covert Critters
Modelers and animators worked in Autodesk’s Maya to create guinea pigs that differed from the real animals slightly in their looks and sometimes more wildly in their actions. “We angled their eyes a little forward because otherwise we could look at only one eye at a time, and we changed their fur patterns subtly,” explains Saliba, who led a crew that expanded to include 68 animators at the peak of the film’s production.
The fur stylists used Imageworks’ in-house hair system. “One of the biggest issues was in getting the hair and fur to interact with all the gadgets,” says Seth Maury, digital effects supervisor. “For that, JJ Blumenkranz, the CG supervisor, and hair lead Dustin Wicke came up with a magnet system.”
Rendering happened primarily through Pixar’s RenderMan for the characters, and primarily through the studio’s Arnold software for the backgrounds. Compositors, meanwhile, used Imageworks’ own Katana system.
“There was a tendency by the executives and producers to want the guinea pigs more human, with big muscles and big arms and small heads, but Hoyt [Yeatman] dug his heels in,” Saliba says. “He was adamant that they play like photoreal guinea pigs. But, in a single shot, they could go from acting like people to skittering across the floor like little rodents, and when they’re around real animals in the pet shop, they have to act like real guinea pigs. So, we had to walk a fine line.”
That meant the team needed to create photoreal guinea pigs they could stylize without drawing attention to the caricature, and that could change from quadruped to biped within a single shot. Although modelers based the characters on real guinea pigs, because their physique couldn’t cope with the extreme poses called for by the G-Force, modelers changed the bone structure of their legs and hands. For example, their hands could handle equipment and stunts, yet still perform in a guinea pig way.
“Their hands needed the same functionality as their feet because they run on all fours,” Saliba says, “and their feet rig needed to allow them to roll from heel to toe easily. Also, they have amorphous bodies that change quite a bit depending on whether they’re climbing, stretching, bunched in a ball, sitting, or lying down.”
Shaper tools that worked within the Maya rig helped with the shape-shifting bodies. And rigs populated with precise controls allowed the animators to quickly switch from, for example, a forward-kinematics mode for their hands when the guinea pigs stood up, to an inverse-kinematics mode when their hands become feet on the ground.
The animals can talk via a translator gizmo invented by the scientist who organized the G-Force team, which converts their squeaks into human speech—an idea also used by Pixar for the dogs in Up (see “The Shape of Animation,” July 2009). That gave the animators freedom to concentrate on the CG stars’ physical performance without always having to lip-sync dialog, something the Pixar animators also used to their advantage.
Guinea Pig’s Eye View
To help envision the shots, the Imageworks’ animators would often videotape themselves acting to the dialog or performing stunts. “They import the video into a Maya scene and use that as reference,” Saliba says. For additional guinea pig reference, they found 20,000 videos of the popular pets posted to YouTube, and observed some office pets.
A search on YouTube to find reference footage for Speckles, the star-nosed mole, however, produced only 76 results, not all of them useful. “For most people, they’re pests, not pets,” Saliba says. “They have tendrils coming off their nose in a star pattern that are prehensile, like another set of hands. They’re not cute and cuddly.”
On set, because the main characters are less than a foot tall, Yeatman brought the cameras down low. “Hoyt and the crew had special lenses and grips that allowed the first-unit camera operator to hand-hold a camera at guinea pig level,” Stokdyk says. Once the shots moved into post, the crew would often add a slight zoom, change the framing a touch, and tweak the timing. “In some cases, if we pushed the limits,” he adds, “we’d go completely virtual. Almost everything had virtual backup.”
To capture the lighting setups, the crew used Yeatman’s “ChirpiCam.” Maury describes it thus: “It’s a one-and-a-half-foot cube that has five cameras with fish-eye lenses mounted inside that takes exposures every two stops, from eight seconds to an eighth of a second up and down. We put it in the middle of the set each time the lighting changed, and pressed Go. It stitches the images into HDRI maps.” Maury says the camera received its ChirpiCam name because it chirps as it captures the HDRI images.
By the time the production unit finished shooting, ChirpiCam had provided the postproduction crew with 800 HDRI images in 360 degrees from a guinea pig’s point of view. Maury followed the images all the way through the digital intermediate process at Company3. “It was important for our comps to hold up at the DI house, so we bracketed every image up and down six or seven stops to make sure the CG highlights would look good in all the different formats,” Maury explains. “We pushed the images around a lot, especially the stereo images because they are not as bright.”
By choosing to shoot the film with standard cameras rather than bulky stereo 3D cameras, Yeatman and the cinematographers were free to choose any lens and lighting, and to shoot at guinea pig’s eye view. But, that choice created interesting challenges for the Imageworks stereo 3D crew.
“In the 2D version of G-Force, about two-thirds of the film includes visual effects, and the rest of the film is live-action plates,” Engle says. “In G-Force 3D, every shot in the movie is a visual effect. So, the first thing we did was divide up the workload.”
Director Hoyt Yeatman put the cameras low to help the audience relate more directly with the star-nosed mole (Bottom) and the guinea pigs, (top). Similarly, the stereoscopic 3D team tuned its cameras to the stereo depth that a guinea pig might see.
Stereo Solutions
Imageworks gave all the shots without CG effects to In-Three to convert those plates to stereo 3D. “They cut out elements by pulling keys and by using rotoscoping to isolate and offset them, to produce the perspective you would see with the other eye,” Engle says.
For the plates that would include the CG characters, Imageworks needed to take a different approach, but even so, relied on a typical pipeline.
“Our crew is organized like most visual effects crews,” Engle says of the stereo 3D artists. “We’re basically using visual effects processes for everything: Layout takes the matchmoves, adds extra geometry to mimic the real world and make sure it’s logical, the camera crew creates the stereo cameras and puts renders in front of me, the lighters reproduce the other eye for all the CG objects and creates the stereo render of the plate, and the painters clean up the problems. The thing that’s different is that we have to fill holes if anything has been cut out of the plate.”
As they would with any visual effects pipeline, the stereo 3D artists started with the matchmove of the plate that the “flat-film” team had generated, which gave them a virtual world—rough geometry representing what was filmed on location as seen from the point of view of the camera used on the set—with the CG animals positioned in that virtual world.
“Then, we added more detail to the matchmove model that the 2D team didn’t have to think about,” Engle notes. “Sometimes the 2D team only matchmoves a 2D surface, like a table, so if a guinea pig is on the table, we’d add other objects. And, we had to fill any holes.”
Once they had filled the holes, which often happened on a per-shot basis, they created a stereo pair of virtual cameras in the computer and rendered images for each eye. “For CG objects, it was easy,” Engle says. “We’d just render the other eye to create the stereo pair. To make the plate itself 3D, though, the easiest way to describe it is that we projected the plate photography onto the geometry in the virtual world and took a picture from the other point of view.”
When the guinea pigs meet Hurley (above) in a pet shop, the animators performed them as if they were real animals, not secret agents. A snake attack, one of the most interesting stereo 3D effects in the film, takes place in the pet shop.
New Stereo Techniques
When the camera was at guinea pig level, the artists tuned the stereo depth to what a guinea pig might see. “Guinea pigs’ eyes are on the order of a couple centimeters apart,” Engle says. “Humans’ eyes are six to six and a half. So, we tended to keep the cameras close together, especially when [the guinea pigs are] in a terrarium, to make the environments feel real.”
The artists also developed new ways of working with the stereo image to heighten the experience. “The 2D release is a widescreen, 2.40 format,” Engle says. “But the studio also wanted a 1.77 format for high-definition home video. So we decided to take advantage of the extra information for the 3D version. The theaters are projecting the film as if it’s 2.35, but we’re delivering 1.85 with letterboxing on the top and bottom, so it looks square-ish. But, every once in a while, we allow objects to extend into the letterboxed region.” To the audience watching with 3D glasses, it looks like the objects are coming out of the top and bottom of the screen.
“One of the shots in the film is of a snake trying to attack a guinea pig,” Engle says. “The point of view is from the guinea pig behind a sheet of glass. The snake rears up, coils, strikes, and hits the glass with its jaws wide open. It’s a great 3D moment. We let the jaw go over the top and bottom of the 2.35 picture so it literally feels like the snake is in the audience’s space. Breaking the mask this way allowed us to have an overt way of saying, ‘This is in your space,’ without bringing objects out into the space and having the audience’s eyes cross.”
In 2008, the film Journey to the Center of the Earth broke new ground by becoming the first to incorporate CG visual effects in a live-action film shot with stereo 3D cameras. G-Force breaks a second barrier. “Rather than shooting the film in 3D, we added the stereo in postproduction,” says Engle. “This is the first time that a live-action film integrated with CG visual effects has been converted to 3D.”
By using visual effects techniques to create the stereo version of the film, Imageworks discovered new ways in which directors and cinematographers might want to create deeper worlds: Rather than using stereo 3D cameras to film the action, they can continue working with the less-limiting and more-familiar systems. Whether or not they have superagent CG guinea pigs as stars.
“I think the way we ended up doing the conversion gives filmmakers a lot of flexibility,” Stokdyk says. “If they have a new movie coming to the table and they’re thinking about stereo and live action, I think they have to consider the extremely efficient way we did it.”
Barbara Robertson is an award-winning writer and a contributing editor for Computer Graphics World. She can be reached at
BarbaraRR@comcast.net.