By Barbara Robertson
It's ironic that the most realistic human created with computer graphics to date was designed to be an in-your-face visual effect for an invisible man. This digital human appears in Columbia Pictures' Hollow Man, which stars Kevin Bacon as Sebastian Caine, a charismatic, egocentric scientist, and Elisabeth Shue as his colleague, Linda Foster. Actually, the digital human is Kevin Bacon-or as close to Kevin Bacon as a CG model of muscle and bone can get.
Directed by Paul Verhoeven who directed Starship Troopers, RoboCop, and Total Recall, the movie, scheduled for release August 4, tells the tale of a man who becomes invisible and begins shedding all social ethics. Invisible man movies are hardly new, and in many, the hero uses his invisibility to do good deeds. Not in this one. The tagline for this R-rated horror suspense thriller reads, "There's more to fear than you can see. Just because you think you're alone doesn't mean you are." As for the visual effects? Senior special effects supervisor Scott E. Anderson says, "Paul filtered out the comic effects and began to focus on the threatening."
 |
| Above, a custom 3D tracking system built in Maya helped the effects crew place a digital human under Sebastian's (Kevin Bacon) image, and then reveal it slowly. Below, Sebastian's body has been replaced with a digital model that is mimicking the a |
Anderson began working on the project at Sony Pictures Imageworks (Culver City, CA) two years ago-July 7, 1998, "This project needed a lot of planning from the beginning," he says. There was the problem of creating an invisible character that could instill fear. And there was the more general question of how to shoot an invisible character at all. But most intriguing was the way in which characters become invisible and visible.
In some invisible man movies, the characters quietly fade into and out of nothingness. In Hollow Man, we watch bodies erode and reappear bit by bit-and it's not a gentle process. "The reveal has always been in Andy's [Marlowe] script," says Anderson. "I knew we were going to have these various stages of Sebastian throughout the film. That is what drew me to the project. To me, it was the right application for creating a digital human, and it seemed like an amazing challenge." He was right.
"This movie has over 550 of some of the hardest shots that I've ever done," Anderson says. Of those, 414 were created at Imageworks and 155 by a crew at Tippett Studios (Berkeley, CA) who were asked to create an invisible Sebastian swimming, moving through smoke, and other similar shots. "These shots were very beautiful and Phil's [Tippett] team is very artistic, so they were the logical shots to give them," he says.
 |
| At left, layers of muscles (in red) were built for Bacon's digital model and each muscle was anchored to a bone (in white) to move properly. At right, the facial muscles were contoured to resemble Bacon's. |
All the shots with the digital human stayed at Imageworks, where more than 300 people worked on Hollow Man digital effects, some for nearly two years. The tools they used included Alias|Wavefront's (Toronto) Maya, Studio Paint, and Composer; Side Effects (Toronto) Houdini; Pixar's (Pt. Richmond, CA) Photorealistic RenderMan; and Paraform's (Santa Clara, CA) Paraform software running primarily on SGI (Mountain View, CA) workstations with some animation on Intergraph (Huntsville, AL) workstations. Around two-thirds of the Imageworks shots involved the invisible Sebastian; one third featured a digital human or gorilla. "There were 100 some odd shots of a human that's not real running around the set," Anderson says.
This is not just a CG shell of a human with a skeleton inside that he's talking about. By the time the Imageworks crew finished, they had created two of the most complex geometric models ever built for a film: the digital human built to replicate Kevin Bacon as accurately as possible, and a digital gorilla. Both models are anatomically correct from capillary to cheekbone, stomach to scapula, muscle to membrane.
Here's why: In an early scene in the movie, Sebastian, Linda, and several other scientists working in a secret underground lab are about to inject Isabelle, a gorilla that they've already made invisible, with an antidote serum. We see the indentations made by her body on the bed, the restraints holding her arms and legs, but nothing more. They give her the shot, and as the serum takes effect, we can see her blood start to flow through veins that seem to float in the air. It's quite beautiful, and after that, the shape of a gorilla is revealed as her circulatory system fills in. And then, bones, organs, and strips of muscles begin to appear. Her heartbeat is erratic and the scientists take emergency measures to keep her alive. More organs appear, then muscles, and finally skin and hair, and we see an entire, living, CG gorilla strapped to the laboratory bed.
 |
| Custom volume rendering tools were used to create the heart so that the interior could visibly erode (or grow back). |
Before long, Sebastian himself tries the serum and takes the gorilla's place on the bed. The serum produces a horribly painful effect, and we watch him writhe in agony, head thrown back, muscles tensing as his body slowly erodes. His skin becomes translucent, then transparent, then disappears, revealing a layer of muscles with veins and arteries laced on top. Muscle fibers disintegrate and reveal the organs and bones beneath. The circulatory system becomes more evident. Finally, there's nothing left but his bones. His body quiets. And then the bones erode.
This is the last scene in which Kevin Bacon appears in the film in his corporeal form; however, he performs Sebastian's role throughout. "I worked hard convincing people that shooting scenes with Kevin was the way to go," says Anderson. "I think it really paid off. The human animation we're doing, which is entirely based on Kevin's performance, is amazingly detailed and realistic." Long before the detailed performance could begin, though, the effects crew had to create a realistic model.
Working from photographs, cyberscans, and detailed measurements of Bacon, and with the help of anatomy professors, a six-person modeling team led by Wayne Kennedy began using Maya to create the NURBS models in late 1998.
They started with the skeleton using information from skin measurements and from anatomy experts to create Sebastian's spine. "Walt Hyneman, who is no longer here, did a lot of the work to determine where the bones would sit in Kevin's skin, which was important because if the bones were off, the muscles would be off," Kennedy explains. "If we didn't leave enough room, five layers later we'd have a muscle poking through the skin." Kennedy estimates that in the torso alone, he sculpted more than 200 muscles and attached them to bones. "It's very complex because they had to bend and move like real muscles," he says. "If we were off it would show in the animation when the geometry is deformed, so we had to be quite spot-on about where we attached them." Moreover, because of the shading techniques the crew would use later, the muscles could not be flat sheets of geometry; each had to be an enclosed volume-even tiny muscles between the ribs. Is he anatomically correct and complete? "Yes," Kennedy says. "Absolutely."
The digital Sebastian has a complete inner mouth with a realistic tongue and a throat. "We could make him swallow if we wanted to," Kennedy says. All of his organs are there-the heart with all its chambers, the lungs, brain, stomach, intestines, and so forth. "The intestines were incredibly difficult because they are tightly packed and we didn't want interpenetration between the surfaces," he says. Interpenetration was always a worry. For example, the circulatory system, which is so intricate and finely detailed that when seen alone it can suggest the shape of the entire body, is actually woven between and around all the muscles, organs, and bones. Sebastian's model, when seen onscreen, is so dense with geometry it looks almost solid.
 |
| Isabelle's transformation into a visible gorilla required multiple procedural processes working together, and 15 to 16 hours of rendering time per frame. |
The gorilla shares some of the same geometry. The organs were first built for the gorilla, then resized for Sebastian and similarly, Sebastian's arms and legs were built first and resized for the gorilla. Apart from gender (the gorilla is female), the major difference in the two is the head. Aside from the obvious variation in size and shape, Sebastian's head and face needed to look like Kevin Bacon, and this is the only part of the model that isn't quite anatomically accurate. Since there would often be no skin over the muscles and no fatty layer between, the modelers altered the shape of the muscles to help make the "muscle face" look more like Bacon.
Now that they had an anatomically correct human, the question became, how would they animate it? Jeremy Cantor, character setup supervisor, spent his first year on the project designing skeletal controls for the overall performance, and Ken Hahn, animation technology supervisor, worked on tools to animate the muscles. Typically in computer graphics, the skeleton is simplified. Not so in Hollow Man. "We made the skeleton complex enough to do the squash and stretch and give and take of a human body," says Anderson. This was necessary because the animators had to match Bacon's performance in every frame that called for the digital Sebastian.
"The performance had to be photorealistic, not just believable," says Eric Armstrong, animation supervisor. "We had to move things in relation to the camera in incremental amounts." Even though they did this using keyframe animation in Maya, because they animated the digital human model to meticulously replace Bacon's body, they called the process "rotomation." On the laboratory bed when Sebastian is first transforming, Bacon is in the flesh but when Sebastian becomes invisible, Bacon wears a green suit and green makeup with blue dots on his joints to make it easier for the animators to align the digital model in the live action plates (the scanned film), and for painters to remove him.
To enable such precise animation, Cantor added pivot points and extra joints to the model so that the wrists, ankles, knees, neck, and spine could twist, not simply bend in a single axis. Then, he created a simple user interface for the animators and wrote expressions to link the interface to the intricate skeletal structure. "The less complex the animator's interface, the more complex the system that drives the joints has to be," he says.
 |
| Sebastian's body erodes as it becomes invisible to eventually reveal, as shown here, deep muscles, organs, and a skeleton wrapped in fascia, muscles, and veins, all moving realistically. |
With the skeleton movement worked out, Hahn's team began creating custom deformers in Maya to move the muscles. "We couldn't have simple little blobs that would fake where a muscle goes," says Anderson. "We had to make these muscles work because we were going to see them." Primarily, joint rotations would drive the muscles although animators could fine-tune the muscle shapes. In addition, the technical directors (TDs) put "locators," or points, on the skeleton that essentially report on what's happening. "We evaluate what those points are doing to try to figure out their interaction," says Hahn. That data can be fed back into the deformer, and that idea helped the team create a facial animation system for animating a skinless face.
For this, the TDs gave animators an interface that looked like a traditional shape-blending, facial-animation system. However, the shapes weren't actually used in the final animation. Instead, points on the shapes provided motion data for the muscle deformer to use. When the muscles were moving correctly, the shapes could be removed. Organs were treated as special muscles with their own deformation rules; internal parts such as the fascia membrane and connective tissue were animated by the deformation of the muscles.
With the exception of a scene near the end of the film in which Sebastian is stuck in a half-visible state, the only time you see the digital models, gorilla or human, are when they're transforming to and from visibility and invisibility. This means that in addition to the character animation of Isabelle and Sebastian and their muscles and organs, another type of animation had to be choreographed and rendered-the appearance and disappearance of body parts during the transformation into or out of invisibility. "Most of the high tech development went into these transformation shots," says Tasso Lappas, technology supervisor and one of several R&D people who were assigned specifically to this project. "Nobody had done anything like this before."
To help time the transformations, the R&D team used curves in Maya. "Curves are a common theme everywhere," Lappas says. "We have timing curves in muscles, veins, arteries, and bones. And based on parameters along the curves we can have these things go on and off." Other curves generate structure: veins, arteries, and the fibers in muscles are generated procedurally from curves via custom RenderMan DSOs (dynamically shared objects for linking libraries of C or C++ code to RenderMan). Particles on the curves in these structures and others have attributes that help guide the shaders.
 |
| Above, dynamic bed wrinkling was created with Houdini. At top right, simple animation controls move complex joints. At right, Bacon's green suit and makeup help painters remove him later. |
"Combining all the pieces of geometry in different ways, and paying attention to all the internal workings and every part has been one of our biggest challenges," says Scott Stokdyk, digital effects supervisor. "We have a whole system for dealing with muscles, veins, and bones and how they transform on and off."
Each muscle has its own shape and way of reacting to the transformation, according to Stokdyk, and there are hundreds of muscles. When the muscles transform, the fibers inside begin pulling back, becoming apparently thinner as they do, because a system of blobbies (implicit surfaces) implemented in Maya has been choreographed to begin growing and eroding the fibers. A similar system erodes the bones and eats away the skin. When the procedurally generated veins transform, a system set up within Houdini continually calculates the distance from the initial animation point to the current length to determine how far to flow blood inside. In addition, a displacement shader ripples the surface of the vein. The detail is amazing.
 |
| First, Bacon was filmed wearing a latex mask. Then, painters removed his eyes and mouth. Finally, the mask's interior was rebuilt and lit with CG tools. |
"We have basically two [rendering] problems to attack," says Stokdyk. "We have the surface look and the surface shaders, which are enormously complex in and of themselves, and then we've got a transforming look. Once we reveal beneath the surface, we use volumetric rendering to reveal the inside."
The transforming heart, for example, is created largely with volume rendering-that is, with an implicit surface reconstruction inside a RenderMan shader that uses a ray-tracing DSO written by Laurent Charbonnel. Brian Steiner wrote the transforming heart shader. "Basically, we're shooting a ray from the camera, and when it hits a surface, it starts sampling small areas," explains Thomas Hollier, CG supervisor. "It keeps moving forward and comparing what it samples to a density function that tells it what areas to shade. And that builds the surfaces inside the volume." The surfaces that the ray hits are animated blobbies. These blobbies are choreographed and their movement determines which surfaces in the heart's interior will grow or erode, and when. A similar shader, written by Laurence Treweek, erodes or grows the interior of bones. Included in these shaders are texture maps that control the look-the wetness, transparency, color, and so forth of surfaces being generated.
"There are so many things that have to come together to get these things to work, and any little mistake ripples through and can't be fixed later," says Stokdyk. "We have to go back to the initial camera move or the rotomation."
In addition, many critical pieces outside the character pipeline were important to the success of the effect. A new, custom 3D camera tracking system implemented in Maya made it possible to do the rotomation. Because it's very difficult to line up the characters exactly and because Bacon's body size changed between the time he was measured for the model and when the plates were shot, painters had to laboriously remove green edges. "The painting for some shots took as long as 20 days," says Jim Birney, CG supervisor who worked on a scene in which Sebastian struggles with Linda. "The green guy's hand is huge and we've got this little, boney digital guy. The joints fit perfectly, but when we removed the green guy, we had big holes in Elisabeth."
 |
| Many elements, including miniature sets, green-screen shots, and live-action stunts were combined in this elevator sequence. Then Sebastian, now scorched and scarred, was fit into place. |
The effects crew also had to work on shots in which Sebastian is invisible. Sometimes the invisible Sebastian wears a latex mask, sometimes a blanket wrapped around him, sometimes nothing, and because green-suited Bacon is in all the scenes that require interaction with other people, these scenes required reconstruction of elements behind him, usually with computer graphics. CG supervisor Heather Baker worked on the mask scenes and one in which Sebastian is photographed with a thermal camera. The latter proved particularly difficult because the thermal camera mapped values in video resolution.
But from a computer graphics standpoint, nothing compares to the work on the digital gorilla and digital human, which has brought us one step closer to creating photorealistic digital people. How close? Armstrong, who worked on the animation, says, "We created a working human being, but as complex as we were, we didn't even scratch the surface." He believes we are so familiar with human complexity we spot every small inaccuracy. "I'm sure eventually someone will create a believable human, " he says, "but not now...maybe 20 years from now."
For now, "muscle man" will have to do.
Barbara Robertson is Senior Editor, West Coast, for Computer Graphics World.115 shots have a digital human (Sebastian) or gorilla (Isabelle)
Each digital model includes:
65,227 procedural arteries, veins, and nerves (42,303 for Isabelle)
50,000 curves for procedurally generated capillaries
10,000 to 50,000 muscle fibers (40,000 typically in Sebastian shots)
10,000 painted texture maps
3150 modeled and physiqued pieces (not including facial animation targets, shot-specific models)
300 to 400 bones
20 organs