If surfing were a religion, the first two commandments would be: “Live in the moment,” and “Have as good a time as you can.” So, if you were filming a documentary about surfers, even if those surfers were penguins and not people, you’d want to capture that spontaneity. To do that using animation, in which every frame is handcrafted and precisely controlled, would be...well...radical, dude. Yet, that’s exactly what the directors and producers at Sony Pictures.
Animation (SPA) did, thanks to a crew of approximately 250 talented animators and artists at Sony Pictures Imageworks. And the result is one of the most unusual animated features ever to hit the screen. In Sony Pictures’ Surf’s Up, a documentary film crew follows a young hot-dog surfer named Cody Maverick from Shiverpool, Antarctica, to the Big Z Memorial Surf Off on Pen Gu Island. It’s the first feature animation mockumentary.
“It was fun to fake that whole style to create the illusion of reality, the believability of characters, to shoot with a handheld camera, to even have access to archival footage,” says Ash Brannon, who directed the fi lm with Chris Buck.Producer Chris Jenkins sparked the idea. Jenkins, who came to SPA from Disney Feature Animation where he worked as a visual effects supervisor and effects animator on Who Framed Roger Rabbit, Beauty and the Beast, The Little Mermaid, The Lion King, and other films, thought of turning a story about surfi ng penguins into a Spinal Tap-like documentary.
“This is a new way to tell a story and a way to realize great animation,” Jenkins says. “We had to create ‘pretend spontaneity.’ We had to pretend to interview someone who we happened to catch. But, it was all planned, of course.” The planned spontaneity in the mockumentary forced changes through the entire production, from recording sessions to water simulation.The characters needed to be self-aware. The camera had to be handheld. The water became a character. To help give the dialog track the freshness of a real conversation, the directors recorded actors two or three at a time, something that rarely happens for animated fi lms. Usually directors record each character’s dialog separately.
“Animation is tending to become very shticky,” Jenkins says. “The characters deliver lines, and there’s almost like music hall pitter-patter to the timing. We let the actors read the script, and then we threw the script away and had them put it in their own words.” For example: During Surf’s Up, the documentary crew making the film about Cody often captures the Sports Penguin Entertainment Network
(SPEN) crew, which is covering the surfing championship event. Real-life XGames sports announcer and surfer Sal Masekela is SPEN’s sports announcer in the film, and two of the top surfers in the world—Kelly Slater and Rob Machado—voice the penguins playing his roving reporters on the beach. If surfi ng were a religion, the moment,” and “Have as good a time as you can.” So, if you were filming a documentary about surfers, even if those surfers were penguins and not people, you’d want to capture that spontaneity. To do that using animation, in which every frame is handcrafted and precisely controlled, would be...well...radical, dude. Yet, that’s exactly what the directors and producers at Sony Pictures Animation (SPA) did, thanks to a crew of approximately 250 talented animators and artists at Sony Pictures Imageworks. And the result is one of the most unusual animated features ever to hit the screen. In Sony Pictures’hot-dog surfer named Cody Maverick from Shiverpool, Antarctica, to the Big Z Memorial Surf Off on Pen Gu Island. It’s the fi rst feature animation mockumentary. “It was fun to fake that whole style to create the illusion of reality, the believability of characters, to shoot with a handheld camera, to even have access to archival footage,” says Ash
Brannon, who directed the fi lm with Chris Buck. SPA from Disney Feature Animation where he worked as a visual
effects supervisor and effects animator on
Rabbit and other films, thought of turning a story about surfing penguins into a mation,” Jenkins says. “We had to create ‘pretend spontaneity.’ We had to pretend to interview someone who we happened to catch. But, it was all planned, of course.”
The planned spontaneity in the mockumentary forced changes through the entire production, from recording sessions to water simulation. The characters needed to be self-aware. The camera had to be handheld. The water became a character. To help give the dialog track the freshness of a real conversation, the directors recorded actors two or three at a time, something that rarely
happens for animated films. Usually directors record each character’s dialog separately. “Animation is tending to become very shticky,” Jenkins says. “The characters deliver lines, and there’s almost like music hall pitter-patter to the timing. We let the actors read the script, and then we threw the script away and had them put it in their own words.” For example: During film about Cody often captures the Sports
Penguin Entertainment Network (SPEN) crew, which is covering the surfi ng championship event. Real-life XGames sports announcer and surfer Sal Masekela is SPEN’s sports announcer in the film, and two of the top surfers in the world—Kelly Slater and Rob
Machado—voice the penguins playing his roving reporters on the beach
“We did several recording sessions with them when we could catch them,” says Buck. “We’d show them footage, and they would riff on it. They added a cool sense of believability.” The directors not only used the improvised dialog, they caught flubs, microphone hits, coughs, and so forth, all of which gave the recording an on-the-spot feeling.
You Talking to ME
When the documentary crew interviews characters on camera, the directors wanted to capture that same feeling of spontaneity. “These characters are not just delivering lines to drive a story in a narrative fashion,” says David Shaub, animation director, who led a team of approximately 60 animators at the peak of production. “Rather, they are aware of themselves and how they might appear on camera. What a character is saying and what he’s thinking (or what he really means) might be entirely different things.” The documentary camera adds another layer of complexity to the subtext. Some characters liked being on camera; others didn’t.
But, what appears to be spontaneous on screen is the result of animators having crafted the performance down to the last eye dart. “The difference between a believable performance and one that is overplayed can be as subtle as a bottom eyelid raised a touch too high,” says Shaub. Because Shaub wanted animators to experiment with acting ideas, he asked for fast, responsive rigs. “From my perspective, it was better to animate with a basic, stripped-down puppet that was faster than a slower rig with a lot of cool features,” he says. “It was a complete joy to animate because we were free to explore without the technical burdens typically found in a character rig with lots of bells and whistles.”
Or, as it turns out, some of the logistical strictures typical of most animated features. To create an animated film, typically the layout department first creates camera moves based on the storyboards. Then, for the most part, animators perform the characters from the camera’s view. The pre-determined camera moves, camera angles, pans, zooms, and so forth rarely change because that would affect the animation. But to capture the documentary feel for Surf’s Up, the action needed to drive the camera and the camera needed to be flexible.
At top, from l. to r.: Imageworks modeled three types of waves with blendshapes,
generated particles from the shape, animated the wave using concentric rings,
added the surfboard and surfer, and then rendered a final image (above)
“When we decided we needed a realtime camera, the challenge was to put that camera inside Maya,” says Chris Juen, digital producer. “We did a lot of development early on to make that work, but it really sealed the documentary feeling of the movie. It affected everything.
”One of the most obvious changes was that layout artists controlled the virtual camera with a real, handheld camera. “We bought a $250 camera on eBay and Frankensteined a motion-capture unit on top,” says Rob Bredow. The motioncapture unit had six lenses that looked at flashing infrared markers on the ceiling to track the camera’s position in real time. The camera operator/layout artist looked through the camera’s viewfinder into the virtual, animated world. As he or she moved the camera, the scene in the virtual world changed. It’s a process similar to that used for virtual sets. “If you turn around, you’re turning around in the virtual world,” says Bredow. “And, you can zoom in and out.”
Here’s how the “Handicam” system affected the animation process: First, layout artists blocked out the approximate positions for the characters and an approximate camera move. “Unlike the usual straight-ahead narratives with specific beats and actions, we had a wideopen palette,” says Shaub. “We could cut loose and animate.”
When the directors approved that basic performance, the layout artists filmed the action with the Handicam. The shots then went back to the anima-“I’d say about 85 to 90 percent of the movie was filmed with the handheld camera,” says James Williams, layout supervisor. “Because it was relatively easy to do in terms of setup, and the technology was robust, we ended up using it even for static shots.
When we’re looking at the characters, we get the slight movements, the slight adjustments in frame.” When they shot from shore, the layout artists used a telephoto lens; when a character was surfing and the camera crew was standing in the water, they’d often use a wide-angle lens.
To clear an area in the middle of their workspace at Imageworks for filming, the 12 layout artists moved their desks to the edges of the room. In the center of the ceiling above them was the grid of infrared lights. “We jumped, we fell over, we did all the things a documentary camera crew would do,” says Williams. When Cody is injured, the camera crew runs after the characters transporting him for help. When a penguin throws shells at a cameraman, he ducks.
“By physically placing a camera on your shoulder, you truly felt present in the scene for the first time,” Williams adds. The scene was optimized geometry that played in Autodesk’s Maya, which meant that once the layout artists captured a camera move, it became the virtual camera move for the scene without needing any translation. In addition to the documentary crew, the SPEN television crew also shot footage during Surf’s Up. For this footage, the layout artists created camera rigs in Maya
that could displace themselves with the movement of the water, as if the SPEN crew had attached the cameras to a boat or surfboard. For helicopter shots, the layout artists translated the camera in Maya using motion paths, and then used their Handicam system to simulate the feeling of a camera crew looking out the window.
The documentary crew cut some of this footage into their film, as well as that from little video cameras on the beach and other cameras. Each of these sources had a different look, from the high-defi- nition quality of the SPEN footage to the footage from yesteryear. “As you’re watching the documentary [about Cody] being filmed,” says Williams, “various crews are shooting the competition as well. And when someone is interviewed, you might see archival footage.”
For the archival footage, the effects artists dragged film prints through the parking lot, scanned them, comp’d the scratches into final shots, and added dust, penguin feathers, and other effects to “age” the images. Some shots in the archival film and in the documentary used cameras with limited depth of field and imperfections around the lens. “All the shots are based on real-world surfing documentaries,” says Bredow, “all the way down to the lenses.”
John-Paul Beeghly, director of photography for the surfing documentary Step into Liquid, offered advice on lenses and camera speeds. When he invited Bredow to accompany him on location at Cortes Bank 100 miles off the Southern California coast where surfers ride some of the biggest waves in the world, Bredow jumped onboard (see “Wrangling. He shot reference of surfers unafraid to ride 60-foot waves and, of course, of the waves. “At that point we still didn’t know how we were going to do the waves,” he says
Back Story
In addition to the documentary crew, the SPEN television crew also shot footage during Surf’s Up. For this footage, the layout artists created camera rigs in Maya
that could displace themselves with the movement of the water, as if the SPEN
crew had attached the cameras to a boat or surfboard. For helicopter shots, the layout
artists translated the camera in Maya using motion paths, and then used their Handicam system to simulate the feeling of a camera crew looking out the window.
The documentary crew cut some of this footage into their film, as well as that from little video cameras on the beach and other cameras. Each of these sources had a different look, from the high-defi- nition quality of the SPEN footage to the
footage from yesteryear. “As you’re watching the documentary [about Cody] being filmed,” says Williams, “various crews are shooting the competition as well. And when someone is interviewed, you might see archival footage.” For the archival footage, the effects artists dragged film prints through the parking lot, scanned them, comp’d the scratches into final shots, and added dust, penguin feathers, and other effects to “age” the images. Some shots in the archival film and in the documentary used cameras with limited depth of field and imperfections around the lens.
“All the shots are based on real-world surfing documentaries,” says Bredow, “all the way down to the lenses.”
John-Paul Beeghly, director of photography for the surfing documentary Step into Liquid, offered advice on lenses and camera speeds. When he invited Bredow to accompany him on location at Cortes Bank 100 miles off the Southern California coast where surfers ride some of the biggest waves in the world, Bredow jumped onboard (see “Wrangling. He shot reference of surfers unafraid to ride 60-foot waves and, of
course, of the waves. “At that point we still didn’t know how we were going to
do the waves,” he says.
Liquid Stage
Bredow had helped develop CG water or dramatic shots in the film Cast Away. But water in those shots had appeared in night scenes. Now, he needed to lead a team that would create believable, surfable CG waves for shots in broad daylight. “There was no room to hide,” he says. And there was a second problem:
“We needed the waves to know where to put the characters, the camera needed to know where the white water was to frame the characters, and we needed to know where the character was so we wouldn’t cause it to disappear in white water,” Bredow says. “We needed all three all the time, and it wasn’t clear which needed to come first.”
Rather than trying to create water with a fluid simulator, they decided to model three types of waves using blendshapes that an animator or layout artist working in Maya could use to change the wave’s action. The idea is similar to using blendshapes to manipulate a model into various facial expressions over time
The modelers used reference footage to create three base models: Mavericks, a huge, gnarly wave that looked like those off the coast at Half Moon Bay, California; Pipeline, a perfect tube wave like those in Hawaii; and spilling breakers that don’t have enough power to throw water over the lip (the part of the wave that curls over at the top). Then, they animated these waves to look like the reference footage, and checked the animation against mathematical equations using a virtual speedometer and gravity balls.
At left, the raw edge of the rendered wave becomes realistic when, at right, particle spray is added.
A speedometer placed in the middle of the wave measured how fast the wave moved forward and broke from left to right. Gravity balls placed on the lip and released as the lip reached its apex measured whether the lip fell at the correct speed.
“Animators or layout artists could animate the waves like characters,” says Bredow. “They could also preview the effects that go along with the waves in real time—the white water and the wake trails from the surfboards. The wake enabled the camera operator to frame the shot properly and position the characters.”
At top, appearing on camera makes some characters uncomfortable.
At bottom, to create “archival footage” for the mockumentary,
the effects crew aged animated sequences in various ways.
Making Waves
John Clark, the wave animation lead and a lifetime surfer, helped refine the Mayabased rig. He also created 22 generic animations for each of the waves that animators could use as starting points. Clark began by modeling the Mavericks wave. “Imagine a huge NURBS plane created by a set of parallel curves,” he says.
“Each one of those curves is responsible for a certain number of isoparms. The rig has a series of concentric rings that run along the surface, and as I rotate those rings, they drive the curves through their blendshape stages.” By swapping blendshape curves, he could use the same rig for the three different wave types.
After testing rigs with various numbers of rings, Clark settled on 19 for most of the waves, although a few special waves needed 38 rings. By rotating the rings, he could drive the wave from flat, to peaking, to breaking, to crashing, to flat again. “By animating one ring at a time and offsetting that animation, I created the down, or the linebreaking action of the wave,” he says. At the same time, Clark would animate such attributes as how far the lip would throw out or curl over, the depth of the trough, the thickness and thinness of the lip, the height and width of the wave, and the roundness of the face. The squarer the wave, the more powerful it was. In addition, for helicopter shots, he could control how much volume of water was behind the wave, and for all shots, the amount of digital noise that added texture to the lip and face.
Clark also blocked out surfing shots so that the layout department could film them. “I had detailed storyboards, but I made some adjustments once I got a character on the CG wave to re-create the feel and action they were after,” he says. “When I had the generic waves set up, I’d bring characters in and block out shots really fast. The nice thing about having the waves and the timing right is that it dictates what the character and camera must do.
The surfer has to keep with the lip of the wave. There is a certain amount of time. And that all dictates what the camera will do.”
Penguin on Board
The water dictated what the surfboard would do. “We constrained the surfboard to the forward translation of the wave,” says Shaub. “It was locked to the surface of the wave, and then we had offsets on the board with moveable pivots. So, we had an infinite number of positions to pivot on the board. If Cody stands on the nose hanging six, the board pivots from there. It swings around from the pivot on the tip.”
To make it possible for the short-legged birds to run and, more important, surf, the animated penguins had to have longer legs than real penguins, and they had to have knees. “We couldn’t have anything that would be bashed by the surfing community,” says Shaub, “from the way they carry the board with its tip down a little bit, to how they surf on the water.” Despite having surfing as the theme for the film, however, much of the action takes place on the beach and in the jungles on Pen Gu Island.
“People look at this film and go, ‘Wow, it’s about the water,’” says Lydia Bottegoni, coproducer. “But there are many impressive effects. The sand moves and interacts with the characters every time they take a step on the beach. Branches and bushes and trees move.” CG supervisor
Daniel Kramer wrote what the group calls the “sandbox” tool, which worked in Side Effects Software’s Houdini. “It was a newer incarnation of the technology we used to put footprints in snow in previous films,” says Bredow. “The simple simulator in sandbox allowed the sand to fill in the footsteps.
Ironically, it was also the core engine for the wakes.” Technical director Tom Kluyskens built a system in Maya and Houdini that produced little breakers rolling onto shore. And wave development lead Deborah Carlson sent waves moving along the surface of the ocean into infinity, whether the weather was stormy or calm. The nearly 50 people in the effects department also created lava, feathered penguins, and one chicken; they animated crowds and generated hundreds of particle simulations.
“Everything generates particles,” says Bredow. “We have spray coming off the boards, off the lip of the waves. The average wave shot has
460 million particles. We calculated that with the number of wave shots in the film; we generated 48 trillion particles.” Lighters working in Imageworks’ new proprietary Katana software sent hundreds of layers to Pixar’s RenderMan for final renders. “Lighting was a huge, complicated problem,” Bredow points out. “The directors wanted the water to look believable and photorealistic.
And, they wanted to art-direct it.” For photorealism, they used raytracing and refraction. For art direction, they provided a “zone system,” which controlled the shading on the waves. “The water is astonishing,” says Jenkins. “You can really feel the waves. always knew we had a very special idea.
But, I couldn’t have imagined how phenomenal it would look. There’s a shot of Cody and Z on the beach. We’re looking at their backs. The full moon is creating glitter on the ocean. When they showed that shot to me, I was kind of overcome. I was amazed by the quality, the technical achievement, the mastery of CG, where we’ve come to now. It’s not the Little Mermaid, that’s for sure.”
Lani, the lifeguard, and Cody share a moment. Their surfboards have
moveable pivots so that when they’re surfing, the board follows their movement.
Barbara Robertson is an award-winning
writer and a contributing editor for ComputerGraphics World.