In the US, we fondly recall our patriotic past through iconic national stories: George Washington at Valley Forge, Paul Revere riding through the night, Betsy Ross sewing the flag. China’s story is told in Red Cliff, a movie directed by John Woo that opened recently. The epic story—first told in the classic 700-year-old Chinese novel Romance of the Three Kingdoms—is about a power-hungry general seeking permission from the Han Dynasty emperor to crush two troublesome warlords. Those two warlords band together to fight the general and his overwhelmingly superior forces, and, with their cunning and bravery, change China’s history forever.
“The story is so engrained in people’s consciousness that there’s really no equivalent in the US,” says Craig Hayes, who was both overall visual effects supervisor of the movie and VFX supe at the now-defunct The Orphanage, which led the digital work on the film. Twelve other VFX vendors contributed, as well.
The film came to The Orphanage through producer Terence Chang, who produced Red Cliff with Woo. “The guys at The Orphanage were friends with Terence from way back,” says Hayes, noting that the studio was on the radar screen of the Asian film community due to its work on the Korean feature The Host. The Orphanage team first met with Woo in Los Angeles to talk about ideas and examine some initial previsualizations, steps that were suggested by Hayes.
Numerous visual effects facilities from around the world contributed work to the Chinese
epic Red Cliff, including Frantic Films VFX, now Prime Focus, which produced the battle
shots above and on the next page.
“I wanted to gauge the production’s ability to utilize previs as soon as possible, so we would know how to schedule it in if necessary,” says Hayes.
Once the job was awarded, The Orphanage’s animation supervisor, Webster Colcord, worked with a couple of animators at The Orphanage on several of the big sequences. The previs work was done using Autodesk’s Maya.
“As the show progressed, I ended up doing previs on set, on my 17-inch MacBook Pro, which was helpful for discussing logistic requirements with the crew,” Hayes points out. “They say a picture is worth a thousand words. When you don’t speak the language, it’s worth a lot more.”
In November 2007, Hayes flew to
China to meet with Woo in his production office—and didn’t leave there for three days. “At this initial point, we approached it like a normal production,” says Hayes. “I pretended to be an AD and asked who was going to do this or that. I would make suggestions for what the art department could build and walked through the storyboard talking about how we’d achieve everything. I was very excited not only to work with John Woo, but also to be in China.”
It was then that Hayes, who had been a free agent, joined the staff of The Orphanage. “It was a unique opportunity to both work in a facility and on the production at the same time,” he says. “The up side is you have more flexibility to horse-trade with your own guys on the production.”
On the next trip to China, Hayes brought along a couple of animators from The Orphanage and previsualizations for one set of storyboards. “The point was to get them excited and to make sure they understood what previs is,” he says about the group in China. “It can be an effective and useful tool, but only if the filmmakers understand what they’re looking at, that it’s not just cartoons.” Previs was crucial for this feature since it centered around three immense battle scenes that required detailed choreography of a huge number of practical and digital effects.
Production Challenges
When production started, Hayes began back-and-forth trips to China, as the shoot spanned all four seasons, much of it near the Yangtze River, which plays an important role in the story. With a crew of 700 people and up to 1000 extras, anything could happen—and it did. “We had a plan, but everything was run-and-gun,” says Hayes. “It was chaos, but with a vision behind it. John was never out of control with the movie he was making.”
Frantic Films created a fleet of 2500 computer-generated boats for a big naval battle that takes
place in Red Cliff. The studio also generated the water and reflections.
Still, the production had its share of disasters. “We had two of our sets built on a flood plain, and you’d come back the next day and the set would be washed away,” Hayes recalls. “The extras were all from the People’s Liberation Army. You’d train 800 people what to do, and the next day you got 800 new soldiers, and all that training was wasted. We had to retrain new guys every day.”
For the last third of the production, five units were shooting day and night. That included a list of heavy-hitter directors and cinematographers: Corey Yuen, action director; Patrick Leung, naval unit director; and Lu Yue, first director of photography, who was later replaced by Zhang Li due to health reasons. According to Hayes, The Orphanage’s producer Ken Kokka and visual effects digital production manager Tiffany Wu (who is fluent in Chinese) also proved extremely valuable.
Woo had decided that Red Cliff—which was budgeted in the $80 million range—would consist of two movies for the Asian markets; another version for the US and Western audiences would combine those two versions into a single release. In the final picture, Parts 1 and 2 combined had 860 visual effects shots. The Orphanage handled approximately 300 of those. “We stopped counting when we got to over 2000 plates,” says Hayes.
A large portion of the budget went to the creation of many practical effects, including an entire fleet of boats. “By the time they had built so many practical boats, we were down to a small figure for the VFX budget,” says Hayes.
Boats, Battles, and More
The Orphanage was the hub for all the VFX work, which included contributions from across the globe: Anibrain (Mumbai, India), CafeFX (Santa Maria, California), Crystal CG (Beijing, China), Digital Dimension (Montreal), Frantic Films VFX (Los Angeles), Hatch (Los Angeles), Kerner Optical (San Rafael, California), Make FX (Los Angeles), Pixel Magic (Los Angeles), Red FX (Montreal), Tippett Studio (Berkeley, California), and Xing-Xing (Beijing).
On the first movie, The Orphanage primarily did comp work. But the VFX house soon shifted into high gear for the second movie, when it was tasked with creating a digital straw boat that would soon be studded with hundreds of arrows—a sequence that comprises approximately 30 shots. The CG model was necessary since it was impossible to get practical straw boats to move in formation. Initially, the group photographed a practical boat model from every possible angle. Hayes used Maya to previs a single frame, and asked Woo what he thought about the framing. When he knew how many boats were needed and what kind of rotation, the team used that one CG model to represent a few dozen computer-generated boats, and then redressed them with new flags for variation.
“We also did a lot of particle arrow shots, fleets of ships, and set extension shots,” says Colcord. Creating the previs for all these shots was perhaps the biggest challenge. “We used Google Earth to find maps of locations and based rough environmental geometry on those. We were also trying to make it realistic in terms of how we would shoot [the sequence] within the limitations as we knew them, realizing we would have no motion control.”
In addition to Maya, the group used Mental Images’ Mental Ray for rendering, The Foundry’s Nuke for compositing, and Vicon’s Boujou and Andersson Technologies’ SynthEyes for tracking and matchmoving in the scene.
To figure out how to set up a formation of soldiers in a tortoise-shell pattern, previs animator Bruce Dahl determined the necessary number of soldiers in a given segment of the formation and how they would move to trap the opposing army in the corridors of the pattern. “I had figured they would make up a whole organizational chart based on our little previs, but when they shot the live action with 300 Chinese army extras and horses out in the blazing sun, they had a print-out of Bruce’s render—and that was it,” he says. “And it worked!”
Other CG work posed specific challenges, as well: “dealing with arrows at a macro distance from the lens as the camera flies 800 yards, cloth simulation, digital soldiers?…?all of that,” Colcord says. “There was a lot of re-projection in the straw boat sequence because we didn’t have the budget to do the large-scale ships in CG at a medium distance. So, for arrows flying ship to ship, the middle part of the shot would be full CG, bookended by re-projected live action.”
CafeFX created this film sequence that is set on the Yangtze River. It, like many of the studio’s
shots in the film, required a great deal of roto and comp work. Digital water also proved to
be a challenge.
Collaborative Effort
A range of companies in North America and Asia completed the other 500-plus visual effects shots, including a first: two companies in Beijing. According to Hayes, Xing-Xing did nice roto and wire removal. “I felt that on Movie 1, it was culturally important to have the Chinese working on it,” he says. “Crystal CG, the second Chinese company that worked on the film, was an architectural company with no feature-film experience. They did some of the spear shots, and they brought in some people from the States to work on this.”
Anibrain also did wire removal and spear shots on the first movie. “On Movie 2, we gave them some more complicated shots,” says Hayes. “It was good to have that progression.” Pixel Magic in Los Angeles worked on some of the arrow shots and a crowd-replication scene that involved a whole cavalry of horses moving downhill, as well as some last-minute wide army shots.”
CafeFX handled much of the Yangtze River boat work. “The art department had built all these full-scale props that were almost entirely historically authentic,” says Hayes. “They floated, but you couldn’t steer them. If a light breeze came up, you didn’t want them on the water. And all the boat captains were local farmers. We were trying all day to make these boats go in formation and fast. They were difficult to photograph—the rule of thumb in visual effects is that shooting on the water is seven times slower than on land.”
Hayes says that experience set a precedent for this project. “We made sure the production knew the difficulties so when we came back in 2008 to shoot the final naval battle, we put the boats on land,” he says. In one spectacular sequence, the camera pulls back to show an immense fleet of boats on the Yangtze River, all done by CafeFX. “The shots they did were beautiful,” says Hayes.
Digital water was a challenge for several of the VFX companies involved: The Orphanage, supervised by Rich McBride and Dav Rauch; CafeFX, supervised by Kevin Rafferty; and RedFX, supervised by Derek Wentworth. All of them worked on creating digital water. The Orphanage worked on both nighttime and daytime sequences.
“CafeFX had some unique challenges in that almost all its shots were actually filmed in the water—tied to a dock with movie crew all over place, necessitating some tricky comp and roto,” explains Hayes. “Frantic and RedFX had a different but equally difficult job of working with ships that were shot on dry land, with limited greenscreen coverage.
Hatch, meanwhile, did a lot of the matte paintings, including an opening scene that depicts the exterior of the Han palace. “We had maybe two dozen matte paintings, and we wanted the ones we did have to be very artistic,” Hayes says. “The scale of everything in China was so big. Hatch held down the fort for the vision of these exteriors in these key wide-establishing shots.”
Production for the second movie began while the visual effects team was still finishing the first one. Hayes bounced back and forth between The Orphanage, overseeing work on the shots from the first film, and China, so he could be on hand for the shots that would impact the visual effects. In 2008, he spent eight consecutive months in China. He also added a number of new VFX companies to the production, including Red Effects, which handled a number of shots for the film’s climactic naval battle.
Frantic Films (which has since been renamed Prime Focus Visual Effects) also worked on that same naval battle, handling 13 extremely large-scale shots. “With shots that called for a fleet of 2500 of the same 26-meter (85.3-foot) boats, giving each a unique handcrafted feel was both a creative and technical challenge,” says
Jason Crosby, VFX supervisor for Frantic Films VFX, who worked with visual effects producer Bridgitte Krupke. In addition to those 2500 ships, the sequence was populated by 70,000 sailors and soldiers, all of which needed to be seen performing on the ships. Lastly, because it was a battle scene, fire, smoke, explosions, and boat damage were required.
“We also had aerial shots—big, high, wide shots looking over the navy,” notes Frantic’s CEO/senior visual effects supervisor Mike Fink. “We built a major pipeline for all the soldiers on deck and a massive render pipeline. We created a procedural way to make the boats float on the water. So that every sailor and soldier wasn’t at the same angle to the lens in the camera, we couldn’t render it like one giant, massive group. We found a way to instance them to every boat out of Massive, and then rendered the boat and the people. So, it ended up being really fast.” The team used three different rendering systems to create the effect: Chaos Group’s V-Ray software rendering system, Cebas’s FinalRender, and Mental Ray.
Frantic Films also spent five weeks building a Massive pipeline to animate the 70,000 soldiers in the 2500 boats. Cebas’s Thinking Particles was used to modify each crew’s animation and then propagate the crews throughout the fleet.
A team of fifteen 3D artists and 11 compositors worked on the sequences, first rotoscoping the practical boats. The Frantic Films team in Winnipeg, Manitoba, generated the water and water wake around them. “If they were on fire, we had to build the reflections in the water,” he says. “The big challenge was that these were huge fires that covered 10,000 boats, so you had to see the smoke propagate from the source of the fire and drift across a mile or more of boats. That’s a lot of particles.”
Previs helped determine how to set up this complex battle formation. CafeFX enhanced the
shot digitally.
For animating the boats, the artists imported the CG models from Maya into Autodesk’s 3ds Max, where they did the bulk of the 3D work. The water was a combination of the studio’s proprietary Flood: Surf tool for the fluid surfaces, rendered in FinalRender, and Flood for the fluid interaction of the wakes and the shots of performers diving overboard. Another proprietary 3ds Max plug-in, Fume, was used for the giant plumes of smoke as well as for some of the flame interactions. Compositing was done with Eyeon’s Fusion.
Hayes notes that Kerner Optical did some “critical work” related to burning ship elements. “They built a number of ‘bucks,’ miniature-ship steel skeletons that were plumbed with gas lines and rigged for explosions and such,” he says. “We photographed these bucks from every necessary angle in many passes, and then had an extensive library of fire that could be comp’d onto either photographic-plate ships or CG ships.”
Indeed, the film’s visual effects were expansive and challenging. Yet, other difficulties plagued the set and crew, as well. Some of the mishaps that marked the production seem nearly unbelievable in retrospect. One of the practical villages built for the shoot was set on fire?…?and then became a real, out-of-control conflagration. There was a typhoon. A crane toppled. “The effects really came together because of Craig’s [Hayes] ingenuity and resourcefulness,” says Colcord. “And the vendors did an amazing job stitching together all sorts of different elements into some unique and cool shots.”
“It was a daunting task,” admits Hayes. “It was the best, and it was the worse. And I wouldn’t have missed it for anything. At the end of the day, not only am I proud of the accomplishment, but John [Woo] and Terence [Chang] are such nice people and such gracious hosts, that everyone put their all into it for them.”