Sherlock Holmes: A Game of Shadows, the sequel to 2009’s box-office smash Sherlock Holmes, sees Robert Downey Jr. and Jude Law reprising their roles as Sherlock Holmes and Dr. Watson, with Guy Ritchie once more at the helm. Produced, like the first film, by Joel Silver, Lionel Wigram, Susan Downey, and Dan Lin, Shadows also sees the return of cinematographer Philippe Rousselot and visual effects supervisor Chas Jarrett. Framestore, too, rejoined the crew for a second foray into Victorian mystery and mayhem.
Shadows sees our latest incarnation of Holmes matching lethal wits with his most famous literary nemesis, Professor James Moriarty (Jared Harris). When an eminent surgeon is found dead in an auction room, Holmes deduces that he has been the victim of a murder that is only one piece of a larger and much more ominous puzzle. Holmes then encounters Sim (Noomi Rapace), a Gypsy fortune teller, whose life he saves and who reluctantly agrees to help him. The investigation becomes ever more dangerous as it leads Holmes, Watson, and Sim across the continent, from England to France to Germany and, finally, to Switzerland. But Moriarty always seems to be one step ahead as he spins a web of death and destruction—all part of a greater plan that, if he succeeds, will change the course of history.
Framestore created nearly 550 shots for Sherlock Holmes: A Game of Shadows, including the film’s climactic encounter at the Reichenbach Falls and a thrilling battle on a speeding train, as well as numerous elaborate environmental shots and set extension work, a scene taking place inside a gun, and some ingenious, near-subliminal puffs of breath.
Shooting took place from September 2010 to January 2011, with the majority of first unit material captured on location around the UK and at Elstree and Leavesden Studios. Supervising the shoots for Framestore were visual effects supervisor Sirio Quintavalle, compositing supervisor Kyle McCulloch, and CG supervisor Ben White.
RAIN OF TERABYTES
(Spoiler alert) At Shadows’ gripping climax, Holmes and Moriarty meet in a fortress high in the Swiss Alps, built above the source of the colossal Reichenbach Falls. They play chess and then—as so often happens—turn to fisticuffs. Holmes realizes that the only way to definitively best his nemesis is to seize and pull him over the balcony into the waterfall below, even at the price of near-certain death. In a classic piece of Ritchie theater, the camera is placed midway down the falls, descending and pointing upward, as the two figures locked together plummet towards it.
They are moving faster than the camera, though, and it pans as they pass it. We switch to ultra-slow motion as we watch Holmes’ and Moriarty’s heads in close-up, inches apart and with very different expressions. The film speed picks up again, and we watch them plunging and separating as they continue their fatal descent. It is 29 seconds of virtuoso filmmaking, the sort that Guy Ritchie has made his own. But such visual audacity places the highest demands on a visual effects team, and Framestore helped Ritchie and Jarrett bring it all together.
VFX supervisor Chas Jarrett’s initial hope was that the sequence could be constructed purely through the use of real waterfall elements. Both Quintavalle and McCulloch traveled far and wide as part of the quest for the perfect falls. They both dangled perilously out of helicopters (Quintavalle in Switzerland, McCulloch in Norway), clutching their trusty Canon EOS 5D cameras as they captured plates and footage. In the end, no real waterfall proved up to the sequence’s demands: for a start, it would have had to be over a kilometer tall—a waterfall so high it was only possible to encounter it inside the Soho offices of Framestore.
The actors—Robert Downey Jr. and Jared Harris—were filmed for the sequence using a technique that is becoming increasingly popular with VFX teams.
They were attached through harnesses to two gigantic KUKA robot arms. More commonly used for making cars and the like, these powerful arms can be programmed to move—to puppeteer, almost—the actors through space. While offering great benefits in terms of precision positioning and performance capture from the actors, this technique does have its downside.
“There was an enormous quantity of clean-up work to be done before the actors were ready to be placed in the shots,” says McCulloch. “We had an eight-strong paint and roto team putting in long hours for many weeks to remove all traces of the harnesses and the arms, as well as patching up various bits of the outfits with CG elements.” This action was shot on a Phantom HD Gold camera, which can go up to 1000 fps, though 432 fps proved the ideal speed for the central slo-mo section.
Unsurprisingly, the water itself was the biggest headache. Ben White was Framestore’s CG supervisor for the project. “Several factors made this water more difficult than usual,” he says. “Waterfalls do not lend themselves brilliantly to fluid-simulation software, which is at its best on rivers and seas—things with a surface. Also, the fact that it was such a long waterfall meant that we were dealing with colossal amounts of data, because we had to have pre-roll for the entire kilometer, and when you factor in the time-warp down to almost 500 fps in the middle of the shot, and the fact that the camera is pointing straight up into the descending water, making it simply hard to see—all this meant we had our work cut out.”
In the end, the group broke up the water into several segments, rather than trying to do it in one piece, with each segment comprising multiple layers to achieve the complexity needed. “We had a team working in [Exotic Matter’s] Naiad, on the water coming down from above, and we had another team working in [Side Effects’] Houdini providing a different element for the lower part of the waterfall where it blends in to the live-action elements,” White says. “We also added CG and live-action elements of water vapor and mist being pulled away from the surface, which helped to give a special relationship to the falling actors in this huge space. The live-action elements were most successfully used when we’re looking straight down at the end, because essentially the camera’s locked off, so the photographed element worked well there.”
Another water element proved more readily susceptible to technological augmentation. As the protagonists pass the camera in close-up, Jarrett and Ritchie wanted to see water droplets in motion around their heads. The compositing team found that the latest version of The Foundry’s Nuke (released just in time to help out) gave them a set of tools that helped the group sidestep a 3D solution.
“We found some photos of live-action water droplets from reference libraries,” recalls Quintavalle, “and we put them onto cards—lots of droplets arranged in motion and distributed as we wanted. It was a temporary measure at first, while we waited for the ‘proper’ particles to get rendered. Then as it went along, we thought, ‘Could we make them move a bit as well, add a bit of warping? Could we have a bit of refraction as you’re looking through them? Can we get the time-warp working properly?’ And each time the answer was ‘Yes.’ The drops just kept getting better within Nuke until we realized that we actually had the shot we needed.”
Half of Framestore’s 30-strong Shadows compositing team worked on this one sequence. It was one of the first to be started in October 2010, and among the last to be finished. It was a labor of love, blood, sweat, tears, and 28TB of data caches. (End spoiler alert)
TRAINED ASSASSINS
The other key sequence that Framestore created for Shadows takes place on a train speeding through nighttime countryside. A band of assassins, disguised as British soldiers, has boarded the train and plan on dispatching Holmes and Watson in a tornado of gunfire. Needless to say, our heroes are up to the challenge and explosively foil their adversaries.
Save for one (studio shot) carriage, the entire train and environments through which it passes were created by Framestore. For interior work, there was a lot of greenscreen insertion of backgrounds as well as complementary reflections in the windows. As the fight develops, Holmes and Watson climb out onto the side of the train, so extensive digital environments were required. Many background plates were shot on panorama-cam and put in by compositing, but the entire foreground of the shot sometimes had to be digital, so White’s team created plants, bushes, and scrub.
Other work by the CG team featured in the sequence includes a characteristically Ritchie journey through the inside of a gun, which is pointing at Holmes as it is fired. We see the pin striking the percussion cap of the bullet, and we then see the bullet fly past the camera “inside” the barrel as it flies towards the end (which Holmes has sensibly spiked and which then blows up in the face of his would be assassin).
“We previz’d and conceived the shot working with Chas,” says White. “There is a lot of very subtle work in it—lots of attention to detail in terms of shading and texturing, as well as some lovely depth-of-field work put on by the compositing team.”
White is also particularly pleased with the steam his team created for the train. “Steam is quite a challenge if you’re going to get the authentic roiling, dynamic, very voluminous-looking stuff,” he says. “We had a couple of guys working together—simulating in Houdini and then rendering it in fRibGen, our in-house rendering interface, and using a lot of proper volumetric shading written by the shading department. It was also an ideal chance to use FMote—a proprietary node-based particle-processing tool, which allows effects artists to manipulate particle data prior to rendering”
The sequence featured dozens of digital set extension shots, which were accomplished by the 2.5D team using a combination of techniques, sometimes starting off in the CG department with some rough lighting passes, and then going to the DMP (digital matte painting) team for their magic, and sometimes without CG input, using a combination of in-house tools in Autodesk’s Maya and Nuke. A relatively straight-forward touch added by the comp team was the illusion of motion as the train rattles along.
Choreographed to reflect the intensity of the action inside, the post-added motion impressed Ritchie enormously with the veracity it brought to the sequence.
BREATH OF LIFE
Another subtle touch that pleased Ritchie and producer Joel Silver hugely was an effect that is all too easy to get wrong: the addition of tiny puffs of condensation caused by characters as they speak and breathe in an ostensibly cold environment. The Swiss Alps castle scenes were all shot in a studio, so to convey the atmosphere of a freezing Alpine night, the Framestore team came up with an ingenious solution.
“We developed a reference system using real filmed footage from a cold room,” explains Quintavalle, “We found different types of ‘puff’ representing different syllables, and then used the audio soundtrack to trigger the appropriate puff. It’s a nice hybrid of real elements with an automated approach and was very controllable and quick. A couple of our artists turned out to be intuitively good at this, and they ended up doing 100 or so shots of breath enhancement. You’ll barely register it when you see it, but that’s really the point—it’s only good if you don’t notice it.”
Other important touches by the Framestore team included environmental re-creations for establishing shots of Charing Cross station, Paris, and the environs around a gentleman’s club wherein Holmes faces another acrobatic (that is, heavily wire-removed) assassin.
Sherlock Holmes: A Game of Shadows has enjoyed an astonishing 98% “want to see” rating on Rotten Tomatoes, and seems set to garner another hefty share of this holiday season’s box office. We can but hope that the eminent detective survives to fight another Christmas.