Unseen Beauty

The films most people think of as "visual effects movies" are those with in-your-face effects, science-fiction locations, and CG characters. But to filmmakers, so-called "invisible" effects are every bit as important and perhaps even more influential. Invisible effects allow filmmakers to shoot a "location" in any kind of weather at any time of day, to control the lighting, to re-create historical settings, and to enhance imagined locations, as three current films attest: 300: Rise of an Empire, The Monuments Men, and The Grand Budapest Hotel.

Warner Bros.' 300, directed by Nuam Murro and written by Zack Snyder, propels audiences into ancient history as imagined by graphic novelist Frank Miller. Columbia Pictures' The Monuments Men, directed by George Clooney, marches viewers into the horrors of World War II. Scott Rudin Productions' The Grand Budapest Hotel, directed by Wes Anderson, waltzes filmgoers back a little further to the time between the two wars. Each film relied on visual effects artists to create and place the setting in its appropriate spot in history by extending full-size and miniature sets on greenscreen stages, altering location shots, and by building and painting digital environments. The challenges they faced ranged from matching the painterly style of a graphic novel, to re-creating the ravages of war, to making miniature sets believable in a live-action world.

We begin with 300.   

Barbara Robertson is an award-winning writer and a contributing editor for CGW . She can be reached at BarbaraRR@comcast.net . 

War Paint

Artists at visual effects studios create CG environments in a graphic novel style

Zack Snyder's 300 was the second film based on a Frank Miller graphic novel, and as did the first - Miller, Robert Rodriguez, and Quentin Tarantino's Sin City (see "The Art of War," March 2007) - it painted the cinema screen in an illustrative style. 300: Rise of an Empire, the sequel to 300, continues that style. Filmed on greenscreen stages at New Boyana Film Studios in Sofia, Bulgaria, the action flick finds a Persian navy ruled by the demi-god Xerxes and commander Artemisia invading Greece. An army commanded by General Themistocles fights the invaders.

"I call it an encapsule - not a prequel and not a sequel - because it encapsulates the entire time frame before, during, and after the Spartans' [battle in 300]," says Richard Hollander, formerly of Rhythm & Hues, who was the on-set visual effects supervisor. Hollander, a Motion Picture Academy member traces his award-winning career in visual effects back to The China Syndrome, Blade Runner, Star Trek: The Motion Picture, and The Abyss. More recently, he was, until 2006, president of Rhythm & Hues, and from 2008 to 2010, at Pixar, where he produced the Academy Award-nominated short film "Presto."

Much of the action in this "encapsule" takes place on the water, and Scanline, under the supervision of Bryan Hirota, managed that complex effects work involving digital ships battling on the digital seas. Hollander took the visual effects helm for the land-based work, following visual effects advisor Boyd Shermis, who started with the project. "Scanline, which had done the water on the first film, had already begun on the beautiful, stylized water," Hollander says. "Boyd did the previs for a really nice sequence when the protagonist is on a horse and runs across several boats. Scanline took it from previs and integrated the shot of an actor on a buck in Bulgaria into the final shot in the CG environment. It was a monumental effort on Scanline's part."

SCANLINE ARTISTS CREATED the water in most sequences for 300. MPC artists added over-the-top effects to land-based environments.

For his part, Hollander worked with the director on detailed previs for the land-based shots and was on set in Bulgaria for the shoot.  "Everything was on greenscreen except for one day when we shot outside," Hollander says. He worked primarily with The Third Floor and Halon on the previs, which continued even after filming began.

"When the actors are on a stage where everything is green for 360 degrees, they can lose the sense of where things are and what the action is," Hollander says. "So we kept adding more and more detail to the previs. For some scenes, we had CG characters [representing actors] with facial reactions turn their heads in the way we thought they would turn. Some environments were completely detailed. But in others, we were more abstract."

The production crew was in Bulgaria for six months and during eight weeks of production, previs continued. "We'd do Cinesync with the previs studios, go through the scenes, and review them on set with the director," Hollander says.

On set, the actors could see the previs on monitors, and markers on the greenscreen represented elements in the environment that would appear in the final shot. "Patrick Tatopoulos designed the environments. But the actual sets were on small greenscreen stages in Bulgaria, so we would need huge [digital] set extensions. I was gritting my teeth knowing how rough it would be later. The big sets were fine, but the small sets were cramped and not big enough to be lit properly. We had green spill everywhere."

When production ended, Hollander moved on to a previous commitment, A Winter's Tale, while Zack Snyder joined the 300 production, bringing along John "DJ" DesJardin, who had just finished Man of Steel, to supervise the visual effects effort in postproduction. 

Geysers of Blood
"It's a whole CG movie," DesJardin says. "Same as the first. Although a lot of it is painterly, everything is pretty much modeled to a high level of detail. We tried to make it more real than you'd think, so Zack [Snyder] could dial it down or up the contrast or push the color in DI. It has the same tone as the first movie. The main thing we did differently is that all our blood is 3D CG so we could control it. We still have geysers of blood, but they are on the ocean. We could do big washes and really be ridiculous about it."

Three studios tackled most of the visual effects shots, with Scanline sailing through all the sea battles and with Cinesite and The Moving Picture Company (MPC) attacking the land-based sequences.

"I came in at a good time," DesJardin says. "I think the hard part was in Bulgaria. The decisions I was part of happened after the fact to enhance the story. Scanline was so far down the line already that the only input I had was to suggest a few more shots to set up the geography. Brian [Hirota] and Scanline had already developed the look. The water doesn't look unreal. If you don't have a natural relationship between the vessels and the scale of the water, you wouldn't like the movie very much. We did put a huge standing wave in the first battle. But in terms of the fine elements, the foam, ripples, and swells, it's pretty naturalistic."

Enhancing the story for land-based shots involved adding elements to the sets and set designs. "MPC started with a tent palace for shots when Xerxes conquers Athens," DesJardin says. "By the time we got done, Xerxes was in the middle of death and destruction. We had bodies tacked to columns."

Charley Henley supervised the work at MPC, which also included the burning of Athens on the Acropolis, the post battle of Hell's Gate on a cliff edge, Xerxes' bridge over Hellespont, and the epic Marathon Battle.

For the Marathon Battle environment, the MPC team shot stills on a beach in Spain, which became a basis for modeling, textures, and matte paintings. They also projected textures onto models to create a CG beach. Digital matte painters created a cyclorama for the mountains and the sky, and added CG elements - rain, blood, ash particles, and severed limbs.

Both studios had a unique challenge: To create what in other films would be outrageous environments. "Charley [Henley] at MPC embraced that right away," DesJardin says. "We went from shots with Xerxes in a palatial litter, to Xerxes being carried by slaves with a whole circus of midget archers and animals - rhinos, elephants, and more - and huge flags carried by guys in long lines. There was so much to be done, if he hadn't gone for it at the beginning, we wouldn't have made it to the end."     

Keeping Cool
The crew at Cinesite, led by visual effects supervisor Richard Clarke, had the additional task of creating environments that harken back to the first film. "They did a good job of pushing the design further than where it was to get the sense of the city layout in the first movie," DesJardin says. "I remember Richard [Clarke] was always worried about going too far. Our challenge to him was to make us tell him it's over the top. We said, 'You can't make us mad. We love over the top, so try it.'"

At Cinesite, 10 artists led by Environments Supervisor Thomas Dyg created the over-the-top shots. Seven were traditional digital matte painters, and three were more generalists. "Normally, we build a 3D setup, and on top of that, we project photographs and paint," Dyg says. "But for this show, we wanted a more painterly feel."

Many of the shots were at water level with a sheer cliff rising behind them to form a backdrop. For those shots, the artists started with a 3D model of a cliff created using photogrammetry from stills taken in Utah. "We altered and extended that cliff to create a whole coastline for our cliff," Dyg says. "It formed a template for the matte painters. Our brief was to have a connection to reality, but to be more painterly than photoreal."

ONLY A FEW SHOTS in 300 were filmed outdoors, and even these were augmented by, in this case, artists at Cinesite.

A sequence that takes place in Sparta sent the artists to DVD players to determine the layout based on the first film. "As with any graphic novel, the shapes were quite simple in order to work as silhouettes," Dyg says. "So we created simple building blocks to put the city sloping up the side of a mountain. It was kind of a 2.5D process, but the artists with the skills to do the look we were after were not used to working with projections, so they just painted."

Once they finished, they handed the paintings to another environment artist working in The Foundry's Nuke who did the projections. "We did all the environments in Nuke, and then, instead of having 10 compositors working on a sequence and risk the continuity going to hell, we had one key person piece together the environments, add the haze, the first passes of glow, and the color correction, and pass that on with various mattes to compositors who did the final adjustments."

The process was similar to that used for previous shows, but it was the last time it would be implemented. "We're still using Nuke for compositing," Dyg says. "But, we're no longer using it for projections. We're using [Luxology's] Modo instead (see The Art of War, page xx). That was a choice I made shortly after 300."

Although the original brief from production to the painters was to create environments using colors they might see in the real world, the palette changed during the process. "As we worked through the project, colors were going out left and right," Dyg says. "The color palette was definitely tuned to a monochromatic feel."

Reality also moved toward surreal with the addition of atmospheric effects. "The compositors added glows, flares, motes, and sparks in the air until they created a dense atmosphere," Dyg says. "It's oddly believable and completely surreal."

"That was the Cinesite version of fountains of blood," DesJardin says with a laugh, referring to Scanline's CG blood for the ocean battles. "I screwed up their paintings with tons of embers and floating things in the air. Zack likes that stuff. It's all for effect and drama. This is not the history channel. It's supposed to be cool."

The Art of War

Historical accuracy underpins the efforts of artists who created realistic CG environments for this World War II film

Facing an impossible mission, a platoon of museum directors, curators, and art historians risked their lives to save artistic masterpieces stolen by Nazi thieves. They became known as the Monuments Men. Writer, producer, director, and star of the film, George Clooney, brought together a team of A-list actors to tell their story: Matt Damon, Bill Murray, John Goodman, Jean Dujardin, Bob Balaban, Hugh Bonneville, and Cate Blanchett. The Columbia Pictures film received mixed reviews, but in a Washington Post review, Ann Hornaday writes, " The Monuments Men looks terrific, with Cinematographer Phedon Papamichael shooting the mostly German locations with a lovely, desaturated patina."

Some of those locations, however, were created inside visual effects studios in London. Angus Bickerton supervised the visual effects created by Blue Bolt, Cinesite, Method Studios, The Moving Picture Company (MPC), One of Us, Senate Visual Effects, and Snow Business.

"The overall approach was that Angus and George [Clooney] wanted everything as real as possible," says Arundi Asregadoo, visual effects supervisor at MPC. "They wanted to base everything on available photo reference. And, they built a lot of miniatures. So we took a 2.5D approach, projecting photographs onto geometry to create the environments."

All told, the artists at MPC created 80 shots including the interior and exterior of Santa Maria delle Grazie, the aftermath of the Omaha Beach landing and an encampment beyond, and the destruction of Cannes.

MINIATURES AND PHOTO REFERENCE helped The Monuments Men filmmakers create historically accurate visuals.

Santa Maria delle Grazie is the church and convent in Milan that holds Leonardo da Vinci's mural "The Last Supper." During World War II, bombs from British and American planes destroyed much of the refectory, and in the film you can see into the partially destroyed building. Fortunately, the wall was sandbagged and the extraordinary mural saved.

The set built on stage for filming included the sandbagged wall with the mural and another wall. "They had printed a re-creation of "The Last Supper" and pasted it onto the wall of the physical set," Asregadoo says. "But the perspective was wrong."

So for the final shots, MPC needed to re-create da Vinci's mural. "Yes, we painted 'The Last Supper' at MPC," Asregadoo says. "We had an Italian do it."

The environments department also built the standing interior of Santa Maria delle Grazie. Previously, MPC was one of several studios that had worked with Bickerton on Angels & Demons, and for this film, the crew employed an evolution of the same techniques (see "Elemental Effects," June 2009).

"Back then, we had to do a survey or use a Lidar scan to create a mesh, onto which we could project geometry," Asregadoo says. "Now we use Agisoft [PhotoScan] to create the mesh. We took a number of photos and used a photogrammetry approach to generate a wire mesh to build the geometry."

To be able to accurately reproduce the interior, one of the artists took the digital stills. "We took a series of photographs inside Santa Maria," Asregadoo says. "The software ingests that, flattens it, creates a camera, and consolidates the images into one scene. It can then generate a mesh. From that mesh, we create clean geometry with a perspective that matches the foreground set."

The geometry "cleanup" happens within Autodesk's Maya. "When the solve comes through, it's incredibly heavy, but it saves on trying to pay for a Lidar scan," Asregadoo says. "We can just send a guy off to take 200 photos and then do a solve. It's very cool. We're using it quite a lot now."

With the geometry in place, the artists projected the appropriate photographs onto the models to finish the interiors. Because they had a number of bracketed shots, they could adjust the exposure on the stills.

"We did a lighting pass through [Pixar's] RenderMan to put a kiss of light on parts of the building and give it more depth," Asregadoo says.

Destruction in France

Similarly, for the rubble in this sequence and in a sequence with destruction in Cannes, the crew relied again on photographs for photogrammetry and textures. Near Limoges in west-central France is the remains of Oradour-sur-Glane, a village destroyed and most of its people slaughtered by the Waffen SS in June 1944. The ruins have been left unaltered as a memorial.

"That village was a basis for our photographic reference," Asregadoo says. "We used a [Seitz] Roundshot camera with a nodal head that takes photographs in 360 degrees, up to 72 images, and stitches them together. When we place a camera within that cube in Maya, we can create a shot, so we used it extensively to record environments."

ARTISTS AT CINESITE used Luxology’s Modo to create and texture geometry, and then populate scenes with procedurally generated rubble.

For the destruction in and around Cannes, the visual effects crew extended a massive set built in an industrial site outside Berlin with smaller sets representing destroyed towns. In addition, Bickerton made a series of destroyed building miniatures at one-third scale. "Imagine a cube divided into four quadrants," Asregadoo explains. "Each had an interior or façade of a destroyed building that we photographed under different lighting conditions. We used those for the mid-ground. Beyond that, we created a digital environment using the village in France."

Similarly, for the Omaha Beach encampment, the MPC artists added to a set built in a Russian armory storage site outside Berlin. "They had 100 tents and filled their camp with extras," Asregadoo says. "We extended it to three times that size using photographs from the set. And, we added dunes to link it to Omaha Beach. We did a lot of research and tried to be as accurate as possible. But we were led by the set dressing. George [Clooney] was keen for us to match that. He didn't want to see a lot of visual effects."

The artists also created the aftermath of the Omaha Beach landing. "We had to make the beach feel like a month after the landing," Asregadoo says. "So we extended dunes, replaced skies, put boats in the horizon. They wanted scarring but nothing showing heavy bombing."

The project took a group of approximately 40 artists that included technical directors, lighters, and modelers, in addition to the digital matte painters and compositors, about six months to create. "It was a tight team," Asregadoo says. "Usually productions are so big you don't see half the team. The thing we pushed most was the photogrammetry. That was a big thing for us. It saved a lot of modeling time. When we were creating the destroyed city and towns, we could use photogrammetry of the destroyed vehicles on set and project them into our scenes seamlessly."

Gold Mine, German Destruction

At Cinesite, Thomas Dyg led a team of artists that created digital environments for two sequences in The Monuments Men: one in the Merkers salt mine used by the Nazis to store gold bars and looted art, and another in the rubble that was once the German city of Siegen.

For this film, rather than creating the environments in The Foundry's Nuke, the artists used Luxology's Modo. "The shot in the mine is a locked camera move with a tiny weave," Dyg says. "Bill Murray flips a switch and lights come on one after another to reveal the full length of the mine with the gold bars and boxes. So, we chose to use 3D for the rough layout and paint on top of that."

On set were the first four or five meters of the cave, a small push wagon, the first rows of gold bars, and the actors. The visual effects team stretched the cave into the distance and put 3D lights inside the rough 3D layout. Then, they painted the walls. "We created a matte painting with the final, fully-lit look and used the 3D lights as a matte to reveal the painting," Dyg says. "The cave is black, and as each light turns on, it wipes on a little of the matte painting. The compositors could work with the client to tweak the timing and the order in which the lights come on. It was traditional matte painting meets a 3D approach."

The crew also used Modo to extend the field of rubble for the Siegen sequence. "It was only about three shots in the film, but Angus [Bickerton] told us the scope of the shots made them among the biggest visual effects shots in the film," Dyg says.

In that sequence, the camera follows Clooney as he walks onto what might have once been a town square. The camera dollies toward him, and then in two following shots, we see the bombed-out city. The production crew shot that footage in a rock quarry. In the plate Cinesite received, they could see rubble for nearly 100 meters. The digital matte-painting group added ruins far into the distance - CG walls and rubble, and far beyond, the hills and forests.

The artists started with photographs taken of the rubble on set. "We had photos from every angle," Dyg says. "We did photogrammetry from that, which gave us crude but reasonable details, and replicated it across a vast landscape. This was the first time we used Modo heavily and for full shot design."

In addition to photogrammetry of the rubble, the Cinesite artists used photogrammetry on photographs of miniatures - buildings with gaping holes that the production crew had fashioned. "For this sequence, we wanted only the odd jarring wall sticking up here and there," Dyg says.

THE CAVE BEYOND four meters is 3D geometry textured with a digital matte painting created at Cinesite. As each light blinks on, it wipes on a bit of the painting.

Then, the artists began placing the rubble and ruins across an undulating plain created in Modo. "We randomized, scaled, and rotated the pieces," Dyg says. "And we did some things procedurally inside Modo. We had Modo place the thousands and thousands of pieces procedurally, which was obviously quicker."

This method turned out to have artistic advantages as well. "We really benefitted from happy accidents," Dyg says. "The procedural approach gave us small areas that were really nice and natural, and also clusters of rubble that we might not have thought of in advance. We'd hit the random generator and keep the areas we liked and refine the others. Traditionally, when we do matte painting, we scour the Web or our own library, cut out pieces, and put them together collage-style. This was like a 3D collage, but procedural. It sounds easy when I describe it. It wasn't quite that easy, but it was very forgiving. If there were dodgy pieces of geometry, it all added into the ruins."

For textures on the rubble, the crew used the same photographs taken for photogrammetry. "The textures held up nicely in the medium to far distance," Dyg says. "And Modo gave us the physically correct lighting, which helped us sell the shot. We knew the date and time they shot the footage in Berlin. Modo has a physical sky where you can dial in a location, date, and hour, and you get a lighting solution that certainly was, in this situation, very close to the plate photography. Not only the angle of the shadows, but the light intensity in most situations. It gave us a really nice starting point for tweaking the lighting."

In all, Dyg is pleased with the new Modo-based procedural approach they took to create environments for The Monuments Men. "You could argue we were lucky because we had a ruined city and maybe for a different shot we couldn't use this approach," he says. "But we've since used it for buildings and trees on another show. It seems that when visual complexity is high enough, it's more about shapes and chaos. Your eye accepts it if it's reasonably decent. It was really nice to get a lot of stuff for free. It was more about being an artist and less about being technical."

Everything Is Under Control

Artists at Look Effects enhance the invented world in Wes Anderson's The Grand Budapest Hotel

What need would the Oscar-nominated director of The Fantastic Mr. Fox, Moonrise Kingdom, and The Royal Tenenbaums have for visual effects?

Gabriel Sanchez, visual effects supervisor at Look Effects, has the answer: "Wes [Anderson] didn't want a CG feel or over-the-top effects; the shots needed to feel like he captured them in camera. But, he still utilized the benefits of CG visual effects."

Invisible effects, that is. With a Wes Anderson twist.

The film takes place in nested time periods: the current time, the 1930s, and the late '60s/'70s. Because Anderson wanted to film everything in camera, when live action wasn't possible, he used miniatures shot on greenscreen, no digital set extensions necessary. The postproduction crew at Look Effects added digital backgrounds to the greenscreen miniatures and integrated them into the plates. Sanchez lists the studio's work as digital matte paintings, fix-it shots, and timing adjustments. Again, though, with a Wes Anderson twist.

"At times when they shot in the city and in areas around the museum, we removed modern neon signs and power lines, but that was the majority of our changes for historical accuracy," says Sanchez. "Basically we had to deal with a lot of integration of live action with miniatures and stop-motion animation, and with the stop-motion animation, we needed to do a lot of speed changes to make it flow and feel real." The artists worked primarily in The Foundry's Nuke, occasionally in Autodesk's Flame, and, for matte paintings, in Adobe's Photoshop.

Two miniature hotels, a pink version for the 1930s and a concrete gray version for the '60s/'70s, provided the setting for the caper - the theft of a Renaissance painting. "We adjusted textures, added a film-grain pass, and cleaned up the miniatures to integrate the models and make them feel like they're really in the shot," Sanchez says. "The director wanted the film to feel real, but he didn't want to lose the charm of the miniatures. If we made it look like a real building, we'd lose the feeling of the actual miniature. So that was a challenge."

To enhance the lighting with proper shadows, the crew built rough geometry that matched the miniatures to work with CG lighting, which was especially important for a shot of the '60s hotel that the filmmakers wanted to play as time-lapse photography. "We had one locked-off frame of the miniature," Sanchez says. "But we needed to make shadows move as the sun moved across the building. So we did lighting simulations using simple geometry that matched the surface of the miniature."

The biggest sequence that called for Look Effects' involvement was a ski chase through the snowy Alps. It opens with the actors composited into greenscreen shots of a miniature observatory built with an operating 360-degree dome. To travel up into the Alps, the characters ride in a miniature aerial tram with frosted glass windows.

As they did for the hotels, the visual effects crew used 3D lighting to make the aerial trams believable. "Wes [Anderson] likes his miniatures, but he's also very particular and detailed," Sanchez says. "As the tram moved across the frame, we got a note saying it didn't feel like there was enough depth, that we should enhance the shadow."

The artists could have created a 2D shadow with a matte, but it wouldn't have been accurate; the shadow had to travel with the tram. "So we made a simple 3D cube  and projected light onto it to get the moving shadows and to put reflections on the glass," Sanchez says. "We rendered them out in layers and gave them to the 2D artists. Wes [Anderson] had no clue I made a CG model of the tram. He thinks it's a 2D gag. I don't think he has done traveling miniatures before."

Then, the camera follows the skiers racing downhill through the miniature trees. "That's where stop motion came into play," Sanchez says. "They had miniatures of the actors, a motorcycle, a bobsled. They'd move something and snap a picture. We created matte paintings for the background that were 2D because they were so far in deep space. Then, we had to deal with the miniatures. They had little trees, and a snow pack the stop-motion characters skied on. We needed to enhance and manipulate the stop-motion characters and create falling snow and mist kicked up by the skiers."

AT TOP LEFT, LOOK EFFECTS ARTISTS painted the far background behind the miniature set. At top right, the hotel is a miniature that the artists integrated with live-action footage.

One major challenge was in the timing. "Since it was stop motion, the timing was correct, but it didn't feel organic," Sanchez says. "There was no motion blur, no soft edges. That made it different from filming a live skier. So we added motion blur and soft edges."

Anderson's attention to detail came into play in another sequence. "Wes is involved with every detail of the film," says Sanchez. "He even checks the grain on every shot, he's that involved. There was, oh my God, a shot where the camera is traveling down a hallway as doors opened. When we saw the shot, we thought it looked like a clean take. We got a note that said, 'Looks great. Now, let's adjust the doors.' He wanted to change the timing of when the doors opened. One needed to be slower, another faster to hit the beat. That's an invisible effect we would never have imagined."

From CG environments that blend seamlessly into a painterly, graphic novel style, to historically accurate reproductions of a war's aftermath, to the fine-tuning of in-camera footage of miniatures and stop-motion animation, visual effects artists working on 300: Rise of an Empire, The Monuments Men, and The Grand Budapest Hotel created "invisible effects" that made the filmmakers' inspirations visible.