He creates a meticulous one-sixth scale town, Marwen, which he populates with doll alter-egos of people from his life, and photographs it to craft a stunning narrative. Continuing his tradition of pushing cinematic boundaries, writer-director-producer Robert Zemeckis puts audiences inside Mark’s imagination for 46 minutes of the movie.
Here we look at the work by two studios in particular – Atomic Fiction (now part of Method) and Framestore – in bringing this film to life.
Atomic Fiction
It was 2013 when Zemeckis first brought the script to his long-time VFX collaborator Kevin Baillie, with an edict. “Actors’ faces are their instruments, just like a violin is to a musician, and we need to feel every bit of their fidelity through the dolls,” Zemeckis said. “How are we going to do this?”
“Traditional motion capture wasn’t an option, so our original idea was to shoot the actors in costume and augment them with digital doll joints,” said Baillie (co-founder of Atomic Fiction, now Method Studios). “That failed horribly, but it inspired us to try the opposite – instead of sticking CG parts on actors, we’d try augmenting the CG dolls with actor parts. We fused lit live-action footage of Steve Carell’s facial performance with the underlying doll structure, and lo and behold, it worked!”
Baillie and Atomic Fiction then developed a proprietary technique to create a seamless blend of actor and doll – for all 17 “dollified” characters. The process of bringing the dolls to life involved motion capture for their body performances and beauty-lit footage shot from Alexa 65s, which were also mocapped, to drive the dolls’ faces and the CG cameras. Intense teamwork between all departments during pre-production enabled the filmmakers to build the Marwen universe in the Unreal real-time engine. Using a custom iPad application, cinematographer C. Kim Miles then was able to pre-light all of the mocap scenes before a single frame was shot on stage. This virtual production process rallied all of the physical departments around a common cause and let filmmakers wrap each day’s shoot with final cameras, actor performances, and lighting that they knew would look great in the finished shots.
Each doll started as a 3D scan of the actor, was adapted into doll form and digitally sculpted by Atomic Fiction. Miniature Effects Supervisor Dave Asling and dollmaking artisans at Creation Consultants then digitally engineered the sculpts, 3D printed, molded, pressure-cast, hand-painted, and hand-finished them. Atomic Fiction created fully rigged, fleshed-out CG versions matching their physical counterparts down to every tuft of fuzz on their costumes. The miniature Marwen town was meticulously handcrafted by a model-making team who packed it with scavenged and created objects. It was 3D scanned, photographed, and turned into an accurate digital Marwen at Atomic Fiction.
With lighting, depth-of-field and face composites all being integral to the result, Atomic Fiction flattened the VFX workflow and called for all departments to work together in parallel so that Zemeckis could review footage in full context from take one. Using digital versions of traditional photographic tools such as tilt shift lenses and split diopters, artists were able to maintain Hogancamp’s tactile photographic look.
Baillie said, “Bob’s vision was to take the self-healing narrative that Mark created through his photography and faithfully recreate it. The result is that all of it feels alive because, in part, it actually is.”
With Atomic Fiction focused on creating the Marwen world, Baillie enlisted Method Studios to transform Vancouver’s Railtown into New York’s Meatpacking District, blend performance takes with digital morphing effects (which Zemeckis called “blorphs”), composite bluescreen window comps in Mark’s trailer, and craft a unique look for Mark’s drunken flashbacks.
Method VFX Supervisor Sean Konrad said, “The scenes in which Mark is trying to recall a past trauma, but his memory is hazy, were really fun to work on and also rather complex. We created this heavily stylized, very dreamlike feeling by stitching together elements from three or four different plates, and fading portions of those layers in and out, giving the audience a sort of drunk feeling.”
Framestore
The distinctive story required a thoroughly unique approach, and Welcome to Marwen promises a feast of cinematic visuals unlike anything seen before as the action switches between Hogancamp’s day-to-day life and the dazzling world of Marwen. Framestore delivered one of the film’s key early sequences, with VFX Supervisor Romain Arnoux overseeing close to 100 beautiful shots.
Baillie engaged Framestore to achieve technical specificities of the film’s first scene, from re-creating a dramatic plane crash to bringing Marwen’s inhabitants to life. “Baillie explained the process — that we had to use the production camera to project faces onto CG dolls — and I was seduced, because I knew there was no pipeline that would do this. I had never seen a project like this before,” says Arnoux.
Framestore’s pipeline consisted of animation files going straight to lighting, where artists extrapolated from the setups used on the motion-capture stage to create credible exterior lighting that matched the live action. Shots then went to compositing, where the team performed de-aging and completed the integration of the live-action and CG components. Framestore’s pipeline was tweaked in that it allowed for back and forths between tracking and compositing departments to ensure high-quality tracking.
Framestore used advanced motion-capture technology to help turn the film’s stars into realistic-looking dolls. At the beginning of the film, Zemeckis wants the audience to be tricked – he doesn’t want them to think the dolls are actually being filmed, or that the story is taking place in a doll-like world. To create the perfect hybrid, Framestore’s artists projected 75% of the actors’ faces onto their respective dolls. As soon as the plane crashes, the audience is introduced to a fully doll-like world, where only Steve Carell's mouth, eyes and part of the chin is preserved to achieve the desired plasticized look.
“Because we only had a texture from one camera point-of-view to project on the doll, we had to be perfectly aligned,” Arnoux noted. “However, there was always some slight topology variation between the doll’s face and the actor’s face. We used custom tools to calculate the disparities and realign the face, but it wasn’t easy — most of the time, the tracking team had to do it by eye,” says Arnoux.
A key moment was Hogie’s dramatic plane crash, which saw Framestore develop a digital P-40 Warplane (based on photos of Creation Consultants’ miniature) and building the plane’s cockpit from scratch. “We created a hybrid asset,” says Arnoux. “The geometry is based on the one-sixth-scale plane, but we used a lot of full-scale textures. The animators recreated the plane as if it were a full-size aircraft.”
Flying at 250 mph, the plane covered a lot of ground during the film’s aerial opening sequence, where roughly 20 miles of northern European landscape was laid out in full CG. The effects team filled the sky with scattered debris emanating from the plane, which was also simulated at full-scale. When the aircraft crashes into the ground the crumpling foliage, churned-up mud and flames were all brought to life in what was an almost entirely digital shot.
“Our goal was for the audience to immerse themselves in a world that’s imaginary but rooted in reality. Being able to bring these two sides of Hogancamp’s life together in such a unique way was absolutely amazing.”
