Star Wars IV astonished audiences in 1977 with unparalleled visual effects created by Industrial Light & Magic artists in LA using practical effects, and who completed the first trilogy in rambling studios north of San Francisco. The second trilogy began in 1999 with A Phantom Menace, created by digital artists working in a new San Francisco campus. Fast forward to today. Hundreds of ILM artists in four global studios have completed work on two films in the third trilogy. Of the eight Star Wars films, seven have received Oscar awards or nominations for visual effects, including the latest, Star Wars: The Last Jedi, episode eight.
3D models, digital environments, and virtual cameras replaced the first trilogy's models, miniatures, and motion-controlled cameras long ago, and painters now paint digital ships in Photoshop, not balsa wood in a model shop. And yet, artists creating the most recent Star Wars film, Star Wars: The Last Jedi, with the most advanced visual effects tools and techniques available, have not forgotten the past. As they did for the seventh episode, The Force Awakens, they looked at the early films to continue the saga.
"Rian wanted to do as much as possible in camera," says Ben Morris, overall visual effects supervisor and creative director of ILM's London studio, referring to Rian Johnson, writer and director of Disney/Lucas-film's Star Wars: The Last Jedi. "So, we literally started with the question, ‘Is there a world in which we build all the spaceships as miniatures and shoot motion-control passes like we did 35 years ago?' Tim Keene [visual effects producer] and I ran some numbers. To do even one battle would have been prohibitive in time and cost."
It didn't matter. They soon convinced Johnson it wasn't necessary.
"We can make an effect look like anything today," Morris says. "We can make a CG model look like balsa wood or sparkly CG. It's entirely subjective now. Once Rian [Johnson] understood we wouldn't leave behind the aesthetic, the methodology became secondary."
In Star Wars, Morris notes, the aesthetic, that is, the standard for believability, is low-tech design. "It's sci-fi, but it takes place in an amazing lived-in universe," Morris says. "The limitations George [Lucas] had in terms of costumes, sets, and props created a junkyard of low-tech design, a language that extends to visual effects. We used model kits as building blocks. And movement is even part of the language. When you look at the stop motion Phil Tippett gave the walkers, it's astonishing. But, not for a modern audience. With our new walkers, the AT-M6s, we wanted to bring that slightly jerky mechanical feel and marry it with high-resolution models, incredible action on the ground, and better weight shift. We aren't breaking the umbilical. We're working within the visual language."
Similarly, the camera moves in Jedi reflect physical cameras.
"Rian didn't want a magic camera," Morris says. "He wanted it to feel like the film could have been shot with a physical camera, not in space, but with miniatures. We tried to avoid fast zooms, breaking too hard, objects that would have torn off a camera lens when they went by. But at the same time, we put together sequences that could not have been executed 25 years ago."
Or, even a few years ago. Take Snoke, for example. In The Force Awakens, the character who appears on screen is a 25-foot hologram. Johnson wondered about putting an actor in makeup on set to create Snoke, but it was clear that Snoke's carved face couldn't be created with makeup. He had to be digital.
"Rian was concerned," Morris says. " ‘Can we achieve that?' he asked. I said, ‘Of course we can. We have the best team in the world.' "
Enter the Galaxy
Officially, there are 1,848 visual effects shots in the film, but Morris also supervised an additional 360 digital makeup fix shots and between 150 and 200 production fixes by Stereo D, bringing the total to well over 2,000.
Preproduction began at Pinewood Studios near London, with Morris and Keene on site, and with concept art and designs under way by ILM's Senior Art Director Kevin Jenkins, who became Lucasfilm design super-visor for the film. Albert Chang from The Third Floor worked with Morris, Johnson, and a small team of artists at Pinewood doing previs that moved storyboards from Dave Allcott into three dimensions.
At the four ILM studios – London, San Francisco, Vancouver, and Singapore – approximately 1,000 artists contributed to the film, along with seven third-party collaborators. The visual effects Oscar nominations given to Morris and ILM London Supervisor Mike Mulholland honors their work.
"Dividing the work was almost like a casting process," Morris says. "We knew we would do [the digital characters] Maz and Snoke in London to continue the work on Force Awakens. And, the big sequence with the Fathiers, the large horse-like creatures, would be in London because I wanted to be close to the animation team. Then, we chatted with the supervisors from different locations. Mike Mulholland in London was keen to do a space battle. Eddie Pasquarello in San Francisco worked with Alex Prichard in Singapore to do everything in the third act. Dan Seddon had the megahangar, the island, and worked with Rodeo FX and Hybrid in Montreal. One of the nice things about these films is that everyone wants to work on them. So, we wanted to give everyone some meat."
Filming took place at Pinewood, on location at Skellig Michael island, the Dingle Peninsula, and other places in Ireland, as well as in Croatia, Bolivia, Iceland, and the UK. Continuing from the last scenes in The Force Awakens, the film begins on Skellig Michael, where Rey (Daisy Ridley) hands a lightsaber to Luke Skywalker (Mark Hamill), but many other shots in the film took place on greenscreen stages with practical sets at Pinewood.
In addition to the actors on set were several characters, some of which would become digital in postproduction. Puppets for the horse-like Fathier appeared in five shots but otherwise were digital, created in London. Half the cute Porgs were animatronic, but to extend their range and humor, digital Porgs were animated in Vancouver and San Francisco. There was an animatronic for the crystal foxes, but it wasn't in the film. They were created in San Francisco. Andy Serkis performed digital Snoke wearing motion-capture gear, and Mark Hamill does a cameo performance as Dobbu Scay, a drunken digital toad alien that burps on BB-8.
"Mark said, ‘Give me a character. I've never done the suit thing,' " Morris says. "So we built this ginger-haired character with a monocle."
BB-8 was an animatronic most of the time, although when he's running and when he rides in the back of Poe's [Oscar Isaac's] X-wing, he's digital.
"The actors love having the practical BB-8 on set," Morris says. "But, you can't tell the difference with the CG version."
Yoda was an animatronic, enhanced with effects to appear as a hologram.
Many of the stormtroopers were actors in suits, although when you see crowds of troopers, most are digital. And, stunt actors and animators performed digital doubles on motion-capture stages in London and San Francisco.
MODELERS REFERENCED SHIPS FROM THE FIRST TRILOGY TO CREATE THE ENORMOUS NEW DREADNOUGHT.
Although the crew motion-captured Lupita Nyong'o's performance as Maz Kanata, because Maz differs so wildly from Nyong'o in appearance and action, animators keyframed the character that appears in the film.
A Shipyard in London
Work on the space battle at the beginning of the film and on Kylo's attack on the resistance cruiser began in London with previs from The Third Floor; the two studios shared Autodesk Maya scenes. Once Johnson approved the previs, ILM artists rebuilt the shots using assets from a library managed, updated, and, when needed, created at the London studio.
"We looked after all the spaceship assets," Mulholland says. "We updated the spaceships from Force Awakens to bring the shaders into the new Pixar RenderMan RIS library, and reworked and remodeled ships like Poe's X-wing that had close-ups. And, we had a whole set of brand-new ships including the A-wing, Resistance Bomber, Dreadnought, and Resistance Cruiser. We distributed the assets around the world."
Jenkins designed a majority of the ships working with Director Johnson and Production Designer Rick Henricks, printing out some models with a 3D printer. At ILM, artists in the digital modeling studio added details and textures. All the artists used digital model kit libraries scanned from practical model kits for Rogue One.
"We added a few more pieces to go into that library," Mulholland says. "It's a fantastic resource that gives us a common language, a common geometry, and an established methodology for adding huge amounts of detail to the ships."
Modelers built some ships from concept art and models that Jenkins created, although for the A-wing, they needed to match a full-size, practical A-wing on a gimbal.
On the other hand, knowing that the Resistance Bomber would be blown up, ILM modelers built it from the inside out, constructing it with destruction in mind so that during the explosion, it would reveal the interior details. Various set pieces, which would be scanned for digital models, gave Veronica Ngo (who played Paige Tico) a practical bomber to work inside.
"The special effects guys built a full-size magazine 20 or 30 feet tall that could drop bombs downward and curve away onto a horizontal plane, which enabled a practical bomb drop," Mulholland explains. "We matched the set and tracked all the bombs individually so we could take over and have digital bombs keep falling vertically."
For the huge Dreadnought and Mega Destroyer ships, modelers approached them as if they were building a city.
"We couldn't load the entire models," Mulholland says. "So we built them in a hierarchical style and approached them in layers with large forms, silhouette shapes, and then three or four levels of details that were almost like fractals."
Space Battles in London
Approximately two-thirds of London's space battle shots created in previs held the same shot composition through production. The rest were redesigned or needed different camera moves.
"We simulated everything having to do with the camera," Mulholland says. "We tried to make sure our virtual camera didn't fly too close to anything. And, Rian [Johnson] wanted our virtual camera to have a volume behind it like a real-world camera, which would have had a physical size and a crew."
Animators referenced footage of planes in dogfights and watched the early Star Wars films to learn why shots in those movies felt the way they did.
"[Senior visual effects supervisor] Dennis Muren gave a great talk and did a comparison," says Steve Aplin, who was overall animation supervisor for the film. Muren has worked on all the Star Wars films and received multiple awards for those films and others. "He had some people from ILM San Francisco put together a test. They took an original shot from Empire Strikes Back and tried to get as close to that with a CG version as they could to see what we could learn from it."
As a result, the spaceships animated for Return of the Jedi don't have much of what Aplin calls "gymnastic action." There are a few barrel rolls and one dramatic hand-brake turn by Finn in a ski speeder as he nears the mouth of a giant cannon.
"One of the hardest shots was Kylo's attack on the Resistance Cruiser," Aplin says. "Although it was split into three shots in the end, we animated it in one long go. Rian [Johnson] wanted that moment to show what an amazing pilot Kylo is. It took a delicate balance of camera and ship choreography, combined with foreground and background explosions, to frame the action and guide timings."
For destruction, the team in London mainly relied on Side Effects' Houdini, as they did for other effects. Other tools included Maya, RenderMan, and Foundry's Mari, Katana, and Nuke, as well as ILM's Zeno, which provided the background pipeline.
Snoke
On set were four "Snokes" – a maquette built by Creature and Special Effects Makeup Supervisor Neal Scanlan and his team; two stand-in actors – one for height and one for skin texture; and Andy Serkis fitted with motion-capture dots and facial motion-capture cameras. Once Johnson had directed and filmed Serkis, the ILM team moved the other three "Snokes" onto the set for reference.
When the shots moved into postproduction, though, it became clear that Serkis's motion-captured performance of Snoke as a twisted and deformed character needed to change, and the maquette design wouldn't work, either, for the same reason.
"Andy's voice is so deep and resonant, it didn't fit," Aplin says. "We redesigned Snoke to make him more formidable, reshaping his face so it wasn't as scarred and damaged, and straightening his body to give him more of a presence. Before, when he sat on his huge throne, he was diminished."
Character developers at ILM captured Serkis's face with the Disney Medusa system as he performed FACS sequences. Modelers used that data to sculpt face shapes on a digital model of Serkis and equivalent shapes on the Snoke model.
A team of facial motion-capture targeters translated the live-action capture to the digital Serkis for validation, and then to Snoke for the animators to refine.
"Andy's performance was fantastic," Aplin says. "But he has an incredible, malleable, fleshy face. Because Snoke's skin is stretched thin over his skull, we had to reduce and tighten the amount of motion."
Horses for Courses
Animators in London also performed the horse-like creatures called the Fathiers for a sequence in which they chase through a casino. To film the actors during the sequence, ILM created previs that drove on-set rigs that Scanlan's team built and the actors rode.
"We went on set to get an idea of these creatures' size," Aplin says. "The rigs were an incredible piece of engineering." To give the huge creatures the agility they'd need for the sequence, animators referenced big cats, horses, and especially a run cycle of a moose found online. Lead Lighting Artist Steve Hardy joined in the chase by creating a library that allowed animators to publish cycles on a path.
"We could change the Fathiers' speed and interactively plant their feet, and they'd automatically follow the curve in the spline of the path," Aplin says. "We could put different sections of animation on the path and piece their timings together. It was simple, fast, and really helpful."
Also joining the chase were animators in ILM's San Francisco studio who took over when the Fathiers land on the beach. Most of the work in San Francisco, however, landed on the planet Crait, where the Resistance fighters take cover in a mine, and a battle ensues in the third act.
Blood-Red Trails
The mineral planet Crait is stunning. General Leia [Carrie Fisher] leads her ragtag band of fleeing Resistance fighters into an abandoned mine here occupied by white Vulptices, fox-like animals covered with sparkling, crystalline fur. Outside, sun glints off the planet's white salt surface, with blood-red soil a scratch beneath the crust. Gouged into the terrain is a long canyon lined with jagged, red crystals.
To create the planet, the generalist (digital matte-painting) team at ILM San Francisco worked with plates shot by Ben Morris at a Bolivian salt flat.
"The first part of the sequence looks rather similar to the reference photography, but as the ground gets torn up, it became more of a challenge," says Pasquarello, visual effects supervisor at ILM San Francisco. "The whole idea is to have it become bloody red during the battle."
On one side are giant, new, gorilla-footed walkers, the AT-M6s, flanking a massive First Order cannon aimed at the mine entrance. Protecting the mine are the Resistance fighters in ski speeders, piloted ships that zip across the surface on one ski.
"The mono ski isn't very big, and it's intended mostly for balance," Pasquarello says. "But we wanted it to kick up a trail of red."
Animators drew guidelines for the ski speeders in digital environments based on previs shots that were updated in postvis to work better with the plates. Then effects artists carved into the crust and sent trails of red dust into the air as the speeders cut grooves into the surface.
"It was a three-step process," Pas-quarello says. "The effects artists used the guides for several layers of particle emitters. The ski has a shovel-like blade at its base, and we emitted particles under the surface from that, and then we had several passes on top. Red minerals get thrown into the air and emit more particles as they hit the ground." Effect artists also created beautiful red explosions and blasts from the AT-M6s that raise clouds of red dust mixed with white salt.
As the Resistance fighters near the oncoming First Order forces, Finn (John Boyega) aims his ship directly into the mouth of the cannon.
"It was supposed to be a serene moment," Pasquarello says. "Violent outside, but inside the cannon beam, almost like pure light. They lit him very red on set. We gave four elements to compositing: a tube of light; a high-energy pulse down the center; plasma – a fluid simulation – running through the tube to give it motion; and an environment reacting to the plasma. Then compositing made it more intense."
Finn doesn't stop the cannon. It blasts the mine door, and Luke Skywalker walks out a door surrounded with flames created by effects artists using ILM's Plume. The AT-M6s aim all their firepower on him, creating craters of red.
"When Luke and Kylo face off, each shot is a one-off," Pasquarello says. "When Kylo lands, the ground is blood red. As they talk and fight, ash starts falling back down, and by the end, it's pure white again. All the explosions were CG, done by our effects team."
Crystal Caverns and Foxes
Earlier during the battle, Rey arrives in the Millennium Falcon and lures the First Order TIE fighters on a fast flight into and through a narrow red-crystal canyon.
"That's an entirely 3D-generated environment," Pasquarello says. "The whole point was that as they ride, the canyon gets increasingly narrow. So we modeled the basic shape of the cave first to get the ride. Then we augmented it with hundreds of thousands of red crystals and stalactites. The ship hits some, so we grew and populated the cave with stalactites, by hand in some cases." (A separate team led by Bill George repurposed many of those assets as well as vehicles and environments from the battle to create a Disneyland ride.)
Meanwhile, crystal foxes inside the mine reveal an escape route for Leia and the Resistance fighters. A practical fox built by Scanlan's team provided lighting reference and moments when the animals were quiet. But once in postproduction, the foxes evolved into creatures that resembled a leaner animal.
"Rian wanted them to have fox characteristics, but at the same time to be more delicate and skittish," Pasquarello says.
Modelers painstakingly placed various sizes of quills and crystals on the creatures, all of which had to be simulated and lit.
"Going from crystal to hair was a rough transition for the lighting team," Pasquarello says. "The crystals are bigger than the quills."
Outside the exit, the generalist team provided a wall of ice and floated the rocks and boulders that Rey lifts to free the Resistance fighters. Artists in San Francisco worked with Isotropix's Clarisse, Maya, Planetside's Terragen, Nuke, Katana, ILM's Plume, and RenderMan RIS, on the studio's Zeno pipeline, to create their shots.
Sharing the work on the third act were artists in ILM's Singapore studio led by Prichard.
For the cockpit shots of the speeders during the battle on the planet surface, the Singapore artists added reflections, layering in the sky, ships from other shots, the red trails in the white salt surface, blasters flying by, and maybe a background ship spinning out. Each of the 30 or 40 cockpits demanded a custom treatment.
THE CRYSTAL FOXES THAT APPEAR IN THE FILM ARE ALWAYS DIGITAL CHARACTERS.
"We might put eight things in the reflections," Prichard says, "maybe 12 things if we add in dirt and grime, all based in reality. You might not notice the trails, but they help focus the eye where we want it to go."
They also opened the third act by flying – and exploding – the Resistance fighters' slow transporters during their retreat from their cruiser to Crait as the First Order's Mega Destroyer blasted down on them. The artists created the explosions and transporter destruction entirely within Houdini.
"We usually add 2D elements, but in this case there were so many we did everything in Houdini," Prichard says. "First, we used Houdini rigid-body simulations to break up the ships. Then we added the explosions with lots of small shooters to help give a sense of scale and size."
The Sacrifice
One of the most visually dramatic sequences takes place during that Resistance retreat when Vice Admiral Holdo (Laura Dern), the last Resistance fighter remaining on the cruiser, turns the big ship, shifts into lightspeed, and aims it straight at the Mega Destroyer. She sacrifices herself to help the Resistance escape.
"When I first read it in the script, it ticked all the boxes," Morris says. "We'd have crazy chaos and destruction. A big moment. But Rian [Johnson] said, ‘You know what? I want to do the complete opposite. No sound. No crashing explosion with huge, deafening noise. Let's think about how that might work.' We put a lot of detail into these shots, but the clever thing was the idea, the visual design."
Although originally slated for London, the shots ended up in Vancouver a few months before the release date.
"We started thinking about how we could represent something moving at the speed of light and destroying things it travels through," says Dan Seddon, visual effects supervisor at ILM Vancouver.
They looked at images of particle accelerators, at the neutrinos branching off in the Hadron Collider. They did tests building up layers and effects elements, while at the same time, concept artists created images with hyper-exposed scenes.
"We combined all those with an over-the-top lighting design," Seddon says. "We looked at how old Sony cameras automatically expose down, and tried that idea. The artists took the animation, roughly lit it, and played around with light and exposure. We have blinding light and then everything is exposed down massively so you can see the intense light searing through the crack of the spaceship. There are seven shots total, slow-motion shots that are almost like still frames. The only things moving are particulates slicing through the ships in the silence, breaking the Star Destroyers inside the Mega Destroyer into little shards. Animators made the shards move slowly and gracefully. Effects artists created sheets of particles that slice between shards. And lighters created massive washes of lights coming out of the cracks."
Despite the impact of these shots, the Vancouver studio's primary work focused on complex shots centered on a hangar inside the Mega Destroyer where Finn and Rose fight Captain Phasma and other stormtroopers until Vice Admiral Holdo attacks. The crew handled the destruction, shots prior to the destruction, and space shots around the sequence.
"Creatively and technically, it was our most challenging work," Seddon says. "Typically, environments tend to be in the background, so we can come up with optimizations. But the Mega Destroyer is a First Order ship that's 60 miles across. The hangar inside is so huge it could fit a regular-sized Star Destroyer. We know because we tried it."
Ultimately, having the Star Destroyer inside didn't work in the shots, but it gave them ideas about how to sell the scale of the hangar. The generalist artists started with CG concept art – models that Jenkins created during production and postproduction – adding details to those models while working in Autodesk's 3ds Max and Chaos Group's V-Ray.
"We weren't able to cheat because we would have to destroy it all," Seddon says. "So we needed a regular 3D pipeline for the effects artists and animators. And, that meant we had to have two environments in terms of lighting – the warehouse and the burning, inferno version."
The generalists created lighting designs and did architecture, set dressing, and lighting for non-destruction shots in 3ds Max and V-Ray. The assets were then passed to modelers working on the regular 3D pipeline, to rebuild and republish for shots with destruction that were ultimately lit through Katana. Effects artists created the destruction in Houdini and with ILM's Plume. They rendered the destruction in the hangar with Plume or with Side Effects' Mantra, depending on what was most efficient. Katana rendering moved through RenderMan. For the initial destruction of the Mega Destroyer, though, the team turned to Clarisse for its ability to manage the vast amount of debris.
SKI SPEEDERS SLICE INTO THE SALT CRUST, CUTTING BLOOD-RED TRAILS THROUGH THE WHITE SURFACE.
THE PORGS ARE ANIMATRONICS AND CG ANIMATRONICS.
Star Wars: The Last Jedi begins and ends in the Jedi village on that remote island off the southwest coast of Ireland. The artists in Vancouver added set extensions and animated the charming little creatures, the Porgs, that inhabit the island.
"They're often animatronics, and we didn't want to shatter that feeling," Seddon says. "We added some humor and nuanced behaviors, made them fly, and created some brothers and sisters. But, we all liked how charming the animatronics were. So, we put a lot of time into making them authentic CGI animatronics. We did a lot of takes trying to capture what Rian liked about them."
High-tech tools and some of the best visual effects artists in the world creating low-tech worlds. Worlds that the most nostalgic Star Wars fans believe are part of the same galaxy that thrilled audiences in 1977. It might look simple. But it isn't easy.
Barbara Robertson (BarbaraRR@comcast.net) is an award-winning writer and a contributing editor for CGW.