A Fight to the Death

By Karen Moltenbrey

"When you play the game of thrones, you win or you die." Those prophetic words spoken by Cersei Lannister to Ned Stark in Season 1 sum up the entirety of Season 8 of HBO's cult series Game of Thrones. During this final season, characters fight to the death to rule the Seven Kingdoms and to sit atop the coveted Iron Throne, while several visual effects facilities also waged battles and created the chaos, destruction, and death that ensues.

Fans waited anxiously over a year and a half for the eighth and final season of the series, although filming for it began right after Season 7 aired. The visual effects prep started even before that, with no pause in the action for this department. "Still, it was a mad dash the entire time right up to the end," says Joe Bauer, visual effects supervisor on the series, who works closely with VFX producer Steve Kullback.

The final season contained more than 3,000 VFX shots, with nearly half of those (1,400-plus) in Episode 3, "The Long Night" (the Battle of Winterfell), and over 900 in Episode 5, "The Bells" (the showdown at King's Landing). The work was shared among a dozen vendors, with the lion's share going to Weta, Scanline, El Ranchito, Pixomondo, and Image Engine. To illustrate just how daunting the number of effects were, in comparison, the entirety of Season 3 contained 800 VFX shots that spanned 10 episodes. By Season 7, the number hovered close to 2,000 across seven episodes.

Due to this enormous workload, Bauer handled Episodes 3, 5, and 6, while Stefan Fangmeier (who assisted on sequences in Season 7) served as VFX supervisor for Episodes 1, 2, and 4. "The number just got so big that we had to divide and conquer," Bauer says. The VFX facilities did likewise, sometimes with multiple vendors working on the same shots and sequences.

Yet, it's far more than simply a numbers game. The effects shots this season were complex and spanned the gamut from CG creatures and characters, to digital armies, to set extensions, a wide range of simulations, and more.

"The Long Night"

GOT has had plenty of experience orchestrating big battles - for instance, the Battle of the Bastards. But for the Battle of Winterfell, the directors and crew took things to a whole new level. The buildup is intense, and once the fighting starts, it doesn't let up until the Night King falls. And through it all, the visual effects are as intense as the performance by the actors (after famously enduring 11 weeks of night shoots in the icy rain and mud outside Belfast) when the combined forces of the kingdom (minus those of Cersei and Euron) face the Night King and his Army of the Dead. It's a showdown viewers have waited years for! The mighty Doth-raki warriors, Unsullied armies, and many beloved knights head toward the foe. A storm rolls across the battlefield, obstructing the view, then seconds of deadly silence before only a handful of battered heroes retreat back to the castle. Soon the wights attack, held back temporarily by a trench of fire. Meanwhile, Jon Snow and Daenerys launch an aerial attack on the back of Dany's two remaining dragons, Rhaegal and Drogon, but the Night King has a dragon of his own, the resurrected Viserion. The battle rages on the ground and in the air, through a fog of war that heightens the action.

HALF OF WETA’S SHOTS IN THE FINAL SEASON WERE FOR THE BATTLE OF WINTERFELL.

Half of Weta's approximately 600 shots in the season occurred during the Battle of Winterfell, during which Weta handled the ground attack, while Image Engine handled the episode's dragon animation during the aerial battle (and throughout most of the season). El Ranchito in Madrid also picked up numerous shots in Episode 3.

"It is a long, sustained battle," Bauer notes.

The Fire Is Lit

In this crucial episode, Weta created the armies and the vast terrain, extending the castle, and conveyed the changing balance of the battle through the storm, along with fire and destruction. The studio created 30,000 undead wights and thousands of Dothraki with flaming swords simulated using Massive. Dynamic Massive sims also generated the forces that tossed the wights into the air when struck by dragon glass and fire.

WETA CREATED THE ARMIES, TERRAIN, AND MORE IN THE BIG BATTLE.

Weta - which handled another recent battle, in Avengers: Endgame - helps Melisandre ignite the Dothraki swords near the start of the battle. "That was an interesting challenge because there's a combination of full-CG shots, which are Massive simulations of horses and Dothraki, and some plate shots and combinations to ground the action in reality," says Martin Hill, VFX supervisor at Weta. "The plate shots contained little if any fire on the swords due to the impracticality of shooting that amount of fire with horses on set."

For the full-CG shots where the sword placement is known, Weta lit them using 3D fluid simulations. But for the plate shots, the artists leveraged a tool called Eddy, which was written in-house and works within Foundry's Nuke. "When the big wave of fire moves across the Dothraki, some of the swords are LEDs. We stabilized the plates and all the swords with optical flow, then essentially painted on all the timings of how and when the swords light up on a single frame," explains Hill. "Then we used that in Eddy, the fluid solver, by projecting it onto a depth map, getting the full 3D position, and using that light source as a fuel source for the fire simulation."

The traditional approach would have been to track all the swords and then track a fire element onto them or do a pre-canned simulation and add that on, or track all the swords and run a full-3D sim. Either way, the process would have been slow and prohibitive. "It essentially meant we could do the fire sim in the comp. We didn't have to go all the way back to effects to simulate and render again," Hill says.

Meanwhile, the big wave of fire that sweeps across the army was full CG placed atop prop swords.

One of Hill's favorite scenes was when the Dothraki ride out and attack the wights, and they get annihilated, as evidenced by their torches becoming extinguished. The action is shrouded in a thick cloud that rolls across the battlefield, announcing the arrival of the Night King - but even though the action was obscured, Weta nevertheless did a full Massive simulation on the wights and the Dothrakis falling off their horses. Weta devised a new approach for handling the large-scale volumetrics and destruction used to generate the storm on the battlefield, which still allowed for artistic control. In addition to hero volumetric simulations, the artists also extensively used Eddy to simulate and render 3D weather elements during the composite.

The ebb and flow of the battle was then conveyed through the intensity of the graphically-driven storm. At one point, the storm is almost like a giant wave, with tendrils, fingers, coming toward you, says Hill, achieved using a combination of guide curves within the simulation while also simulating all the physics of the revolving storm.

"We got some shots where Dany is riding Drogon, under the wave of the fingers that are coming over, and when Drogon flames, the fire uplights the dragon and casts a shadow on the underside of the storm wave," he adds. "So you have this giant dragon shadow following the dragon. It made for quite an impressive graphic of the dragon looming over the crowd." For this, Weta used its proprietary Deep Compositing tools to compute the light path through shadowing objects, enabling them to cast accurate shadows on the misty atmosphere of the dragons.

Massive also played a big role in the swarming and climbing of the wights as the battle raged on; it also helped disintegrate them. One particularly difficult crowd scene involved more than 500 wights crawling over and attacking Drogon when he is on the ground, injured - a simulation by Image Engine. "The wights are reacting to each other, but they're also reacting to the moving surface of the dragon because he is performing as well. It was a lot of technology thrown together to achieve that," says Bauer. "If you were trying to keyframe that, you'd never get through it."

For that animation, the studio used Sim Cycles, developed at Image Engine by Jason Snyman, animation supervisor, whereby a model retains its shape and maintains an animation cycle, but also dynamically reacts to another moving model as they touch. "The results turned out to be pretty cool and believable. You get really distinct interaction between the characters that I didn't quite expect," he says. "The characters are reacting to each other as they collide, and the neighbor will then react to being bumped. The whole thing works together as this massive system."

A month after Snyman's initial test of Sim Cycles, Image Engine artists were using it on 13 shots of Drogon being attacked, with the dynamics passed from Drogon's movements to the swarming wights.

Aerial Fight

While Weta was handling the fight on the ground, Image Engine animated the dragons and their riders (Weta lit, did the FX, and comp'd them into the scenes). According to Bauer, some of the biggest effects shots in Season 8 involve the dragon fight as Dany and Jon take to the sky pursued by the Night King and his dragon, Viserion, and then as Jon and Viserion face off in the courtyard. "There were so many pieces to it. The dragon fight was 100 percent an animation tour de force," he says. "The difficulty, too, was that it was taking place in the middle of a dense storm, at night. We had to show enough of these creatures, which are 200 feet long, so audiences could at least understand what was going on."

Similar to the ground fight, Jon and Dany's aerial battle is partially obscured by clouds generated in SideFX's Houdini by Image Engine. "We had to lay out this entire aerial combat zone in actual scale so the dragons had perfect continuity from shot to shot, and as the dragons passed through a cloud, that cloud would become live and react to their movements," says Thomas Schelesny, VFX supervisor at Image Engine.

Creating a lengthy sequence with just one person atop a dragon is challenging enough, but when you have two or three riders that are plate elements and computer-generated dragons, it's another thing entirely, says Schelesny, adding, "it becomes exponentially more difficult to make that work as a cohesive performance."

The Third Floor previs'd the face-off among the dragons, while Image Engine pre-animated the fight, then The Third Floor solved the pre-animation coordinates for the motion-control photography of the actors. Finally, Image Engine solved the actor plates back onto the dragon performance during the final animation of the sequence. 

The aerial dragon fight, crafted by Image Engine as one massive animation, has the beasts clawing, biting, colliding. In all, the animation spanned 120 seconds, or thousands of frames.

"Later we added the cameras because we needed perfect continuity from shot to shot, not just in their movements, but where they were spatially. On this show, we never cheated where we were in 3D space," says Schelesny.

Snyman's pre-animation served as a guide for placement at specific points, "but everything around there was kind of free game," he says. "It had to work at all angles, because as soon as we had a camera following Jon, we might see a glimpse of the Night King in the background, or a wing, or a little claw, or a tail flick." As Snyman notes, the team maintained the choreography as well as all the camera work in the actual space of Winterfell, making sure the dragons maintained a realistic speed through the environment, as many VFX would be affected.

The decision to animate the dragons for one continuous piece and doing so in real space required more work upfront, though in the end, the continuity of the animation was correct and accurate, and didn't have to be fixed. "If you assign 10 shots to 10 different artists, you'll get a different style or feel for each shot, so this  saves you from fixing them," says Snyman. "You have that continuity right out of the gate."

Dragons & Dragon Fire

Weta and Image Engine were responsible for destroying two of Dany's Dragons. The first death occurs when Viserion corners Jon in the courtyard in Episode 3, with Image Engine animating the dragon and Weta disintegrating him.

Digital artists also helped Rhaegal once again become airborne, only to be killed off in Episode 4. Injured, after being torn up by Viserion - wounds inflicted by Weta - Rhaegal is finding its wings once again thanks to Image Engine, which animated, lit, and comp'd the shots as the dragon begins to fly again. Then, shockingly, the beast is taken out by enormous bolts fired from Euron Greyjoy's fleet in the harbor outside King's Landing, and Weta picks up the action. Weta developed concepts for Rhaegal's wings and quills using a mix of fresh and healing tears.

"There's a shot where the dragon raises its wing and you can see the subsurface scattering from the backlighting, to see just how torn the underside of the wing is," says Hill. "Then, in that moment, the arrow hits, and it's a shock - and brutal, as well."

In Season 7, Drogon gets hit by a Scorpion arrow, but it doesn't kill him. But for this scene, it had to be apparent that Rhaegal has been taken out. Three arrows find their mark, but instead of three different shots, Weta combined them into one majestic orbiting camera move from the point of impact, past the dragon's head, all the way down to the point of view of Dany. "The end of it is from her perspective," Hill says. Then we see the impact of the other arrows tearing through his chest and neck, and we see the beast's face, before coming around to Dany as his wings fold up and he crashes into the water. The dragon's finality is punctuated with a last gasp of breath as blood sprays out from its mouth.

"The Bells"

After the defeat of the Night King, the victors from the North head toward King's Landing in Episode 4, where they are met by  the evil Euron Greyjoy and his Iron Fleet, and that's when Rhaegal is killed. Dany is reeling after losing her "child" as well as her best friend, Missandei, at the hands of Cersei. She starts to become unhinged. Jon and Dany meet up at Dragonstone, where Dany accuses her advisor Varys of treason for supporting Jon's right to the throne. Furious, Dany orders Varys' death, and issues the command to Drogon to burn him to death - work done by Pixomondo.

With Euron's Iron Fleet lying in wait in the harbor to King's Landing, the second battle ensues. Weta's work was in the harbor, simulating all the rigging, sails, and flags on the boats so layout and animation could reposition the vessels for each shot with all the dynamics on the boats working correctly. The water in the scene was CG, as was the sky and the backgrounds. Then, Weta destroyed it all via Drogon, amping up the beast's fire power to reflect its rage for the death of its brother as the digital artists generated both fire and water simulations and interactions. The studio developed effects technology that enabled the artists to combine fire and water simulation and rendering for CG and plate-element Drogon fire with the ocean water and explosions.

MANY SHOTS AND SEQUENCES WERE SHARED AMONG STUDIOS. HERE, IMAGE ENGINE HELPS RHAEGAL FLY BUT WETA TAKES HIM DOWN.

The fight moves on to King's Landing, where Cersei observes the melee from her perch in the Red Keep.

"The Iron Throne"

Despite the ringing of the bells in Episode 5 that signify the surrender of the citizens in King's Landing, Dany is undeterred from raining down death and destruction throughout the city from atop her remaining dragon, Drogon, who was animated again by Image Engine. The city itself was a 3D model based on a photogrammetric scan of the physical locale that was then built out to be much larger.

KINGS LANDING, A 3D MODEL BASED ON A PHOTOGRAMMETRIC SCAN OF THE PHYSICAL LOCALE AND THEN EXTENDED, IS DESTROYED.

The city burns and buildings are destroyed by Scanline. Also, Pixomondo created the destruction and debris that falls around Jaime Lannister and Cersei on their way to their reunion. Using Houdini, Pixomondo simulated falling debris as the ceilings and stonework crumble, referencing special effects debris on set. Both Pixomondo and Scanline shared segments of the sequence wherein the Hound and Mountain face off in the crumbling tower.

Just prior to the fight, Jaime Lannister finally reaches his beloved sister and they attempt to escape but become trapped in the Skull Room by falling debris, and die in each other's arms - construction and destruction levied by Weta. "We did all the set extensions for the Skull Room but had to rearrange the room for that shot [of the room collapse]. You can see over their shoulders quite deep into the room, with all the arches," Hill describes. "And then you see the collapse of the crypt happening farther away, the percussion effects of each of the arches collapsing and getting closer and closer."

FIRE IS OFTEN PRACTICAL IN THE SERIES, FROM A MOTION-CONTROLLED FLAMETHROWER.

Meltdown

Dany and Jon meet up in the Throne Room in the final episode. They embrace, and during a passionate kiss, he stabs her to death. Drogon enters, sees the Mother of Dragons dead with Jon beside her. Jon expects the same fate as Varys, but instead the dragon directs its rage at the Iron Throne, and melts it to the ground before snatching up Dany in its massive claws and carrying her away.

Pixomondo was involved with this scene early on, creating previs based on the storyboards, as well as completing pre-animation and preparation for the shoot, directed by showrunners David Benioff and DB Weiss. "[The scene] was the best-kept secret of all the sequences," says Sven Martin, VFX supervisor at Pixomondo, which began working on the sequence two years ago. "We laid out the whole sequence with the dragon and Jon and Dany so the directors, in this case, the showrunners, could have it for timing purposes; it is a slow-paced sequence but a very special one. On set, the camera was tracked live in simulcam so the camera operator and the directors could see the dragon pre-animated, as opposed to just a tennis ball on a stick, and reposition the camera and better frame the dragon." The pre-animation was a single, two-minute piece.

"I think Pixomondo completely nailed their work in Episode 6," says Bauer.

Scanline participated in the meltdown when the dragon was absent. When it is present, Pixomondo handled the work. The fire, however, was not CGI, but rather motion-controlled  flamethrower footage, as is most of the dragon fire in the series.

"The challenge was creating a subtleness to the animation, transporting emotions and [Drogon's] thoughts without making him look like a human being. It's a fine line making the audience understand what he is thinking but not making him a cartoony character or humanizing him," says Martin.

Another challenge was accommodating Drogon's enormous size for the scene. "You have to squeeze him in but also get a good pose, especially when he is nudging Dany after she is killed and having all this facial performance in the close-up," explains Martin. "It's all about emotional moments, which is very hard to translate on a CG character, especially when it is meant to be a real animal, more or less." To locate the dragon's performance in its real scale within the set, the crew used simulcam once again.

Insofar as Drogon snatching up Dany, while the scene may not look complicated, it required a lot of preproduction and design. "Drogon has no hands. He has these huge claws on his feet, and he had to stand on one leg and carefully and gently slide the other foot underneath her, lifting her up and tucking her safely in his foot," Martin says. "We did a lot of different poses before coming up with the right one that's very slow and a calm moment." Dany, meanwhile, was outfitted with a special rig and harness on a long gimbal arm that lifted and rolled her to the side, referencing the animation Pixomondo prepared prior to the shoot.

End of an Era

 The VFX supervisors and studios were tasked with some difficult shots throughout this final season. While Weta's Hill notes there were so many distinct one-off shots that were really complicated in their own right, he believes the biggest challenge was in the diversity of the work. "We were combining live-action elements with CG elements, and there's set extensions and big, epic simulation shots. There's just so much complexity across the board," he says.

For Pixomondo, it was definitely an end of an era. "It was really weird seeing the last episode on screen, and I was thinking, 'OK, this is really it now.' It's strange not to have a discussion with Joe [Bauer] and Steve [Kullback] about the next season. But after eight years, I am very thankful to have been a part of this," says Pixomondo's Martin.

While the number and complexity of effects on GOT continued to increase each season, Image Engine's workload jumped by a factor of four between the last two seasons alone. "[The effects] had to be better, but we also had to do more," says Schelesny, as the dragons commanded larger roles. And like the other studios, Image Engine did not want to disappoint audiences in this final season of the cultural phenomenon known as Game of Thrones, but rather exceed all expectations - and even deliver some surprises.

"We knew right from the start that we might be working on something unique and special that may not be repeated by another series for a very, very long time," he adds.

So, who won the game of thrones? Bran Stark is crowned King of the Six Kingdoms, as his sister Sansa reigns as Queen of the North, keeping it an independent kingdom. In reality, the big winners were the viewers, thanks in no small part to the armies of VFX artists who helped bring this fantasy adventure to life for over eight years.

WITH EACH SEASON, THE DRAGON MODELS BECAME INCREASINGLY MORE COMPLEX.