Hybrid Drama

If necessity is the mother of invention, then adversity can be considered its father. Both are fitting descriptions of the tack producers of the NBC crime thriller The Blacklist took after filming abruptly ceased when stay-at-home orders were instituted because of COVID-19.

Unlike some television series that had already completed production and even post-production for the season, The Blacklist still had a number of episodes to go before its Season 7 finale. In fact, The Blacklist was in the midst of shooting Episode 19 of 22 when the coronavirus lockdown went into effect.

What to do? The network already had renewed the series for an eighth season, and audiences expect an edgy show, especially one as dramatic as The Blacklist, to close out the season in a thrilling, suspenseful way. Realizing that completing the planned season-ending Episode 22 was out of the question, the producers believed Episode 19 would make the second-best ending. The problem was, that episode was only halfway through filming in New York.

Nevertheless, by thinking outside the box and using CG animation to fill in the missing segments, the showrunners were able to accomplish their goal with a very unique and suspenseful episode that leaves audiences counting the days until the series returns.

This is not the first time that CGI has saved the day for films and TV series that have encountered unforeseen obstacles during the middle of shooting, mostly involving actors who have unexpectedly passed away before filming was completed (Paul Walker in Furious 7, Oliver Reed in Gladiator, Brandon Lee in The Crow, and Nancy Marchand in The Sopranos, to name a few). But, what happens when the situation adversely affects not just one person, but the entire cast - and crew? The solution was CGI. Only this time, entire sequences were animated in a comic-book/graphic novel style - including actors as well as environments - and integrated into the live-action that had already been shot.

The toon-style animation meshed well with the series’ spy drama theme.

Creative Problem-Solving

The Blacklist stars James Spader as Raymond "Red" Reddington, a former US Navy officer-turned-high-profile criminal who agrees to work for the FBI to capture (or kill) the world's most dangerous operators and terrorists on the government's "Blacklist" (in exchange for immunity) - all the while following his own secret agenda. The twist: This antihero will only work with the young agent Elizabeth Keen (Megan Boone), who may or may not be his daughter.

Season 7 Episode 19, called "The Kazanjian Brothers," finds the FBI Task Force investigating an accountant with criminal clientele, and to find him, they try to locate a pair of brothers who are hired to protect him. Meanwhile, the cat-and-mouse game between Liz and Red is coming to a head, as is Liz's relationship with her mother, the dangerous Katarina Rostova, as Liz is forced to choose sides in the deadly game between Red and Katarina.

Shooting in New York was only partially complete when the cast and crew were sent home. During brainstorming sessions, series creator Jon Bokenkamp and executive producer John Eisendrath mulled over a number of options, including a live cast reading to fill in the missing shots. Then they pondered whether a hybrid live-action/animation approach would work. After all, The Blacklist has been made into graphic novels in the past - not such a large leap given the show's pulp-fiction feel to the various blacklisters and even Red himself, who dons a signature trench coat and fedora. Eisendrath knows Ron Frankel, owner of visualization company Proof, Inc., and approached him to see if the idea was doable and to seek general advice and information due to their unfamiliarity with the medium. After a great deal of discussion, Bokenkamp and Eisendrath decided to proceed, and hired Proof to help them end the season with Episode 19, which would be fleshed out with animation by Proof, whose own staffs in Atlanta and London were under stay-at-home orders and working from home.

According to Proof producer Patrice Avery, all the characters with unfinished parts became animated, which included all of the main and featured casts. "It was pretty much, 'Here is what we've shot so far, and this is what we need.' So, sometimes we were transitioning from a close-up of Liz to an animated Liz," she explains. "It was a matter of making it work with what we had [filmed]. It was an interesting ride, that's for sure."

The episode begins with an introduction by the cast explaining the reason for the hybrid direction. Then throughout, there is an animated morph from the live-action into the CGI. And within the animation itself, comic-book chyrons are sometimes used to provide context and to emphasize subtle emotional beats.

The use of the two methods is split nearly 50/50, with approximately 22 minutes of live action and 21 minutes of animation.

"We all knew the aesthetic Jon Bokenkamp was going for and found it fairly straightforward to adapt their ideas," adds Adam Coglan, visualization supervisor at Proof in London.

Because of the comic-book/graphic novel series, the showrunners had an aesthetic in mind; they also pointed to animated films such as those by Richard Linklater (Waking Life, A Scanner Darkly). "They were fond of rotoscope-style animation, and were familiar with the work at Proof," says Matt Perrin, senior visualization supervisor at Proof in London. "That was kind of the jumping-off point for us, and we developed the look from there, creating our own toon look that was more sophisticated [than what we use for previs] and more in line with what they wanted."

The work was indeed a departure from the typical animations Proof usually does for visualization work. Yet, most of the animators there, including Perrin and Coglan, had at one time or another worked on projects for final delivery. And as both point out, although the visualization work is intended as a pre-production planning tool, the quality has grown quite sophisticated in recent years. But, it was not quite ready for this prime-time application. To this end, the Proof team in Atlanta began developing special shaders for the imagery within Autodesk's Maya.

"Because we were pitching for final delivery on a show for broadcast rather than for previews, we had to redevelop our toon shader to add more detail to the models and to get a closer likeness to the actors with hatching on the faces," says Coglan.

Proof generated CG versions of the actors with unfinished parts in the episode.

Animation Meets Live Action

Due to time restrictions - the team of 30-35 artists and TDs (20-plus in London and 10-15 in Atlanta) had just over five weeks to complete the work - there was no time to devise an entirely new process. "We had to rely on what we had done all along, and use that as a model and adapt as we went along," adds Coglan.

One immediate adaptation involved a remote workflow due to the at-home restrictions. As a result, the animators had to get used to communicating remotely through Zoom and WhatsApp. They also relied heavily on Teradici technology to log in to the Proof studio network after the OpenVPN they had been using was unable to handle the added workload. "The process went extremely well. The IT guys rebuilt our infrastructure to enable us to work from home, something we never really did before, so we could work on some other shows we had going prior to The Blacklist," says Coglan.

The team also had to ensure that the final output was void of all the shot information that appears on visualizations. "Logging and tracking that information without the HUD that we usually depend on was a new process for us," says Perrin. Additionally, they used Apple's ProRes video compression format for the final export, which is not very common in the feature-film world.

One of the biggest changes came in the workflow, as the project schedule for The Blacklist had to be greatly compressed, which led to more overlap among the departments - asset building, R&D, layout, animation. "There was no time to wait for one department to finish before we put it through to the next," explains Coglan. "So, for a long time, the producers had to look at grayscale characters that didn't look anything like their respective actors."

Because the imagery for The Blacklist was so heavily stylized, that meant redeveloping the asset pipeline, and all of the Maya-based shaders and lighting rigs the group typically used had to be revised to achieve the toon look and the hatch lines. The artists still used Maya for the animation and most of the rendering, along with Maya, Pixologic's ZBrush, and Adobe's Substance Painter (formerly from Allegorithmic) for texturing and sculpting. They also used Adobe's After Effects for the warped transitions from live-action to animation. An Adobe Photoshop filter was used to toon-ify the backgrounds.

In addition to characters, Proof also matched live-action environments.

Characters with Character

To create the characters, an asset builder at the London location pulled as many of the bodies, clothing, and other items from its generic character libraries as possible, and then built the remainder within Maya. While that was ongoing, proxy versions of the characters (grayscale stand-in models that didn't resemble the actors) were employed for the animation, to get a feel for the scenes, since "we were literally building everything from the ground up," says Perrin. There were no storyboards - the animators had to do all the staging themselves.

Later, using detailed cast information (photos, body measurements, and so forth), the modelers continued the character builds so they resembled the respective actors. The finished models were then sent to the Atlanta-based studio to be toon-ified and hatched; the hatching was often augmented with hand-drawn lines on the main characters' faces, as it's a look that requires a degree of finesse, notes Perrin.

Afterward, the models were passed back to the London team, where the client signed off on them and they were placed into the scenes.

The scenes progressed at different rates, with some in the animatics stage while others reached final animation. "And [the producers] understood that. Often they were still looking at their hero characters in grayscale, but they trusted that everything would look right at the end. They knew what to look for - shot design, story progression - with the details to be filled in near the deadline," Perrin points out.

Initially, the animators weren't going to incorporate much lip sync, but Coglan and Perrin determined they could use Proof's previs rigs to do some basic work in that regard. "The rigs were not set up to do full-on facial lip sync, but the animators were doing the lip sync as they were going along anyway, so all we had to do was switch it on in the end, which pleased the Blacklist group," says Coglan.

Meanwhile, the audio files were recorded by the actors themselves from their homes.

The artists also had to toon-ify the backgrounds. Although, the hybrid production enabled the producers to be more flexible with the planned environments. "When John and JB (Jon Bokenkamp) committed to doing the comic-book [style], they committed completely; it inspired them to get a little more exotic with some of their locations and play with things they perhaps hadn't thought about when they had the restraints of shooting [the DC-based show] in New York," says Perrin. "All of a sudden, a scene that took place in a dimly lit apartment was now at the Washington Mall, which was a big build for us." There were more crowd scenes, too, and even one sequence wherein a helicopter lands in a forest clearing as the accountant attempts his getaway.

While some scenes enabled the artists to flex their creative muscle, other scenes that had already been filmed required them to translate those physical locales into CGI, which they did with the help of blueprints, drawings, and even references in past episodes. For instance, the artists built the medical facility where Liz's grandfather lies in a coma using set drawings from the crew. "They had already filmed some shots in that location, so it was important for us to match the continuity as we transitioned to the comic-book world," says Perrin.

It's not easy to transition production on dime, but that is exactly what the actors, producers, crew, and Proof animators did when faced with the sudden shutdown. And, they were able to find a successful solution thanks to the use of computer graphics, even embracing the flexibility the medium provided in terms of action and storytelling. ¢