Stargate Studios utilized a wide range of Blackmagic Design products in creating “ThruView,” the company’s cutting-edge, proprietary real-time virtual production system.
Stargate founder and award winning visual effects creator Sam Nicholson, A.S.C., originally developed the concept of real time compositing in the early 2000’s, when production had all but halted due to 9/11. “Shows based in remote locations were suddenly unable to travel to their respective locations,” said Nicholson. “So, the question was, could we convincingly put ‘ER’ in Chicago, ‘CSI’ and ‘Ugly Betty’ in New York City and ‘Grey’s Anatomy’ in Seattle, without ever having to leave LA?”
Using clever production design, extensive plate shooting and a fair amount of green screen on set, Nicholson’s team was able to convert all of the shows the company serviced to virtual production. “We established the Stargate Virtual Backlot, shooting high resolution 360-degree backgrounds of cities all over the world,” said Nicholson. Stargate Studios’ Virtual Backlot library is now the largest Virtual Environment / Set Extension library in the world, with over 300,000 clips.
But the solution was not perfect. Using green screen on set was not ideal, with directors and actors feeling limited by working against green only, with nothing to which they can react. Directors of Photography also found it difficult to match lighting to backgrounds they couldn’t see. Add to that the increased time and cost of compositing scenes in post production, and Nicholson knew there had to be a better way. He began to ask, “Could we achieve what we had been doing in post production in real time? Could we achieve finished pixels on set?"
Nicholson’s team began to put their research into play, and leveraged it for their next production, HBO’s new series “Run.” The show required the characters to travel across the United States on a train, but production never intended to leave Toronto. “During preproduction we proved that with multiple Blackmagic Design DeckLink 8K Pro cards we could stream 10 simultaneous streams of 8K footage to the Epic Unreal game engine for real time playback on forty 4K monitors on a 150' long train set.”
Utilizing a custom tracking and lighting tool developed in house, the system was able to deliver a photorealistic moving image, displayed through the train windows, with animated lighting to match the plate, tracked to and composited with the shot in real time. “We produced the entire series with this system which we now call ‘ThruView.’”
The “ThruView” system relies on a number of new technologies that are finally developing into robust, reliable tools, both proprietary and off the shelf. “We developed targetless, wireless camera tracking at Stargate, as well as a pixel mapped, RGBW DMX lighting system. Tools such as 8K image capture are now workable with Blackmagic Design‘s 8K playback and Davinci Resolve Studio on set color correction,” said Nicholson. Affordable 4K monitors, increasingly fine grain, modular LED panels, NVidia‘s accelerated GPU graphics processors and Epic‘s Unreal 4 game engine combine with the Blackmagic Design tools to make “ThruView” possible for Stargate. “The challenge is to bring these various technologies together and make them dependable and durable enough for the pressures of multicamera live action production.”
“The Stargate ‘ThruView’ team employed three Davinci Resolve Studio systems, simultaneously on the ‘Run’ set,” said Nicholson, “We relied on multiple Davinci Resolve stations to coordinate all of our assets for playback on set. Resolve’s extensive capabilities in real time color correction, editorial tools and Fusion VFX were essential to manipulating a large quantity of high resolution footage at the rapid pace of a live action set.”
In addition to Resolve and the DeckLink 8K Pro cards, the “ThruView” system also incorporates a variety of Blackmagic Design products as part of the system backbone. This includes multiple Smart Videohub 12G routers, DaVinci Resolve Micro Panels, UltraStudio 4K Extreme, a Teranex AV standards converter, an array of Micro and Mini Converters, as well as an ATEM Constellation 8K switcher, which was instrumental in handling the high resolution images.
Nicholson relied on Blackmagic Design because he knew he could depend on the tools. “Blackmagic products are very dependable. They work right out of the box and continue working. It‘s a rough working environment on set, so the tools you are using have to be bulletproof to survive.”
“Real Time Virtual Production, which uses a combination of photographic and computer-generated assets, will become the new norm as processors get faster, memory deeper and large scale LED displays become more affordable and more durable,” said Nicholson. ”Virtual production offers a practical solution to the challenging production situation we are currently facing. It is much easier to bring the location to a production than move an entire production to remote locations. That said, it is important to remember that virtual production is not for every production. It requires a much more diligent preproduction process to acquire your creative assets and test the specific technologies you will be using in principal photography.”
But despite the caution, Nicholson feels strongly that the industry is headed firmly in the direction of virtual production, and that “ThruView” is positioned to answer that call. “Ultimately, I do feel it will replace approximately 50% of global green screen shooting in the next few years. Our system is ideal because it works and, thanks to Blackmagic Design’s components, it can be offered at a reasonable price. Because no matter how good a piece of hardware is, it’s not good if you can’t afford it.”