Looking behind the scenes of your favorite TV shows and movies, you might be surprised by how many famous scenes were created on virtual sets using techniques like green screens.
However, there is a new technique that is starting to replace a lot of green screen work, and looks to be the future of virtual film and TV production: LED walls. Huge walls of LED screens are used as a background, in front of which the actors act out the scene. This increasingly popular virtual set technique could save huge amounts of time and money for TV and film companies, among other benefits.
In ABC’s recent crime thriller Big Sky, real-life scenery was filmed on location using a 360 camera, Insta360 Titan, and then displayed on the LED walls. Compared to previous projects where the background footage is composited (added) digitally, this was one of the first major network television productions to use 360 footage with an LED wall.
Here, Kent O’Connor, VFX supervisor on the Big Sky production, talks about his experience creating a virtual set for this project.
Could you briefly introduce yourself and some of your previous work/projects?
My name is Kent O’Connor and I’m a VFX supervisor. The most recent TV shows I have worked on are ABC’s Big Sky, A&E’s
Project Blue Book and Netflix’s
Altered Carbon. I also worked on features like
Jurassic World: Dominion, Sonic, Midway, and
Skyscraper.
Why did you decide to film parts of Big Sky using a virtual set?
Turnaround times in TV are short, which makes working on VFX workloads difficult. So, we decided to use a virtual set to cut down on some of our post workload. The easiest way to do that was to replace green-screen driving work with virtual set playback using an LED stage.
We used the Titan to shoot 360-degree plates (footage that is later combined with other footage) for the sequences to be displayed on an LED stage. Many people immediately think Madalorian when they hear LED stage, but ours was quite different. In the Mandalorian, the environments were built digitally, whereas ours were captured practically with the Titan. We used the Titan primarily for driving footage for scenes in vehicles.
Back on set, we parked the cast in a car and surrounded them with enormous LED screens. We used the 360 footage as backgrounds on those walls. It saved us a ton of time and money that we were then able to reallocate.
Why did you use a 360 camera to shoot the VFX plates?
We needed a full 360-degree environment because the directors wanted the ability to move freely between angles. We also needed top and reverse angles to match every background because we needed the reflections for the vehicles. Once all of the criteria were set, there weren’t many feasible solutions.
Given the image quality requirements, the alternative to the Titan was an array of nine cinema cameras pointed in different directions that were all bolted to a single steel plate. As you can imagine, we were pretty excited when we found the Titan. Having the flexibility of a single device and not nine devices paired together was hugely advantageous. By the time we finally had locations selected, we couldn’t have shot the plates with any other solution.
How did you test the camera before production to make sure the footage would be up to standard for a major network TV show?
We didn’t know much about the Titan so we needed to test the footage rigorously before we felt comfortable using it. Particularly because we were going to use the footage on very unforgiving screens that were 13 feet tall and about 60 feet long. We tested specs like color depth (the overall range and quality of the color information), resolution, distortion, motion blur and stabilization.
But we also needed the device to be flexible since locations and preferences were changing constantly. We needed the camera to be able to keep up, be user friendly, and more than anything; be reliable.
What pleased you most about the Titan? How was the final result?
I was grateful for the simplicity of the UI, but the overall quality of the footage was what really won me over. The processing power was impressive. It’s easy enough to have a still camera and capture a 360 environment but driving shots require an incredible amount of data processing and the Titan nailed it.
10-bit color depth was also crucial for us. In addition to seeing the LED walls behind the cast, the footage was used to light the entire scene so colors needed to be vibrant and accurate but also adjustable. It was also very easy to use; ridiculously intuitive. I think I read 1 page of the manual before using it.
It was a new workflow but by the time we finished using the Titan footage on the LED walls everyone knew that it was the right decision. I’m not sure if everyone realized how much time and money we saved by using the Titan, but it was a lot of both.
What are the major challenges involved in shooting a TV show with a virtual set?
When there is a new tool introduced like virtual sets there can be a misunderstanding of limitations. LED sets are only as good as the footage you feed them and the wall hardware. Most panels are not designed to be too close to camera or sharp in focus.
But most importantly, decisiveness is key to getting it right. Indecision creates delays which have a crippling ripple effect on the workflow. With green screen, decisions can be pushed until later (creating a different set of issues). But with virtual sets, decisions need to be made early enough to deliver a finished product so that everything can be ready on the day. The biggest challenge is progressing away from the ‘fix it in post’ mentality and moving towards the ‘fix it in prep’ mentality. If your team can do that, then it’s easy.
Big Sky returned to ABC on April 13.