Virtual Sets for All

I also remember seeing virtual sets for the first time (in the mid-1990s), and they were pretty amazing, too. The demos always made everything look so easy, but lighting was just as important, if not more so, than on a real soundstage. And the end results tended to look fake, because the actors were usually overlaid in some sort of early era, computer-generated, Alice in Wonderland-type background that just didn’t look natural.

Keying has been around even longer, of course (since the mid-’60s, though it wasn’t much in use then), and keying effects have been an increasingly regular feature in editing systems for the last dozen years. For keying to work best, you typically need to start with a solid, evenly lit, colored background, usually green or blue. That color, or color range (which can’t appear on the foreground talent or their clothing), is used to create the alpha channel for keying the background screen to the image. If that color range is too wide, it gets extremely difficult and time-consuming to minimize keying errors in the foreground actor or objects and on the edges of the key.

Computer animation and the ability to create realistic-looking sets and locations have come a long way over the last decade. Still, keyed effects, and especially virtual set material, aren’t common in many modest-budget videos, because good, natural-looking keying is hard to do. Shooting at a real location is usually easier and less expensive.

Ultra 2, a major new version of a virtual set and keying application from Serious Magic, may address some of these problems for smaller companies that haven’t been able to use virtual sets before. To begin with, Ultra 2 costs only $495, compared to the thousands of dollars you need to spend for traditional professional keying tools. With this product, Serious Magic hopes to make virtual set keying something that any videographer can use on any shoot, which means it’s also designed to be easy to use and of good quality, as well as affordable.


The secret is in how Ultra establishes the key. Instead of employing the traditional approach of keying off only a specific, evenly lit color, Ultra analyzes a digital image of an empty virtual set, complete with any shadows, wrinkles, or unevenness, before you start filming the on-screen talent. (If you can’t get a clean-slate image before shooting, Ultra allows you to create one in postproduction by selecting a series of points-such as a well-lit area, and one with shadows-on the key background to build an effective clean slate after the fact.)

From there, Ultra builds a mathematical model of the virtual set using a technology Serious Magic calls Vector Keying. It writes equations for how the different areas of the background relate to each other. When an actor steps in front of the virtual set, Ultra can use a variety of mathematical techniques, including pixel substitution, color vector differences, edge recognition, and traditional keying, to build an alpha channel. The Ultra interface does include a variety of traditional keying tools like color-range sliders, edge softening, and cropping, but the idea is that most users should not have to use them.

In Ultra 2, which began shipping recently, Serious Magic introduces two clever features; the first of which is support for high-definition content. You might not think there’s anything particularly special about this, because HD has become an obligatory part of any content creation tool. But in addition to supporting traditional HD content from HD cameras, Ultra can actually create true HD footage from an SD camcorder.

This sounds like digital trickery or “upconverting,” but it isn’t. Serious Magic lets you simply turn a 720x480 camcorder sideways 90 degrees and shoot in 480x720 portrait mode, thus yielding a full 720 vertical lines of your on-camera foreground talent. That can be matched to any size high-resolution computer-generated or video background. The Ultra 2 software automatically compensates for the rectangular pixels by determining the native aspect ratio of the source and adjusting horizontally.

Ultra 2 also leverages the virtual lighting technology from typical 3D creation tools to draw virtual shadows on virtual sets from real on-camera subjects. By simply choosing a light angle, you can make your keyed-in actors blend more naturally with the virtual set. Even more remarkably, Ultra 2 can retain real shadows created by real actors standing on or in front of a greenscreen background and, thanks to Vector Keying, can map those real shadows onto the generated virtual set.

Perhaps the most interesting aspect of Ultra is that by enabling a wider use of virtual sets, Serious Magic is opening the doors for digital artists and animators to become the new virtual soundstage designers for video makers.