Nvidia is making VR more immersive, and VR development easier, with the release of VR audio and 360-degree video stitching as part of its VRWorks software development kit.
Released at the GPU Technology Conference, kicking off May 8 at the San Jose Convention Center, the VRWorks Audio SDK provides real-time raytracing of audio in virtual environments, and is supported in Epic’s Unreal Engine 4. The VRWorks 360 Video SDK addresses the complex challenge of real-time video stitching.
VRWorks Audio for Physically Accurate Audio
Traditional VR audio provides an accurate 3D position of the audio source within a virtual environment. However, sound in the real world reflects more than just the location of the source — it’s also a function of the dimensions and material properties of the physical environment.
Nvidia VRWorks Audio helps create a truly immersive environment by modeling sound propagation phenomena, such as reflection, refraction and diffraction. And it does so in real time, thanks to the massive processing power of GPUs.
Using Nvidia OptiX raytracing technology, VRWorks Audio traces the path of sound in real time, delivering physically accurate audio that reflects the size, shape and material properties of the virtual environment.
It’s the only hardware-accelerated and path-traced audio solution that creates a complete acoustic image of the environment in real time without requiring any “pre-baked” knowledge of the scene. As the scene is loaded by the application, the acoustic model is built and updated on the fly. And audio effect filters are generated and applied on the sound source waveforms.
The software release consists of a set of C-APIs for integration into any engine or application, and an integration for Epic’s Unreal Engine 4 that’s now available on GitHub.
VRWorks 360 Video for Real-Time Stitching
Also at GTC, Nvidia is releasing the first public beta of the VRWorks 360 Video SDK, which enables real-time 4K 360 video capture, stitching and streaming.
In addition, Nvidia is demo’ing an upcoming release of the VRWorks 360 Video SDK that enables real-time stitching in stereo. Using two Quadro P6000 GPUs, the SDK can stitch eight 4K cameras in stereo as demonstrated with Z CAM’s V1 PRO VR camera system.
“The fact that Nvidia manages to stitch 4K 360 stereoscopic video in real time, making live-streaming possible, changes the production pipeline and enables entirely new use cases in VR,” said Kinson Loo, CEO of Z CAM.
The Nvidia VRWorks 360 Video and Audio SDKs are available for all developers to download from developer.nvidia.com/vr. VRWorks 360 Video SDK for mono is available now as a public beta. VRWorks 360 Video SDK for stereo will be released soon.