Virtual and augmented reality are making quite a comeback. Indeed, VR made an impression in the 1990s, perhaps even in spite of its role in the creepy science-fiction/horror film Lawnmower Man. In the real world, however, VR applications were limited by the available computer power (or lack thereof) and became mainly the domain of those that could afford it: military, medical, automotive, and aerospace. Often the applications played out in CAVES, with users tethered to cumbersome devices.
A few years ago, I tried out an early version of the Oculus Rift at the Game Developers Conference, and as its design evolved, so, too, did interest in the gadget. So much so, in fact, that in early 2014, Facebook purchased the company and, in doing so, motivated others to buy into the concept of virtual reality. Now, more companies are following suit: Sony’s PlayStation VR and HTC/Valve’s Vive for games, and Google’s Cardboard and Samsung’s Gear VR for smartphones, to name a few. To accompany the VR viewers/headsets, there are gloves and other devices to enhance the VR experience.
All this equipment, in addition to software for generating the VR imagery, can be used in a wide range of applications, from training (military, industry, athletics), education, entertainment, design (industrial, urban planning, architectural), therapy, and so much more.
Here, we look at the growing world of VR through the eyes of Denise Quensel, who coordinated last year’s SIGGRAPH VR Village.
Did the resurgence of VR take you by surprise?
I think because I work in academia and R&D, it came as no surprise at all that VR came back into focus so quickly. For me, it was actually a bit more like ‘Oh, thank you, finally!’ Those who have worked in this area have pushed for so long to see improvements in hardware and software that allow for excellent VR content, both in entertainment and practical application. What is surprising is how well the VR pioneers work with young people who are new to VR, that despite the gap in age, they share extremely common philosophies.
What sparked the interest again?
I think the Facebook purchase of Oculus was a huge part of the mainstream interest in VR. As soon as that happened, it was like an overnight switch was flicked that made the general population go from ‘VR is only for games and geeks’ to ‘VR is a social experience and conduit to change.’ Also, most people before 2015 had zero interaction with immersion technology beyond stereoscopic 3D in movie theaters and 3DTV. Stereo 3D may be a window rather than a screen, but VR is not the window; it is ‘presence’ to a new world. One minute inside a head-mounted display, and that becomes immediately apparent. More people are getting their hands on development kits and items like Google Cardboard now, so we’ve hit the ultimate form of affordable VR.
What is different about VR/AR now as opposed to in the 1990s?
The hardware finally works! But in all seriousness, the technology really was the main reason why VR fell out of favor in the ’90s. There is a significant amount of scholarly papers, reports, and experiments from the ’80s through now that demonstrate the possibilities of VR and show that people have known all along what VR can do. We just needed to see it work, with hardware and software that can keep up. The literature should be used by all as a resource in creating VR now; it is right there at our fingertips. Even things like simulator/VR sickness and latency issues are becoming well understood, with research pointing to a combination of hardware display issues and also motion techniques as causing problems.
Who is driving VR currently: professionals, consumers, or both?
Today, it is still driven by professionals in academia and industry. People come from entertainment, medical, journalism, anthropological, artistic, and scientific backgrounds. It is incredibly diverse. VR is unique in that it fits into any discipline of literally any industry –
so more than anything, industry is curious about what it can do. Very soon consumers will start to have requests, and we will find out which applications are starting to make the most commercial sense. Applications that make commercial sense often make sense on an emotional level for consumers, and this suits VR progress well because VR is one of the most successful mediums at provoking empathy and engagement that we have ever known.
What recent technologies have enabled the resurgence?
The most obvious technology that helped pave the way for wider adoption of VR is the HMDs. Five years ago, one cost $40,000, and now we can order them for less than $500. Also, fast high-performance GPUs that allow high frame rates (over 60 frames per second and some even at 120 fps) in stereo 3D, which is key to delivering great VR experiences from a PC.
What new technologies are needed to bring VR and AR to the next level?
Beyond HMDs and high-performance GPUs, we are seeing an increase in equipment that can capture 360-degree content. This is key for live-action capture, of course. Photogrammetry and LIDAR also are becoming increasingly affordable techniques for creating VR works. More and more people are opting to try photogrammetry or LIDAR for the capture of 3D environments, something that is going to completely change the role of a cinematographer.
What types of VR applications are prevalent today, and which will surge soon?
The remarkable thing about VR is that its application is endless. While gaming appears to have driven the interest in VR, there are some really common, everyday practical applications that have made their way into our daily lives without much hype. One of my favorite apps for a mobile device is the common star gazer app that lets you use your device in augmented reality to identify constellations. It sure beats lugging a Starball globe around – remember when we had to use one of those? This is going to be the theme of VR; it is going to mix into our everyday lives without any kind of fanfare. The medical community is starting to adopt VR in really interesting ways also, everything from training tools for doctors and nurses to solutions for chronic and acute pain.
What industry can benefit most from VR and AR?
I am really encouraged by the work being done in journalism (see Nonny de la Pena of Emblematic Group’s work) because a real human connection is being made there. People love stories, and the phenomenon that happens with VR is that the content reaches you on a whole new level. People end up really engaging with the things they experience in VR, and because they are so engaged, they end up caring about it. There are many reasons behind this that involve psychology, human cognition, and perception, as this area is extremely well researched.
Because we can become so engaged within VR, we are able to connect to one another and the world around us in ways that traditional media cannot provide. Other than journalism, I can see the communications industry really benefiting from the public embracing VR. Telepresence is being adopted in our workplaces and also in our interactions with family and friends. I am interested in seeing the creative ways in which VR telepresence is utilized over the next couple of years.
What do you see as the next step in VR and AR?
It is going to be really important to get VR in the hands of as many people as possible, and keep it as a warm, welcoming community. Sometimes I meet people who describe themselves as ‘imposters’ in the VR community, who either come from other industries than gaming or haven’t worked on a VR project yet. Truthfully, we’ve all been beginners, and the best thing for those curious in VR to do is get online, grab some tutorials, and start creating. We still don’t have a lot of best practices and workflows in VR, so there will be more curriculum in universities/colleges (and, hopefully, also primary schools) covering VR and workshops targeted at industry once people become confident in the VR creation process.
Do you see AR and VR as a trend that will fizzle soon, a la stereo 3D?
There is always going to be a lot of fuss about catchphrases and trends with any new tech, and VR won’t be any different. The key will be to not overpromise what it can deliver, and focus on what we can do with it today. That way, there are no unrealistic expectations about it being the technology that changes the world overnight. Stereo 3D was in a bubble largely dominated by television manufacturers and theater owners. That surcharge on 3D movies really impacted the public’s enjoyment of stereoscopic content, and the decision by studios to convert a 2D film to 3D as a financial decision (opposed to creative) really didn’t help.
It is no secret that these things happened; other issues existed, like excessive strobing and flicker caused by the low 24 fps standard frame rate and reductions in brightness from polarized equipment and glasses. While these issues were acknowledged, they weren’t made a priority for repairing on an industry-wide level. On one hand, VR has its own set of issues, like VR sickness, vergence-accommodation (vision) conflicts, and field-of-view limitations. Yet as a whole, the community is extremely dedicated to solving these issues and taking them very seriously.
What else can we do?
It helps for any new technology to have as easy a learning curve and as few barriers as possible. That sounds like common sense, but it is surprising how many devices, operating systems, and products have issues with incompatibility and setup. If it is too difficult to use, or if someone finds out their brand-new purchase cannot display the content they wanted to see, they are going to become disenchanted pretty quickly. This is one of the reasons there is a lot of support by developers and creators of VR around open-source software and hardware.
Is VR a stepping-stone for AR, or will both coexist?
Right now, VR and AR coexist and sometimes even together in mixed reality (MR). The first usage of this term and explanation that I know of comes from a paper authored by Paul Milgram, Haruo Takemura, et al, titled ‘Augmented Reality: A class of displays on the reality-virtuality continuum.’ At SIGGRAPH’s 2015 VR Village, we had a couple of strong projects demonstrating this: uSens’ ‘Super Reality’ Impression Pi blend of VR and AR, and Dassault Systèmes’ ‘Never Blind in VR.’ It should be noted that a lot of people, including myself, do use VR as a catchall for VR, AR, and MR.
Are there industries where AR is better suited than VR?
I don’t think AR or VR is industry-specific, but rather application-specific. Say the goal is to create immersion in a virtual environment, even a photorealistic one – the choice will be to go VR. If the real world simply needs augmenting, then AR will be the default choice. There may be certain situations where VR is impossible. For example, I wouldn’t want someone to be performing surgery on me while wearing a head-mounted display! But I can absolutely see a doctor wearing an AR device as a means to perform procedures.
Images in this Q&A are from projects in the 2015 VR village.
What are some of the more exciting developments of late?
At VR Village, we demonstrated physical, embodied VR in really large spaces. In one case, ‘C.a.p.e,’ by CREW_EricJoris, actually had attendees walking around the entire hall while inside an immersion (an experience that involves wearing an HMD and being guided by someone through an environment). The reaction of other attendees was brilliant – they’d turn around after looking at an exhibit and see someone just meandering on by, inside a head-mounted display with headphones. The ability to be so mobile in VR is really new. People are going to be able to turn their rooms into a VR environment without moving furniture away, and with minimal preamble. Right now, most people can only experience VR in conferences, meetups, and academic institutions, so it is hard to imagine what our future will look like one year from now. I do think it is going to be a very mobile future, where sitting in chairs gets banished from most VR experiences.
How will that impact the course of VR and AR?
Problems with VR motion sickness are well known and something we need to reduce. We know that motion sickness is caused to a large degree when vestibular stimuli do not match stimuli from our other senses. The vestibular system acts as motion detectors, and introducing physical movement (or even the sensation of movement) that matches the visual cues in a virtual environment greatly reduces VR sickness. Physical movement that works well with visual cues also contributes to a great sense of ‘presence,’ the proverbial holy grail of VR. So movement and the ability to be mobile doesn’t just make the experience more interesting, it also helps achieve the highly sought-after presence and part of the fix for motion sickness in VR.
Is there a certain application outside of the 2015 VR Village that especially interests you?
I am rather curious about the use of AR and VR in education, both as a learning tool and as a new language. In a conversation with Dr. Ken Perlin of NYU, he demonstrated to me how we may be able to animate, or ‘physically describe,’ an idea or concept that would have otherwise been difficult to explain with words or diagrams. It becomes as simple as using traditional symbols in a new virtual space, which works toward creating a language that hadn’t existed. I find this potential to be fascinating, along with the knowledge that we are now creating virtual wayfinding aids so we don’t get lost in our virtual environment, and that we are also creating ‘redirected walking’ when the virtual environment exceeds the real-world space we are in. These are all unique to VR, and the potential is incredible.
Karen Moltenbrey is the chief editor of Computer Graphics World.