Five decades ago, when man first landed on the moon, children everywhere began dreaming of becoming astronauts, imagining what it was like to visit space. Today, their own children do not have to simply imagine; they are able to get a fairly realistic sense of what astronaut training entails, at least as earthly as possible without leaving the ground, via a VR experience called “Earthlight.”
“Earthlight” is a narrative-driven game that tells a story only a few people will ever experience – that of human space flight – as players live the unique and unusual journey of one of NASA’s talented astronauts. Indeed, there have been many space-themed games over the years. However, using the term “game” in the typical sense does not fully describe “Earthlight.”
Yes, it is an entertainment experience, created by a game developer: Opaque Space, a newly formed game company in Australia. However, it strives to be as realistic as possible – and it has science behind it. It is developed in collaboration with various space agencies, including NASA and its laboratories, to ensure the content is as authentic as possible. And, the title is created in VR, enabling the “astronaut-in-training” to not simply “play,” but “experience” the scenarios.
Parent company Opaque Media Group, founded approximately five years ago, launched the game company to explore interactive entertainment experiences, especially – but not limited to – those geared to space. In contrast, Opaque Media Group is more focused on emerging technologies to reshape entire industries through user and developer experiences. It embraces all kinds of applications and hardware relating to augmented reality, virtual reality, mixed reality, and more – whether VR projects for the film industry or education experiences, such as its award-winning “Virtual Dementia Experience,” which used early VR technology to give caregivers and families an empathic understanding of the disease.
“Earthlight” started out as a tech demo from Opaque Media Group, to demonstrate the capabilities of VR. In mid-2016, the company decided to turn it into a game with the same name. “It had so much traction with consumers and investors, that we decided to turn it into a game with a narrative and a fully fleshed-out experience with far more mechanics and functionality,” says Jennifer Scheurle, game design lead at Opaque Space.
Developers in Space
For the past year, a team within the Opaque Media Group worked to turn the tech demo – in which the user traverses the outside of the International Space Station (ISS) to perform a maintenance task – into a narrative game experience. Development was handed off to the newly formed game company in the beginning of the year.
This month, the initial fruits of their labor are ready, as “Earthlight” is released as a 15- to 20-minute game experience on Vive Arcade; it will also be available in other arcades. A longer-form version – an all-VR episodic release, in all probability – is also under way with parts of the game already built; it is expected to be ready within a year, depending on funding from investors.
The game tells the life story of Ana, as players follow her experience and journey involving research and training, even before she gets to space. To ensure the experience is as authentic and realistic as possible, the Opaque team spent a tremendous amount of time researching this world, just as Ana and the player do in the game.
“Before any basic implementation of mechanics, we had to do a huge amount of research because our game is twofold,” says Scheurle. One, it is a narrative game experience, “and we want to stay as close as possible to reality in what astronauts do in space and in their training. Sometimes, though, we had to ‘trick’ the player into learning certain things. But overall, we stay true to their experiences.”
Two, some aspects of the game are also being examined and tested at the newly built NASA Hybrid Reality and Advanced Operational Concepts Lab for researching new training methods for astronauts, using emerging technologies (see “Space Training,” page 22), as NASA explores the use of VR and MR in that process. To this end, the space agency is collaborating with Opaque and a few others in that process, with the game company sharing assets and animations, and performing some basic prototype testing. And it’s this collaboration and the need for authenticity of motion that poses the greatest degree of difficulty for the group.
The Opaque-NASA relationship started when the Opaque team visited the Johnson Space Center to obtain research insight. “The trip originally was for research purposes and getting photos of materials, such as suits and objects for the game, to make it as authentic as possible,” says Scheurle.
The parallel between the game’s entertainment aspect and its real-life value really hit home earlier this year during Opaque’s second visit to the Johnson Space Center, when Opaque Space CEO Emre Deniz experienced NASA’s Active Response Gravity Offload System (ARGOS), a bridge/crane-like apparatus designed to simulate reduced gravity. There, the group played a demo version of the game and a version they had altered for NASA that would enable astronauts to get a more accurate feel for moving around and using tools in this altered-gravity environment.
“That was our first test run of what we could possibly do in the future in terms of training,” Scheurle says.
Building Space
The game is experienced from Ana’s point of view. “Our challenge is to get a person comfortable and used to moving around in an environment they have no concept of, as opposed to a VR game in which you move in a way that is close to a human experience,” says Scheurle.
Hindering the motion further is the fact that the EVA space suit is extremely large and cumbersome, and the artists had to tweak the animations based on reference videos to incorporate the restrictions. In addition, the player has to adjust to seeing the environments and objects through a viewport in the virtual helmet.
The group is prototyping some new suits in collaboration with NASA and is building new environments to test them. As Scheurle points out, it is easier to prototype in VR than to have the actual material tested in a laboratory. The overlay on the helmet display was used for prototyping in collaboration with NASA as well.
All the assets in the game were built in-house by the Opaque artists based on photographs the team took during their visits to NASA or from other sources during their extensive research. “We’ve watched hours and hours of astronauts in training. We also watch all the NASA space walks. They are amazing reference material for how mission control speaks to astronauts on the ISS and best practices for [performing tasks], such as tethering yourself,” explains Scheurle.
Sometimes, though, photos and references do not always provide a complete picture. According to Scheurle, it is often difficult to figure out which tools are currently used on the ISS. “For instance, last week I researched the bags they take with them on space walks, and there are so many kinds that they use,” she says. A call to NASA’s Hyper Reality Lab answered that question.
Nevertheless, the need for accurate information is seemingly endless. And some of the biggest challenges the artists face result from the limited ability to access information and references. “Even after our visits to the NASA facilities, we found it quite difficult deriving information about specific assets and content. Largely, we were accessing photos, PDFs, and schematics, and following URL breadcrumbs online,” says Deniz. “We did, however, re-create some assets to degrees of fidelity that were praised by some personnel from NASA, and in other instances, like that of the NBL (Neutral Buoyancy Lab), have given us avenues for greater collaboration.”
Currently, there are two environments within the game: the ISS and the NBL underwater training area that contains a replica of the ISS. As Art Lead Stefan Allaki notes, while a lot of movement and gameplay technology can be shared between the environments, the art assets are unique to each.
Modeling of the ISS was generally done within open-source Blender, however the team began transitioning to Autodesk’s Maya as the project shifted focus from the demo to the game, making it a better solution for animation in that it is compatible with the Maya-to-Unreal Engine 4 rigging tool set. Texturing is done with Allegorithmic’s Substance; the ISS is built using complex tileable materials as well as individual assets handcrafted within Substance Painter. The artists also use the Adobe tool suite, in particular Photoshop and After Effects. Epic’s UE4 serves as “mission control.”
The ISS is the most complex asset, with many different detailed sections and materials, notes Allaki. “It’s a delicate mix between complexity and readability, with only so much information you can throw at the player before it becomes an overload,” he says. “The only way to build an asset like this is with careful planning.” In fact, there was a long pre-production phase that entailed gathering the relevant references as well as figuring out how the Canadarm (a remote-controlled mechanical arm that can move around the ISS) would operate.
OPAQUE'S REALITY-BASED SPACE IMAGERY.
“The Earth asset is also quite complex. We had to develop a new approach to re-creating the surface of the Earth to keep the fidelity high, as VR has a tendency to expose texture details,” says Allaki.
The Earth surface is the main focus while outside the ISS, and it is being used at the Hyper Reality Lab for NASA’s virtual ISS model. As a result, this imagery needed careful attention to ensure its accuracy, but also to confirm that the team was satisfied with the clarity of the surface while seen in VR. “We didn’t want it to simply look like a texture excessively stretched over a sphere, so the material is quite complex to make it look as dynamic possible,” Allaki explains.
Also, the artists created an aurora effect that was challenging to solve. “Many projects have created solutions for seeing auroras from Earth or far from Earth, but there are rarely in-game examples of auroras from that middle ground as seen from the ISS,” Allaki says. “Auroras seen from Earth are generally created as bands of light animating above, but from the ISS, they have much more shape, height, and volume.”
The view in space was created through a rather labor-intensive process, as the artists sourced 96k Earth satellite photos and then partitioned them into slices of 8k textures, which further are used in 4k sequences whereby the UV from the model of the Earth pans over it. The cloud maps are also similarly derived from satellite cloud maps, created using the cloud map plus an offset black “cloud map” to represent the shadows.
The Earth textures are also, at parts, hand/matte-painted where there were artifacts in the photographs and scanning errors.
Meanwhile, the water in the NBL training pool model is created using a combination of depth maps, caustics, a lot of overdraw, and transparency. Allaki cautions that the water is a great case study in how environmental design in VR can dictate the intensity of performance overheads. “Don’t do underwater VR without expecting large performance issues in rendering underwater scenes,” he warns.
New Frontiers
As Scheurle points out, anytime you work with new technology, solutions are always needed. “A lot of people here are from the gaming world, so we are used to working with new and strange concepts,” she says. “But when you work with VR, you have to be prepared to adapt and test. We test a lot.”
Deniz agrees, noting the group faced some challenges with the production pipeline: Because of the ever-changing and shifting nature of VR and supporting game engines, the artists found themselves having to retool and rethink their development at certain points.
GETTING A FEEL FOR REDUCED GRAVITY IN THE NASA ARGOS.
“Performance is more of a factor in VR, so having to plan the deployment of assets and also design our content around factoring player agency meant that occlusion culling was a bit more difficult –
predicting where/how/what a player looks at or interacts with in VR is already incredibly challenging, but building content around that was a new obstacle we had to get our heads around,” Deniz says. He adds that the group has gotten much better at dynamically adjusting the level of detail and also managing the player paths, as well as relying on more mature engine support and support from manufacturers such as Nvidia with its VRWorks.
“Our main goal is to bring an experience to people in their own homes that is very unique and one that only very few humans have experienced,” says Scheurle, noting that 556 people have been to space so far – out of the entire human race. “This is an amazing way to get people to experience something they would never get to do otherwise,” she says of “Earthlight.”
In addition, there is the collaboration factor that may prove beneficial to future training of astronauts in VR and MR. “We are still trying to figure it out, and part of my role is to find out where the overlaps are. There will be things we need to build here specifically for NASA and their apps that are too complex or scientific, or even boring, for a game,” says Scheurle. “The tasks might be generic and minute, but they have to be performed in a high-pressure environment, and that is what makes them so complicated.”
So while this group continues to use new technologies as they reach for the stars, they are likewise fulfilling the ultimate dream of so many children, and adults, by making virtual space travel readily available for the masses. And in the process, collaborating to develop tools to train astronauts in the future.
Karen Moltenbrey is the chief editor of CGW.